[FSSDK-11571] Python: Add holdout support and refactor decision logic in DefaultDecisionService

[esrakartalOpt]

Summary

• Added holdout support for decision service

Test plan

PR checks

Issues

• FSSDK-11571

[Mat001]

I think variation here needs to have variation assigned.

holdout_decision: Decision = Decision(
experiment=None, # check if this should be None or holdout
variation=variation, # Should contain the bucketed variation !!
source=enums.DecisionSources.HOLDOUT,
cmab_uuid=None,
holdout=holdout
)

Javascript has this:

if (bucketResult.result) {
const variation = configObj.variationIdMap[bucketResult.result];
if (variation) {
return {
result: {
experiment: holdout, // Holdout as experiment
variation: variation, // Variation set
decisionSource: DECISION_SOURCES.HOLDOUT
},
reasons: decideReasons
};
}
}

[Mat001]

AI says this:

Location: optimizely/decision_service.py:79-86

Issue: The Python implementation uses a different structure than JavaScript/Swift:

• Python: Stores holdout in separate holdout field, experiment=None, variation=None
• JavaScript: Stores holdout AS the experiment field, with variation properly set
• Swift: Returns variation directly, holdout passed as experiment in FeatureDecision

Current Python:
class Decision(NamedTuple):
experiment: Optional[entities.Experiment]
variation: Optional[entities.Variation]
source: Optional[str]
cmab_uuid: Optional[str]
holdout: Optional[Dict[str, str]] = None # ❌ Separate field

JavaScript (reference):
{
experiment: holdout, // ✅ Holdout stored as experiment
variation: variation, // ✅ Variation set
decisionSource: DECISION_SOURCES.HOLDOUT
}

Recommendation: Either:

1. Store holdout in the experiment field (like JavaScript)
2. Or ensure variation is properly set and document this design difference

[Mat001]

Hmm, check decision logic here.

Suspicious Logic in get_decision_for_flag

if (decision.experiment is None and decision.variation is None and decision.holdout is
None):
continue

Problem: This checks if holdout is None, but since holdout is passed as a parameter to
get_variation_for_holdout, the holdout field in the returned Decision will never be
None when processing holdouts. This logic doesn't make sense.

Expected: Should check if user was actually bucketed:
if decision.variation is None: # User not bucketed into holdout
continue

---

✅ CORRECT IMPLEMENTATIONS (for reference)

JavaScript SDK:
if (bucketResult.result) {
const variation = configObj.variationIdMap[bucketResult.result];
if (variation) {
return {
result: {
experiment: holdout, // Holdout as experiment
variation: variation, // Variation set
decisionSource: DECISION_SOURCES.HOLDOUT
},
reasons: decideReasons
};
}
}

Swift SDK:
if let variation = bucketedVariation {
let featureDecision = FeatureDecision(
experiment: holdout, // Holdout as experiment
variation: variation, // Variation set
source: Constants.DecisionSource.holdout.rawValue
)
return DecisionResponse(result: featureDecision, reasons: reasons)
}