Synathra.Sport Ablation Note v0.1
Event-only baseline versus full hybrid_score_v0_1 across five football validation groups.
The note answers whether the hybrid event-chain layer adds measurable ranking value beyond calibrated event risk alone.
Main result: the full frozen hybrid_score_v0_1 improves the event-only baseline in all five validation groups. Mean top-10 lift increases from 2.57x to 4.79x. Mean ROC-AUC increases from 0.637 to 0.742.
1. Purpose
The public v0.5 report showed that a frozen hybrid score transfers across men's international tournaments, club football, and women's football. The remaining reviewer question was whether the full score adds value beyond a simple event-risk ranking. This ablation note compares an event-only baseline with the full hybrid score on the same target and the same five validation groups.
2. Compared scores
Event-only baseline
The baseline ranks events by event_risk_calibrated only. In the v0.5 exported event-level tables, the operational column used is synathra_calibrated_score, because the public scoring script falls back to this column when a literal event_risk_calibrated column is absent.
Full hybrid score
The full model ranks events by the frozen public score hybrid_score_v0_1. It combines event risk with chain membership, local chain-delta context, and subtype reliability.
The validation target is future_goal_h10: a positive case means that a goal occurs within the next 10 event steps after the evaluated event. Top-10 lift compares the positive rate in the top decile by score with the base rate of the corresponding validation group.
3. Group-by-group results
| Validation group | Domain type | Rows | Positives | Base rate | Event-only lift | Full hybrid lift | Lift gain | Event-only AUC | Full AUC | AUC gain |
|---|---|---|---|---|---|---|---|---|---|---|
| FIFA World Cup 2018 | same_family_base | 71 838 | 590 | 0.82% | 2.51x | 4.66x | +2.15x | 0.637 | 0.742 | +0.105 |
| FIFA World Cup 2022 | same_family_cross_season | 234 637 | 1 808 | 0.77% | 2.59x | 5.01x | +2.41x | 0.632 | 0.748 | +0.116 |
| UEFA Euro 2024 | cross_competition_mens_international | 187 924 | 1 140 | 0.61% | 2.84x | 4.77x | +1.93x | 0.670 | 0.756 | +0.086 |
| Ligue 1 2021/2022 | club_transfer | 101 766 | 810 | 0.80% | 2.58x | 4.69x | +2.11x | 0.626 | 0.734 | +0.108 |
| Women's World Cup 2023 | gender_transfer | 226 118 | 1 664 | 0.74% | 2.33x | 4.84x | +2.52x | 0.618 | 0.729 | +0.111 |
4. Interpretation
The full hybrid_score_v0_1 improves the event-only baseline on top-10 lift in every validation group. The mean lift increase is +2.22x, or approximately 87.4% relative to the event-only baseline. This supports the product claim that Synathra.Sport is not merely a calibrated event-risk ranking: the chain-context and subtype-reliability terms add measurable value for identifying the highest-risk event windows.
ROC-AUC also improves in every group. The mean AUC gain is +0.105. This global ranking improvement is smaller than the top-decile lift gain, which is expected because the product use case is concentrated in the highest-ranked event windows used for reports, chain cards, and analyst review.
5. Product conclusion
The ablation result strengthens the public v0.5 validation package. The hybrid layer is empirically useful: it preserves transfer across domains while improving ranking quality over event-risk alone. For media, academy, and small-club use cases, the practical implication is that Synathra.Sport can prioritize a smaller set of interpretable high-signal sequences rather than only ranking isolated events.
6. Caveat
This note uses synathra_calibrated_score as the event-only baseline because the v0.5 event-level exports do not contain a literal event_risk_calibrated column. This follows the fallback order used by the public compute_hybrid_score_v0_1.py script. If a future pipeline exports a literal event_risk_calibrated column, the ablation should be re-run with that exact column name for strict nomenclature alignment.
7. Data files
The computation draft is also available as Markdown: ABLATION_NOTE_v0_1_COMPUTATION_DRAFT.md.