Voidly
Sentinel forecast — calibration status

Does the uncertainty hold?

Check observed coverage against the model’s uncertainty. Calibration does not establish shutdown-onset skill.

Empirical coverage and rolling accuracy are withheld until archived issue times, complete target windows, and independent outcomes are verified.

What features the model actually uses

Sklearn feature_importances_ on the underlying XGBoost. 39 features total. Top-3 sum: 0.327 · Top-5: 0.427 · Top-10: 0.582. Healthy distribution — no single feature dominates the model.

  • 1.block_rate_roll30_mean17.0%
  • 2.recent_shutdown9.1%
  • 3.critical_incident_7d6.7%
  • 4.month6.1%
  • 5.block_rate_roll14_mean3.9%
  • 6.week_of_year3.8%
  • 7.election_in_7days3.7%
  • 8.blocked_count_roll7_mean2.7%
  • 9.gdelt_unrest_30d2.7%
  • 10.high_urgency_signals_7d2.5%
  • 11.incident_count_7d2.5%
  • 12.high_importance_event2.5%

Interpretation: The forecast model's top feature is gdelt_unrest_30d (0.25) — protest + conflict signals from the GDELT 1.0 global news feed. recent_shutdown, block_rate rolling means, and incident counts follow. risk_tier — the leaky country-level encoding that dominated our older classifier at 85% — contributes only ~2% here. Healthy distribution; no single feature dominates.

Raw JSON: /v1/sentinel/feature-importance

About this view

Earlier Sentinel responses included source-labeled interval fields. Empirical coverage is withheld here until archived issue times, complete target windows, and independent outcomes are verified. Source history lives at /v1/sentinel/calibration/history. A response timestamp alone does not validate interval coverage.

Model history & calibration scope
May 2026 refit: On 2026-05-20 we refit isotonic regression on 810 live (predicted, observed) pairs. The base XGBoost was underestimating risk by ~15× in the dominant prediction range.Brier: 0.5904 → 0.2231 · Calibration MAE: 0.6040 → 0.0000 · Iran 7-day risk: 0.146 → 0.74These are historical self-reported refit figures, not a validated current scorecard. Read the full refit writeup →
Calibration ≠ onset skill. Calibration measures whether predicted probabilities have the right magnitude. It does not show the forecast can predict a new shutdown before it starts — a separate audit found it cannot. The May 2026 onset audit found that the legacy seven-day forecast largely followed the current regime: forward-temporal AUC 0.589, and onset-row AUC about 0.33. Those historical figures are not a current rolling scorecard. Read the onset-skill finding →
Archived online conformal method: The May 2026 writeup described an online update (Gibbs & Candès, NeurIPS 2021) to a conformal quantile. Its replay figures do not establish current updates or validated coverage.Archived replay claim (840 outcomes, Apr 17 → May 14): α = 0.10 → 0.21 · empirical coverage 91.3% · cron 03:45 UTCThe earlier API contract exposed ACI fields in /v1/forecast/{cc}/7day responses. Their presence is not proof of active updates or qualified coverage. Full ACI methodology →