Uncertain forecasts
Help the classifier learn
These are the 54 recent forecasts the classifier is least sure about — predictions sitting near the 0.5 decision threshold. Each one is a high-value labeling opportunity: a single confirmed outcome here teaches the model more than ten labels on easy cases.
Click any candidate to see its full SHAP breakdown, evidence permalinks, and an anonymous label-suggestion form. No login. No personal data required. Three independent reviewers must agree before a label is promoted into the next retrain dataset.
Method: Impact-aware ranking — Settles 2009 (UW TR-1648). Uncertainty fallback — Lewis 1994. Calibrated uncertainty — arXiv 2510.03162 (2025). · Refreshed every 15 min · Raw JSON · Methodology
Does the loop actually improve the model? Honest answer: not yet.
Loop state: plumbed_idle. The loop is wired end to end (queue -> submit-label -> aggregate -> promote -> retrain) and is idle until reviewers label candidates at /atlas/active-learning. We surface the simulated lift even when it is negative.
Queue: 527 candidates. Human label-submission counts are internal operator metrics and are not shown.
- Active-learning value depends on humans actually labelling. loop_state plumbed_idle means no human labels yet.
- simulated_f1_lift uses PROXY labels (incident-table positives within +/-3 days, else negative), not human labels. It is an optimistic estimate; treat it as "is the queue surfacing learnable rows", not a promotion signal.
Live source: /v1/sentinel/active-learning-loop-status
How submissions become labels
How submissions become labels
- You read the evidence, pick a label, and submit anonymously.
- Submission lands in an append-only audit log (
active_learning_submissions.jsonl). Nothing is overwritten or deleted. - Once 3+ independent reviewers agree on the same candidate, the daily aggregator promotes a consensus label.
- The next weekly classifier retrain consumes the consensus file and gates promotion as usual.
Privacy: contact email is optional and only stored in the audit log — never echoed in any read endpoint and stripped from the consensus output. Your IP is hashed for spam-flood detection only (16-hex truncated SHA-256, useless for re-identification).
60 candidate forecasts — ranked by uncertainty
Related
- Impact-aware ranking methodology — why we replaced uncertainty-only sorting with a 3-factor heuristic EER
- /sentinel/labeling — read-only queue view with API-key submission flow
- /atlas/changelog — model registry / retrain history
- /methodology — retrain gate logic
- Settles 2009 — Active Learning Literature Survey (UW TR-1648)
- Calibrated Uncertainty Sampling (arXiv 2510.03162)