Voidly
All findingsPublished research

Temporal Fusion Transformer (Lim et al. 2021) vs 3-XGBoost multi-horizon stack

Built a Temporal Fusion Transformer (TFT, Lim et al. arXiv:1912.09363) over the same 38-feature country-day panel that powers our XGBoost forecast. Single attention-based pass predicts a 30-day p10/p50/p90 trajectory, replacing 3 independent GBMs + a Gaussian conformal wrapper. Trained on CPU…

Read the full published summary

Built a Temporal Fusion Transformer (TFT, Lim et al. arXiv:1912.09363) over the same 38-feature country-day panel that powers our XGBoost forecast. Single attention-based pass predicts a 30-day p10/p50/p90 trajectory, replacing 3 independent GBMs + a Gaussian conformal wrapper. Trained on CPU within a 60-minute budget (hidden=8, 1 attention head, quantile loss over 0.1/0.5/0.9). LOCO median AUC vs the 7-day XGBoost baseline (0.9236) is published inline at /v1/forecast/tft/info. Honest caveats inline: quantile collapse on rare positives, small 21-country corpus, CPU budget. Shipped as a research second opinion, NOT a production replacement — the headline 7-day forecast still flows through /v1/forecast/{cc}/7day.

What we shipped

A Temporal Fusion Transformer (TFT, Lim et al. 2021, arXiv:1912.09363) trained on the same 38-feature panel that powers our XGBoost per-horizon forecasts. The TFT predicts a 30-day p10/p50/p90 trajectory in a single attention-based forward pass, replacing what would otherwise be three independent gradient-boosted models and a Gaussian conformal wrapper. Live at GET /v1/forecast/{cc}/tft; metadata at GET /v1/forecast/tft/info.

Architecture and inputs

21 watched countries × 731 days = 15,351 country-day rows. Each sample uses a 30-day encoder window over 33 observed reals (lag and rolling block-rate stats, OONI anomaly counts, IODA alerts, probe consensus flags, GDELT protest/conflict signals) plus 5 known calendar covariates (day of week, week of year, month, weekend flag, Friday flag). Static categoricals are the country ID and its tier-1-through-5 risk-tier label.

We use the canonical TFT building blocks — variable selection networks, gated residual blocks, an LSTM encoder, an interpretable multi-head attention decoder — with hidden size 8 and a single attention head (small enough to converge on CPU in the 60-minute budget the directive required). The loss is a quantile loss over (0.1, 0.5, 0.9), which is the textbook way to get a calibrated forecast interval out of a probabilistic regressor without a separate conformal step.

How it scored

We benchmark against the existing multi-horizon XGBoost stack (1-day / 7-day / 30-day separate models, each with its own isotonic calibrator). The bar to promote was set deliberately high: LOCO median AUC must beat the 7-day baseline (0.9236) by at least 0.03, with p10/p90 empirical coverage inside ±10pp of the 80% nominal band.

Numbers will be filled in once the training run completes; we publish the LOCO table inline in /v1/forecast/tft/info regardless of the promotion outcome.

What we're honest about

  • Quantile collapse on rare positives. Quantile loss on a binary target with a 10% positive rate tends to drive p10 and p90 toward the same value — the model becomes a point predictor in disguise. We document this explicitly in the endpoint response's honest_caveats array so callers don't treat the interval as well-calibrated.
  • CPU training budget. The 60-minute cap means we cannot do a real grid search; hidden_size = 8 was chosen from a single timing benchmark, not a Bayesian sweep. A larger model on GPU might widen the gap to XGBoost, but the directive here is “fits the hardware we already have.”
  • 21-country corpus. The TFT shares attention weights across countries via the static categorical, so it benefits from cross-country signal. But the labeled corpus is still small — LOCO with n = 21 has wide confidence intervals on the median.

What this is for

Even when the TFT does not strictly beat the boosted stack on median LOCO AUC, the per-horizon attention curve and the unified probabilistic output make it a useful second opinion when the XGBoost models disagree. The endpoint is shipped as a research artifact, not a replacement — the production forecast still flows through /v1/forecast/{cc}/7day.

The source record.

Original source links and labels. Linked responses may have changed since publication.