Why STL is different from DBSCAN
Voidly Atlas already runs a CenDTect-style DBSCAN anomaly detector at
/v1/anomaly/dbscan/{cc}. DBSCAN
is shape-anomaly detection: it takes a country-day's
12-feature vector and asks how far it sits from any DBSCAN core
cluster in the prior 45-day window. That catches days where the
signal-type mix or block-rate has an unusual shape.
STL catches a different kind of anomaly. Egypt may always have a high anomaly_rate on Fridays. DBSCAN sees that Friday spike as “normal” for Egypt because it's shape-typical. A non-Friday spike, however, breaks Egypt's own weekly rhythm — STL flags it; DBSCAN often won't.
The two detectors are orthogonal: they catch different failure modes from the same evidence. Running both gives us a wider catchment of weird country-days without trading off precision.
How the model works
For each country with at least 60 days of evidence in the trailing 90-day window:
- Build a daily time series of
anomaly_rate = blocked / total measurements. - Fit
statsmodels.tsa.seasonal.STL(period=7, robust=True)— Cleveland, Cleveland, McRae & Terpenning (1990). Therobust=Truevariant downweights outliers in the trend and seasonal fits so the residuals capture real deviations, not just the outliers themselves. residual = observed − (trend + seasonal).- Standardize residuals within the country and flag any day with
|zscore| > 2.0.
First-run numbers
- 43 countries had enough data (≥60 days) for a viable STL fit. 168 countries were skipped (sparse coverage, mostly small islands and microstates).
- Today's top |z|: Belarus +4.38 — observed anomaly_rate 1.000, trend 0.24, seasonal −0.04, residual 0.79. BY is the only country with |z| > 2.0 today.
- Trailing 7-day STL anomaly set: BY, NG, QA, SA.
- Today's DBSCAN flagged set: AU, BR, CA, DE, EG, ES, FR, GB, IN, IQ, JP, KR, MA, MX, NL, SG, US, ZA.
- Overlap: 0 countries. The two detectors caught entirely different country-days — exactly the orthogonal complement we wanted.
Endpoints
GET /v1/anomaly/seasonal/{cc}— latest day's residual + z + decomposition. Supports?date=YYYY-MM-DDfor backtest and?history=Nfor the last N days of decomposition.GET /v1/anomaly/seasonal/leaderboard— top-K today by |z|. Supports?only_anomalies=trueto filter to |z| > 2.0 only.GET /v1/anomaly/seasonal/info— sidecar transparency + honest caveats.
Honest caveats
- STL is descriptive, not causal. A high |z| means the country broke its own weekly rhythm — that could be censorship, a fiber cut, a holiday, or a measurement-coverage artifact (a sudden drop of probes in country X will look like a seasonal break even if nothing was actually blocked).
- Sensitive to data sparsity. 168 of 211 countries in our evidence table didn't have 60 days of dense daily observations — they get skipped entirely. The 43 countries we do fit are heavily weighted toward the high-measurement set (G20, MENA, censorship-heavy regimes).
period=7assumes weekly seasonality. Countries without weekly cycles (constant-state censors, regions with shifted weekend equivalents) still get a fit, but their seasonal component is small — the residual then captures almost all the variance, and |z| collapses to a noise estimate.- The single-day flag count is small by design (1 today, 4 in the trailing week). |z| > 2.0 is a ~5% gate in normal-residual assumptions; if we lowered to |z| > 1.5 we'd get 6× more flags but most would be noise. The hard 2.0 gate keeps the signal-to-noise high.
Cron
Daily at 04:45 UTC, after the DBSCAN refresh and
before the atlas-digest builder.
scripts/build-stl-seasonal-anomaly.py
rebuilds the parquet at
/opt/voidly-ai/data/stl_seasonal_anomaly.parquet
and the sidecar at
/opt/voidly-ai/models/stl_seasonal_anomaly_v1.json.
The Flask endpoints mtime-cache the parquet so they pick up the
new fit on the next request without a service restart.