Voidly
All findingsPublished research

STL seasonal anomaly detector (complement to DBSCAN, orthogonal signal)

New per-country anomaly detector using STL (Seasonal-Trend decomposition via Loess; Cleveland 1990) that learns each country's own weekly rhythm and flags days that break it. ORTHOGONAL to DBSCAN at /v1/anomaly/dbscan/{cc} — DBSCAN catches shape-anomalous days against a 45-day rolling cloud; STL…

Read the full published summary

New per-country anomaly detector using STL (Seasonal-Trend decomposition via Loess; Cleveland 1990) that learns each country's own weekly rhythm and flags days that break it. ORTHOGONAL to DBSCAN at /v1/anomaly/dbscan/{cc} — DBSCAN catches shape-anomalous days against a 45-day rolling cloud; STL catches days that deviate from THIS country's own seasonal pattern. Egypt may always have high anomaly_rate on Fridays — DBSCAN sees that as normal, STL flags non-Friday spikes as anomalous. Implementation in scripts/build-stl-seasonal-anomaly.py: 90-day per-country daily anomaly_rate time series, statsmodels.tsa.seasonal.STL(period=7, robust=True), residual = observed - (trend + seasonal), z-score within country, flag if |z|>2.0. Cron daily at 04:45 UTC. Sidecar at /opt/voidly-ai/models/stl_seasonal_anomaly_v1.json, time-series at /opt/voidly-ai/data/stl_seasonal_anomaly.parquet. First run: 43 of 211 countries had ≥60 days of data for viable fit. Today's top |z|: BY +4.38 (Belarus, only flagged today). Last-7-day STL set (BY, NG, QA, SA) has ZERO overlap with today's DBSCAN flagged set (AU, BR, CA, DE, EG, ES, FR, GB, IN, IQ, JP, KR, MA, MX, NL, SG, US, ZA) — 4 unique STL signals + 18 unique DBSCAN signals, exactly the orthogonal complement promised. Honest caveats: STL is descriptive not causal (high z != censorship — could be holiday, fiber cut, measurement-coverage artifact), sensitive to data sparsity (168 of 211 countries skipped for <60d data), period=7 assumes weekly seasonality (constant censors get a small seasonal component but still get residuals).

Why STL is different from DBSCAN

Voidly Atlas already runs a CenDTect-style DBSCAN anomaly detector at /v1/anomaly/dbscan/{cc}. DBSCAN is shape-anomaly detection: it takes a country-day's 12-feature vector and asks how far it sits from any DBSCAN core cluster in the prior 45-day window. That catches days where the signal-type mix or block-rate has an unusual shape.

STL catches a different kind of anomaly. Egypt may always have a high anomaly_rate on Fridays. DBSCAN sees that Friday spike as “normal” for Egypt because it's shape-typical. A non-Friday spike, however, breaks Egypt's own weekly rhythm — STL flags it; DBSCAN often won't.

The two detectors are orthogonal: they catch different failure modes from the same evidence. Running both gives us a wider catchment of weird country-days without trading off precision.

How the model works

For each country with at least 60 days of evidence in the trailing 90-day window:

  • Build a daily time series of anomaly_rate = blocked / total measurements.
  • Fit statsmodels.tsa.seasonal.STL(period=7, robust=True) — Cleveland, Cleveland, McRae & Terpenning (1990). The robust=True variant downweights outliers in the trend and seasonal fits so the residuals capture real deviations, not just the outliers themselves.
  • residual = observed − (trend + seasonal).
  • Standardize residuals within the country and flag any day with |zscore| > 2.0.

First-run numbers

  • 43 countries had enough data (≥60 days) for a viable STL fit. 168 countries were skipped (sparse coverage, mostly small islands and microstates).
  • Today's top |z|: Belarus +4.38 — observed anomaly_rate 1.000, trend 0.24, seasonal −0.04, residual 0.79. BY is the only country with |z| > 2.0 today.
  • Trailing 7-day STL anomaly set: BY, NG, QA, SA.
  • Today's DBSCAN flagged set: AU, BR, CA, DE, EG, ES, FR, GB, IN, IQ, JP, KR, MA, MX, NL, SG, US, ZA.
  • Overlap: 0 countries. The two detectors caught entirely different country-days — exactly the orthogonal complement we wanted.

Endpoints

  • GET /v1/anomaly/seasonal/{cc} — latest day's residual + z + decomposition. Supports ?date=YYYY-MM-DD for backtest and ?history=N for the last N days of decomposition.
  • GET /v1/anomaly/seasonal/leaderboard — top-K today by |z|. Supports ?only_anomalies=true to filter to |z| > 2.0 only.
  • GET /v1/anomaly/seasonal/info — sidecar transparency + honest caveats.

Honest caveats

  • STL is descriptive, not causal. A high |z| means the country broke its own weekly rhythm — that could be censorship, a fiber cut, a holiday, or a measurement-coverage artifact (a sudden drop of probes in country X will look like a seasonal break even if nothing was actually blocked).
  • Sensitive to data sparsity. 168 of 211 countries in our evidence table didn't have 60 days of dense daily observations — they get skipped entirely. The 43 countries we do fit are heavily weighted toward the high-measurement set (G20, MENA, censorship-heavy regimes).
  • period=7 assumes weekly seasonality. Countries without weekly cycles (constant-state censors, regions with shifted weekend equivalents) still get a fit, but their seasonal component is small — the residual then captures almost all the variance, and |z| collapses to a noise estimate.
  • The single-day flag count is small by design (1 today, 4 in the trailing week). |z| > 2.0 is a ~5% gate in normal-residual assumptions; if we lowered to |z| > 1.5 we'd get 6× more flags but most would be noise. The hard 2.0 gate keeps the signal-to-noise high.

Cron

Daily at 04:45 UTC, after the DBSCAN refresh and before the atlas-digest builder. scripts/build-stl-seasonal-anomaly.py rebuilds the parquet at /opt/voidly-ai/data/stl_seasonal_anomaly.parquet and the sidecar at /opt/voidly-ai/models/stl_seasonal_anomaly_v1.json. The Flask endpoints mtime-cache the parquet so they pick up the new fit on the next request without a service restart.

The source record.

Original source links and labels. Linked responses may have changed since publication.