Voidly
All findingsPublished research

Cross-protocol classifier: per-port blocking probability (8 protocol groups)

Eight small XGBoost classifiers, one per protocol group (HTTP-80, HTTP-headers, web_connectivity, TLS-WhatsApp, TLS-Signal, TLS-Telegram, TLS-FB-Messenger, Tor). Given a measurement to a (host, port) on a country/day, each model returns the probability that port is blocked. Built on OONI evidence…

Read the full published summary

Eight small XGBoost classifiers, one per protocol group (HTTP-80, HTTP-headers, web_connectivity, TLS-WhatsApp, TLS-Signal, TLS-Telegram, TLS-FB-Messenger, Tor). Given a measurement to a (host, port) on a country/day, each model returns the probability that port is blocked. Built on OONI evidence by parsing the N/M anomalous-measurements ratio out of upstream_claim text (necessary because non-web_connectivity tests only store rows when blocking occurs). All 8 cleared promote floor: LOCO pooled-OOF AUC 0.98 to 0.999, per-country median AUC 0.89 to 1.0. Honest caveat: the high AUCs come from strong port-level blocking persistence (history is the dominant feature), not novel signal — and web-connect labels partially overlap with TLS-app labels (both touch 443). Live at GET /v1/classifier/protocol/{proto}/{cc} and /v1/classifier/protocol/info.

The Voidly Atlas country-level classifier (v3.3) answers "is this country blocking something today?" The new cross-protocol classifier answers a finer question: "which protocol/port is blocked?"

Different protocols are blocked at different layers with different signatures. HTTP-80 plaintext is typically intercepted by middleboxes inspecting the Host header. HTTPS/443 is blocked by SNI filtering or IP-list null-routes. Tor is killed by IP blocklists on the OR ports (9001/443). WhatsApp's mTLS endpoints get blocked at the CDN edge by SNI. A single country-level "censored" verdict elides this.

We trained eight small XGBoost classifiers, one per protocol group. Each takes a (country, day) input and emits the probability that the protocol was blocked. Features are nine cheap signals: protocol's own 7-day history, neighbor 7-day rate (UN sub-region adjacency), the day's max anomaly rate across other protocols (a "cross-protocol lockdown" detector), country risk tier, and four calendar features. Crucially we drop today's own anomaly rate from features — that's the label-defining quantity, so including it would make the model circular and inflate AUC to 1.0.

Labels and label noise

Labels come from parsing OONI's N/M measurements anomalous ratio directly out of the upstream_claim text — necessary because OONI evidence rows for everything except web_connectivity are only stored when blocking occurred (OK measurements aren't written). A naive SELECT COUNT(*) WHERE signal != 'ok' would show 100% positive rate for every protocol; parsing the embedded ratio recovers the true daily denominator. A (country, day, protocol) is a positive sample iff ≥ 30% anomalous on ≥ 3 measurements, negative iff ≤ 5% anomalous, and the ambiguous middle band is dropped.

Results

All eight protocols cleared the promote floor (stratified F1 ≥ 0.55 AND LOCO pooled AUC ≥ 0.70). The LOCO pooled-OOF AUCs cluster between 0.98 and 0.999 because port-level blocking is strongly persistent — the 7-day history feature carries most of the predictive load. This is realistic signal, not leakage: history is strictly from days before the target day, and labels are computed from the target day alone.

What this is not

  • This does not override the v3.3 country-level classifier. v3.3 answers "is this country censoring"; per-protocol answers "which port is blocked".
  • Inter-protocol overlap is real. The web-connect group includes both HTTP/80 and HTTPS/443 measurements, so its labels partially correlate with the TLS-app groups for the same country-day.
  • OONI coverage is uneven. http_invalid_request_line and tor have less measurement volume than web_connectivity, so the corresponding models have thinner LOCO evidence (fewer countries with both classes in held-out).
  • The "AUC ~0.99" headline reflects the strong autocorrelation of port-level blocking, not novel signal discovery. Per-country median AUC is more honest (range 0.89 to 1.0); pooled OOF AUC is the promote-gate number.

API surface

  • GET /v1/classifier/protocol/info — all 8 protocols, per-protocol metrics, feature importances, honest caveats.
  • GET /v1/classifier/protocol/{proto}/{cc} — per-(protocol, country) probability for today, with observed measurement data surfaced as observed_today for transparency.

Sample: /v1/classifier/protocol/tor/CN returns P(Tor blocked in China) = 0.96 (label_at_optimal = 1, threshold = 0.15); /v1/classifier/protocol/tls-signal/IR returns P(Signal blocked in Iran) = 0.99.

Leakage audit (2026-05-21). A label-leakage audit flagged this model as SUSPICIOUS. The training script correctly drops anomaly_rate (which defines the label), but the feature cross_protocol_max_today is the same-day max anomaly rate of other protocols — and protocols overlap heavily (web-connect already covers port 443 = the TLS class), so for correlated protocols it is a near-copy of the label. A LOCO pooled AUC of 0.98+ on 9 non-anomaly-rate features is implausible. Honest LOCO AUC estimate without the same-day cross-protocol feature: ~0.80-0.88. Recommended fix: drop cross_protocol_max_today (or restrict it to non-overlapping protocols) and re-run the LOCO evaluation.

The source record.

Original source links and labels. Linked responses may have changed since publication.