Voidly
All findingsPublished research

Per-blocking-method specialized classifiers (4 methods, 2 promoted)

v3.3 is a single classifier for "is this country-day censored?" But censorship has different mechanisms. We trained 4 specialized XGBoost classifiers (DNS / TCP / HTTP / TLS), each with the same 16-feature v3.3 input. Per-method positive rates are 2-12%, so F1 at threshold 0.5 is unfair — we added…

Read the full published summary

v3.3 is a single classifier for "is this country-day censored?" But censorship has different mechanisms. We trained 4 specialized XGBoost classifiers (DNS / TCP / HTTP / TLS), each with the same 16-feature v3.3 input. Per-method positive rates are 2-12%, so F1 at threshold 0.5 is unfair — we added an alt-AUC promote path (strat AUC >= 0.80 AND LOCO median AUC >= 0.75 AND optimal F1 >= 0.50). HTTP-blocking (515 pos, strat AUC 0.901) and TLS-blocking (313 pos, strat AUC 0.918) clear the alt path. DNS (194 pos, AUC 0.952 but opt F1 0.484) and TCP (163 pos, AUC 0.903 but opt F1 0.389) skipped. Endpoints return available=false with honest reason for skipped methods. v3.3 stays as the default classifier — this is additive transparency for journalists asking HOW a country is blocking.

v3.3 is a single classifier for "is this country-day censored?" But censorship has different mechanisms — DNS poisoning returns a wrong IP, TCP RST kills connections mid-handshake, blockpage injection serves an HTTP block page, TLS interference drops the SNI based on hostname. A classifier specialized for DNS might catch DNS attacks that the generic classifier misses, and vice versa. Journalists want to know HOW a country is blocking, not just whether.

We trained 4 specialized XGBoost classifiers, each with the same 16-feature v3.3 input. Labels come from evidence.signal_type: a country-day is method-positive iff that date had ≥1 evidence rows matching the method's signal_type pattern. Negative samples are the same v3.3 negatives shared across all 4 method datasets.

Per-method metrics (4 classifiers, last-365-day evidence)

Methodn_posStrat AUCStrat F1Opt F1LOCO median AUCLOCO median F1Promoted
http-blocking5150.9010.5530.5700.8070.314yes (alt-AUC path)
tls-blocking3130.9180.5070.5060.7950.336yes (alt-AUC path)
dns-blocking1940.9520.4710.4840.9310.417no
tcp-blocking1630.9030.3580.3890.7930.235no

Why two promote paths?

Original gate was stratified F1 ≥ 0.65 AND LOCO median F1 ≥ 0.70. Every method failed it. Root cause: per-method positive rates are 2–12%, so F1 at the default 0.5 threshold is unfairly punitive — the classifiers have strong AUC (0.79–0.95) but the default threshold is wrong for the heavy imbalance.

We added an alt-path: strat AUC ≥ 0.80 AND LOCO median AUC ≥ 0.75 AND threshold-optimal F1 ≥ 0.50. Two methods (HTTP, TLS) clear the alt-path. DNS came closest on AUC (0.95) but couldn't hit the 0.50 optimal-F1 floor on only 194 positives. TCP is genuinely weak with 163 positives concentrated in a few countries.

What this is and isn't

Is: an additive transparency layer. v3.3 generic classifier is still the default for "is this country-day censored?". Per-method scores answer a different question: given that something is going on, how is the country blocking?

Isn't: an oracle. Labels were derived from signal_type heuristics, not curated mechanism tags from incident reports (only ~10% of confirmed incidents have a mechanism field populated). A "true" per-method classifier would need a method-labeled ground truth from researcher confirmation, not signal-type aggregation.

Promoted endpoints

  • GET /v1/classifier/method/info — model metadata + per-method metrics + honest caveats
  • GET /v1/classifier/method/http-blocking/{cc} — per-method probability for that country today
  • GET /v1/classifier/method/tls-blocking/{cc} — same for TLS

Skipped methods (dns-blocking, tcp-blocking) return available: false with a failed_promote_floor reason, so callers know the metrics exist but the model isn't live.

Pipeline

  • scripts/build-per-method-classifier-features.py — per-method labeled datasets
  • scripts/train-per-method-classifiers.py — train 4 XGBoosts, evaluate, persist as one pickle dict
  • scripts/patch-per-method-classifier-endpoint.py — idempotent, ast-validated
  • Sidecar: /opt/voidly-ai/ml-deploy/per_method_classifiers_v1.json

The source record.

Original source links and labels. Linked responses may have changed since publication.