The v3.3 classifier was promoted with strong honest metrics (LOCO median F1 0.870, stratified F1 0.729) but those numbers only describe performance on data that didn't try to evade us. A censorship regime is an adversary, not a passive distribution. So we ran a benchmark designed around six tactics a regime could plausibly use to hide from the model.
The setup
We sampled 200 known-positive country-day incidents from the v3.3 training set, stratified across 127 countries so no single regime dominates the result. For each positive we generated six perturbed copies, each one simulating a specific evasion strategy applied to the feature vector the model sees:
- half_anomaly — anomaly_rate × 0.5: the regime blocks 50% fewer users.
- noise_x2 — measurement_count × 2: drown the signal in extra clean traffic.
- half_probe_rate — probe_block_rate × 0.5: selectively unblock during probe windows.
- smoothed_spike — spike_magnitude × 0.3: ramp up gradually instead of step-change.
- isolated_regime — neighbor contagion features zeroed: the regime acts entirely alone.
- all_combined — every perturbation above, applied simultaneously.
Each of the 1,400 perturbed vectors gets scored through the live production v3.3 model (no defensive retraining, no smoothing layer, no adversarial fine-tuning). We measure detection rate at the default threshold 0.5 and report both unconditional detection and conditional retention (of the incidents the model originally caught, what fraction does it still catch after the attack).
Headline results
| Attack | Detection rate | Retention (conditional) |
|---|---|---|
| baseline | 0.930 | — |
| half_anomaly | 0.900 | 0.941 |
| noise_x2 | 0.880 | 0.903 |
| half_probe_rate | 0.930 | 1.000 |
| smoothed_spike | 0.885 | 0.952 |
| isolated_regime | 0.905 | 0.952 |
| all_combined | 0.950 | 0.962 |
Honest reading of these numbers
The most fragile single attack is noise_x2:
doubling the measurement count drops conditional retention to 90.3%.
That makes sense — the model has a high_evidence
feature gated at 50 measurements, and several mid-impact decision
paths rely on the rate_count_interaction
product. Doubling the denominator dilutes both.
The surprising result is that all_combined doesn't cause the worst damage — in fact it lands a touch above baseline (0.95). The reason is structural: GradientBoosting decision trees route by thresholds, not by linear combinations. When every feature moves at once, the vector lands in a different leaf with different statistics. The combined attack happens to push these particular positives into leaves trained on the dataset's high-measurement / low-spike modes, which are also positive. So a regime doesn't get a free lunch by stacking tactics; in 38 of 186 incidents the combined attack strengthens the prediction.
This is not a defense claim. It's a measurement claim. A determined regime that knows our feature schema can almost certainly do better with white-box adversarial optimization (PGD-style projected gradient descent on the feature space, gradient-boosting equivalent). We did not run that, because it would require a threat model where the regime knows the exact feature pipeline — a stronger assumption than is realistic today.
Where v3.3 is actually weak
- noise dilution: 18 of 186 originally-caught incidents flip negative when measurement_count doubles. Operators of restrictive ISPs can encourage this naturally by silently proxying probe traffic through clean nodes.
- smoothed spikes: 9 of 186 incidents lose detection when spike_magnitude is cut to 30%. The model learned to look for step-changes; a slow ramp-up over several days will degrade signal.
- isolated regime: 9 of 186 incidents lose detection when neighbor contagion features zero out. This is already the default state for ~80 of 131 training countries (negative or low-overlap correlation neighbors) — so the measurement here is partly about feature coverage, not just adversarial robustness.
Where v3.3 is structurally robust
- half_probe_rate retention is 1.000. The probe network is supplementary in v3.3 — the model leans much harder on OONI-derived anomaly_rate and the contagion features. Cutting probe blocking signal in half doesn't change a single prediction.
- Baseline median probability is 0.95. The model is confident enough on the average positive that a 5-percentage-point probability hit doesn't push most incidents below threshold.
What we did NOT measure
- False-positive shift. We only perturbed positives. A defensive smoothing layer that improved robustness here could just as easily inflate false positives on negative country-days. That's the next benchmark.
- White-box gradient attacks. We used hand-crafted perturbations modeling realistic regime behavior. PGD-equivalent adversarial search would almost certainly find more damaging perturbations.
- End-to-end attacks on the ingest pipeline. The perturbations apply to the country-day feature vector. We did not model adversarial behavior at the probe-network or measurement-aggregation level (e.g., colluding probes, fabricated measurements). Doing that requires modeling the ingest pipeline itself, not just the model.
Why we published this
When journalists cite a detection-rate number off our model, they should know that number was measured against passive evaluation — not against an adversary actively trying to fool us. The honest detection rate of v3.3 under realistic single-tactic evasion is 88-93%, not 99%. The retention rate under the worst single tactic is 90.3%. Those are the numbers a regime would have to design against.
We will re-run this benchmark on every promoted model. The harness is
public:
scripts/build-adversarial-perturbations.py +
scripts/evaluate-classifier-robustness.py.
The sidecar is served at
/v1/classifier/robustness.