- Corpus build: pull incident titles and descriptions from the DB at model training time, skip empty or short text, then deduplicate matching lowercased text after whitespace normalization using SHA-256. The model contains 1,195 documents.
- Vectorize: tf-idf, unigrams + bigrams, min_df=2, max_df=0.95, vocab capped at 5,000, English stopwords + domain stopwords (boilerplate like "cenalert", "detected", country names) ⇒ 623 terms.
- NMF sweep: fit sklearn NMF with k ∈ [8, 24] step 2, init="nndsvda", beta_loss=frobenius, score by NPMI coherence on top-12 words per topic. Pick the k that maximizes coherence.
- Assignment: each doc → argmax topic (hard assignment). Documents with W weight below 1e-6 fall into an "unlabeled" bucket.
- Labels: auto-generated by checking the top-N words against a hand-curated keyword map (election / VPN / DNS / etc). Falls back to "Topic N: word1 / word2" if no keywords match. These are heuristic — they exist to make scanning faster, not to be authoritative.
k sweep
Coherence by k (NPMI on training corpus, top-12 words per topic):
Scroll to see every column.