Library: 103 Sigma detection rules across 10 MITRE ATT&CK tactics Date opened: 2026-04-11 Date closed: 2026-04-12 (single mechanically-verified PR) Status: Complete — allowlist empty, validator in strict mode
The first run of a newly authored content validator against the Sigma rule library flagged 36 duplicate-ID errors spanning 69 files, grouped into 33 distinct collision sets (including three 3-way collisions). The library had been passing all prior integrity checks.
On inspection, the id field of most colliding rules was not a real UUIDv4 at all. It was a sequential placeholder pattern:
1a2b3c4d-5e6f-7a8b-9c0d-1e2f3a4b5c6d
2b3c4d5e-6f7a-8b9c-0d1e-2f3a4b5c6d7e
3c4d5e6f-7a8b-9c0d-1e2f-3a4b5c6d7e8f
...
Every value matched the shape regex ^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ but failed the UUIDv4 version bit — the third group should start with 4; these started with 7, 8, 9, etc. Because the pattern was sequential and the library had been batch-authored, the same placeholders repeated across tactic directories, producing 33 distinct collisions.
The prior integrity check for the detection library was a PowerShell counter at scripts/verify/verify-counts.ps1. It counted *.yml files and confirmed the count matched a locked snapshot. That check never had any view into YAML content — it couldn't see id fields, couldn't detect collisions, and couldn't distinguish a real UUID from a placeholder. As long as the file count matched, CI passed. The bug was invisible by construction.
This is worth dwelling on: a count check and a content check are not substitutes. Content can be corrupt while the count is stable. Count can be correct while every file is silently wrong. The library needed both, and until the new validator was authored, it only had one.
The new validator — validate_detection_content.py — was written on 2026-04-11 to fill the gap. On first run it parsed every YAML file in the Sigma tree, extracted the id field, and checked:
- Is the id a valid UUIDv4 (shape + version bit + variant bit)?
- Is the id unique across the library?
First-run output: 36 errors across 69 files. Those errors were the thing nobody had ever seen before.
Before doing any remediation, the validator was given a temporary scaffolding allowlist (validation_exceptions.yml) listing the 33 duplicate IDs and their 69 involved files. The allowlist rules were strict:
- Downgrade a duplicate-ID error to a warning only if both the UUID and the involved file set match the allowlist exactly.
- If a new file joins a collision group, fail hard.
- If an allowlisted file is renamed without updating the allowlist, fail hard.
This kept CI green while remediation was in progress without giving ground on enforcement. The allowlist was generated programmatically from the live tree — not hand-typed — so it couldn't be wrong about what was in scope.
The remediation had two groups:
- Group 1 — 33 collision groups / 69 files. The allowlisted set.
- Group 2 — 32 additional non-colliding placeholder UUIDs. Files that used the same sequential-hex pattern but hadn't happened to collide. They passed the validator on shape regex but were not real v4 UUIDs and represented latent collision risk.
Total scope: 101 files, 1 allowlist file to empty.
The remediation ran as a single mechanically-verified pass rather than the originally planned tactic-by-tactic cohort split. The cohort split was abandoned because the collisions were not cohort-aligned — almost every collision group spanned 2+ tactic directories, so a clean split wasn't possible.
for each of 101 files:
generate fresh UUIDv4
verify uniqueness against all other new UUIDs
verify uniqueness against all existing Sigma ids in the tree
regex-match the `id:` line exactly once
edit the id line in place
re-parse the YAML to confirm it still loads
abort on any per-file failure
Defensive properties:
- Uniqueness among new UUIDs caught any duplicate in the generated set.
- Uniqueness against existing tree ids caught accidental collisions with the 2 files that already had real UUIDs.
- Regex single-match caught any file where
id:appeared in an unexpected place (comments, documentation strings). - Re-parse caught any edit that corrupted the YAML structure.
- Abort on failure guaranteed no partial state.
| Metric | Value |
|---|---|
| Files updated | 101 |
| New real UUIDv4s generated | 101 |
| Collisions in new set | 0 |
| Collisions with prior ids | 0 |
| Allowlist entries remaining | 0 |
| Validator errors (post-fix) | 0 |
| Validator warnings (post-fix) | 0 |
| Sigma file count (before and after) | 103 (unchanged) |
The allowlist was emptied in the same commit. The validator was left in fully strict mode for the Sigma lane — no allowlist, no exceptions, no loopholes.
Final audit confirmed all 103 Sigma IDs are unique and version-4 compliant: third group starts with 4, fourth group starts with 8, 9, a, or b. Both local and CI validator runs reported zero errors and zero warnings.
- Detection coverage: unchanged. No rules added, removed, renamed, or modified in logic.
- CI integrity: upgraded. The library now has a content-aware gate, not just a count check.
- Audit trail: one PR, one commit, one cleanly documented remediation pass.
- Future risk: bounded. The validator in strict mode means no new collision can reach
mainwithout failing CI.
The prior PowerShell counter was doing exactly what it was designed to do — verifying that nothing had silently been added or deleted. It was not designed to verify that the content of each file was internally correct, and it couldn't have been retrofitted to do so. The lesson isn't "the counter was bad." The lesson is that integrity has multiple independent dimensions, and each one needs its own gate.
The most useful detection a new validator will ever make is usually the one that surfaces the bug nobody knew existed. If you've run a new validator and it found nothing, that's fine — but you should wonder whether it was asking the right questions. This validator's first run found 33 collisions that had been silently present for months. That was the test of whether the tool was worth writing.
An allowlist that can only grow is a loophole. An allowlist that is:
- time-bounded (scheduled for deletion),
- file-scoped (can't silently absorb new violations),
- auto-failing on drift (hard errors if the collision set changes),
- and owned (someone is on the hook for emptying it)
…is scaffolding. It lets remediation happen without taking CI offline. The allowlist in this remediation was emptied the day the fix shipped, which is the shape it had to have to be acceptable.
101 files edited by hand would have introduced at least one typo. 101 files edited by a script with uniqueness + regex + re-parse checks was provably correct. The correct use of automation here wasn't to go faster — it was to make the operation verifiable.
"All our rules pass CI" is a weak claim. "We wrote a validator, ran it against our own library, found 33 collisions that had been invisible for months, remediated 101 files in one pass, and CI has been strict ever since" is a much stronger claim — because it proves the gate actually catches things. A CI pipeline that never rejects anything isn't enforcing anything; it's decorating.
Sanitized summary of the remediation plan and post-mortem. Rule counts, collision counts, and remediation scope preserved exactly from source evidence.