CI caught this on the previous commit's first push: the credential scan flagged
ci/selftest.sh itself, twice. The file is tracked, the scan reads every tracked
file, and the fixtures were credential-shaped literals sitting in it.
It passed locally because I ran ci/static.sh before `git add`, so selftest.sh
was still UNTRACKED and therefore invisible to a scan reading git ls-files.
That trap is documented four lines into the same file, about the fixtures it
plants. Knowing it and applying it to your own validation are two different
things, and the second is the one that counts.
Fixed by assembling every fixture value from pieces, which is stronger than any
exemption:
- A filename exemption would switch the scan off in ci/ and tests/, which is
exactly where a key gets pasted to test it. A sibling plugin reached the
same conclusion from the other side when its redactor fixture went red.
- A placeholder marker in the value would stop the fixture triggering the
check it exists to trigger.
- Split literals leave the WRITTEN file fully credential-shaped while the
SOURCE contains no matching string at all, so nothing needs excusing.
Verified both directions: the source no longer matches any pattern in the scan,
and each assembled value still does. static.sh green on the real tree with
selftest.sh tracked (45 files read, up from 44), self-test 26/26.
The pipeline worked as designed — static.sh runs before selftest.sh in the
static job, so the real tree was checked first and the failure named the file
and both rules. Nothing about this needed a human to notice.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three rounds of fixes to this kit, and each round found the previous round's
blind spot: a check that examined nothing, then a check that examined
everything with a pattern that could not match, then a flag that made one rule
skip binary files without a word. None of them were visible from a green run,
and each was found by a sibling plugin rather than by anything here.
So the battery stops being a shell loop somebody retypes and becomes a script
CI runs. 26 assertions: 13 credential shapes that must be flagged, 6 lookalikes
that must stay quiet, the direct-access count, the four text-domain verdicts,
the annotated path being a real file, and a plugin.conf with its floors
stripped failing to load.
🔴 It asserts on the thing that has CONSEQUENCES — the exit status of the
script CI runs, and the specific failure message. Never a filename, a count, or
a log line that merely sits near the signal. The previous ad-hoc battery
asserted "the fixture's filename appears in the output", which passes on grep's
own `binary file matches` diagnostic, because that diagnostic prints the
filename in exactly the place a real failure would. A sibling plugin recorded
two binary cases as PASS on that basis. The probe had the same defect as the
check it was testing.
The self-test was then checked to go red: making the AKIA pattern inert fails
exactly one assertion, naming the shape.
Its first run failed an assertion, and the fixture was at fault rather than the
product. This script sources ci/lib.sh, so the floors are in the environment
and a plain subshell inherits them — ci_load_conf then passed over a
plugin.conf that does not define them. Fixed with an explicit unset, and the
reason is inline so nobody weakens the assertion instead. A failing probe is
evidence about the probe until you have looked at it.
Wired into the static job in BOTH workflow roots, byte-identical: .gitea hides
.github completely rather than merging it, so a step added to one root only
would not run on Gitea and would look exactly like a runner that had not picked
the job up. ci/workflows-in-sync.sh passes.
Excluded from the release zip, and it plants nothing in the real tree — the
fixtures go into a throwaway copy under mktemp, several being credential-shaped
by definition.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
