db0cdd4e19697b0b016a9e80dc4ff835d5957758
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 kristianwindandClaude Opus 5
|
db0cdd4e19 |
1.6.3: the tested-WooCommerce version now agrees with itself
The plugin header said "WC tested up to: 11.1.2" and README.txt said 11.1.0 — since 1.6.2, which is the release whose whole content was bumping that value, and whose commit message says both were bumped. Only one was. The check that exists for this printed green the entire time, and both of its readings were correct: WordPress: README.txt says 7.1.2, current release is 7.1.2 WooCommerce: plugin header says 11.1.2, current release is 11.1.2 It reads each value from the file that value belongs in, for a good reason written in the comment above it: WordPress reads "Tested up to" only from readme.txt, WooCommerce reads "WC tested up to" only from the plugin header. So each check asks "is this value current" of the authoritative copy — and a SECOND copy of the same value in the other file is compared with nothing. A correct answer to a different question, which is the shape most false greens here have. Two checks, two questions, kept apart: - "Is this value current" stays advisory, per the comment already there: a build that goes red on release day for a header nobody has had a chance to update is a build that gets ignored. - "Do our own two copies say the same thing" is a fact about this tree with no release-day excuse, so it is fatal. Three pairs are actually duplicated here — WC tested up to, Requires at least, Requires PHP — so the step has live rows rather than one. Proven red one field at a time with a landing check each, and the count is on the green line, because a readme carrying none of these fields has nothing to disagree and would print the same silence as one where every pair matches. 50 selftest assertions. static=0 selftest=0 site=0; site still 69 on WordPress 7.1.2 with WooCommerce 11.1.2. README.txt ships, so this is a version bump: 1.6.3. The changelog keeps 1.6.2's entry — my first pass sed'd 1.6.2 to 1.6.3 across the file and rewrote that heading, which would have made the changelog describe a release that never happened. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
kristianwindandClaude Opus 5
|
539abf946c |
Stop reading a verdict out of xargs' exit status
`if git ls-files -z … | xargs -0 grep -nE …; then` has a false green in it. xargs exits 123 when ANY of its invocations exited 1-125, and grep exits 1 on no-match, so a run where one batch matched and another did not exits 123 — which `if` reads as clean, after grep has already printed the hit. Measured: one batch, a match xargs exit 0 -> caught one batch, no match xargs exit 123 -> clean, correct two batches, one match xargs exit 123 -> CLEAN, and it printed the hit Three checks here used that form: debug leftovers, short open tags, and the git-identity scan added two days ago. All three are one batch at this tree's size, so none of them is wrong today — this is a latent false green that arrives with the file list, not a bug anyone would see. Nothing in the output distinguishes the cases. scan_tracked() replaces it: one grep per file, verdict from the OUTPUT rather than any exit status, and a grep failure (exit > 1) reported instead of swallowed. Batching stops existing. The loop also yields the file count, which is the other thing all three were missing — "none" over zero files and "none" over nineteen were the same green. They now read "none in 19 tracked PHP files" and fail below a floor of one. The identity scan is also widened from ci/*.sh to every tracked file. A workflow step writes an identity into the runner's clone just as permanently, and ci/*.sh was scope chosen from where the first instance happened to live — the same "what are your checks pointed at" question as the contrast check. 13 files to 48. The new selftest row plants a write into a workflow file, which the old scope could not see at all. Reported by the Nolimit Security session, who hit the xargs case after widening their own scan from 13 files to 102. Three of my own probes were wrong before they were right, all the same class: the assertions grep for the check's message text, and widening the scope changed the wording from "A ci/ script sets …" to "A tracked file writes …". Five rows reported NOT FLAGGED while the check was working perfectly. They now match the stable half of the message, with the reason written beside them. And the form check matched the comment in static.sh that quotes the bad form in order to explain it, so it is anchored with ^[^#]* — the paragraph documenting a rule is text the rule applies to. That form check is a drift check on the shape and says so: it asserts the pattern is gone, not that the replacement is correct, because this tree's 48 files are one batch and nothing here can measure the fault. 46 selftest assertions. static=0 selftest=0 site=0; site still 69 assertions on WordPress 7.1.2 with WooCommerce 11.1.2. ci/ ships nothing, so no version bump. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
kristianwindandClaude Opus 5
|
97818ec4db |
Let the identity check tell a write from a read and from its own repair
Widening the scope group to --[a-z-]+ was wrong in a way the previous
commit's matrix could not see, because it had no repair row: --unset is
matched as a scope flag, and user. follows it, so
git config --local --unset user.name
is flagged as a violation. That is the remedy this check recommends in its
own failure message. A build that fails on its own repair is how a correct
check gets deleted — the failure accuses the check, and the next person
believes it. The same widening flags a read, echo "$(git config user.name)".
Two halves, both needed. Enumerate the three real scopes, and require a
VALUE after the key: user\.[a-zA-Z]+[[:space:]]+[^[:space:])]. The trailing
class is what makes a read legal — $(git config user.name) has ) where a
write has a value. Sixteen rows measured one at a time, 16/16, and eleven of
them now run in ci/selftest.sh (44 assertions).
Reported by the Nolimit Security session; three of us reached for
--[a-z-]+ independently, so the comment above the step says explicitly not
to, with the reason, rather than leaving it to be re-derived. The read row
came from nolimit-seo.
Two things found while proving it, both about the probe rather than the
check:
ci/selftest.sh is itself a ci/*.sh file, so its fixtures trip the check they
test. Spelled out in full, the first version of this block reported six
false positives and no count, while the same rows measured standalone were
16/16 correct — a failing probe is evidence about the probe until you have
looked at it. Fixed the way the credential fixtures three hundred lines up
in the same file already were: assemble the value through $gc so the source
matches nothing while the written file matches fully. The labels need it
too, so they name the shape instead of quoting the line.
And a plant that lands in a SOURCED file is not inert text. nolimit-seo
planted into ci/lib.sh, which ci/static.sh sources, so the line ran and set
the clone's identity for real. The target here is ci/build-zip.sh and only
ci/lib.sh is sourced — verified rather than assumed — and a final assertion
checks the clone still has no local identity, because if a fixture had
executed then every verdict above it was measured against a tree nobody
intended.
static=0 selftest=0 site=0; site still reports 69 assertions on WordPress
7.1.2 with WooCommerce 11.1.2. ci/ ships nothing, so no version bump.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
kristianwindandClaude Opus 5
|
cc0aaf9272 |
Scope the compatibility bump's git identity to its own commit
`ci/bump-tested-up-to.sh` ran `git config user.name "compatibility check"` before committing. That writes .git/config and outlives the script, so from the first run onward every commit in that working copy is authored by "compatibility check <noreply@localhost>" with no Co-Authored-By trailer — including a release made by a person days later about something else. git announces an identity it cannot FIND and says nothing about one that is wrong, so the missing case is loud and this one is silent. Nothing fails, nothing warns, and the author is only visible to somebody reading `git log` for another reason. Measured across the family by the Nolimit Security session: six misattributed hand commits in one sibling including two releases, two in another, one in a third. This tree was clean — no local identity, 0 misattributed — and one run of the script away. The commit still signs itself as the robot, which is right: that one is not a person's. It just does it with `git -c … commit`, which configures the command and not the clone. Guarded in ci/static.sh, because the fix is one line and reverting it would be invisible for exactly as long as nobody reads an author field. Widened past what was reported: the pattern accepts any scope flag, since --global is the same fault one scope wider and reaches every repository on the box. Proven with a full matrix rather than one trigger — no flag, --local, --global and --system each go red with the named message and exit 1; a commented mention, `git -c user.name=… commit`, `git config core.autocrlf` and `git config --global core.pager` all stay green. The correct form being a control matters: a check that flagged the fix would get one of the two deleted. Five of those assertions are now in ci/selftest.sh (36 total), each with a landing check that the plant is in the file before its verdict is believed, and the passing line carries the script count — "every script is clean" reads the same over zero files as over eleven. ci/ ships nothing, so there is no version bump and no release here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
kristianwindandClaude Opus 5
|
d1703dcc03 |
Check the direct-access guard by tokenising, not by grepping for it
The check asked whether "defined( 'ABSPATH' )" appeared anywhere in the file. A comment containing those words therefore passed it. Measured with a planted file whose only mention was in a comment: 15 shipped PHP files examined, every one guarded Green over a file with no guard at all — one that runs for anyone who fetches it directly, outside WordPress and outside every capability check the code assumes has already happened. This is the check with the worst consequence of being hollow, and it was hollow in the one way a grep always can be. From the cache session by way of security: a check that must RECOGNISE its own code has one blind spot per recognition method, and one that reads what the source actually says has none of that kind. PHP's tokeniser cannot be fooled by a comment, because it is the thing that decides what a comment is. ci/check-abspath.php parses each shipped file and looks for a real defined() call on ABSPATH or WP_UNINSTALL_PLUGIN. Comments and doc blocks are tokens of their own and are simply never examined. The count and the floor stay, for the reason every check here has them. Verified through the container path with php hidden from `command -v`, since static.sh runs on a Gitea runner that has no php: identical output either way. A 31st self-test assertion plants the same fixture. It is grade 2 evidence — a planted instance of exactly the failure the check describes, rather than an arbitrary trigger that only proves the check can go red at all. Two things that assertion caught about itself on its first run: The check's success line now says "parsed" where it said "examined", and the existing assertion on the count was still matching the old word. It went red rather than quietly matching nothing, which is why the wording is asserted at all. And `static.sh | grep -q` reports STATIC.SH's exit status under `set -o pipefail`, not grep's — and static.sh is expected to fail in that fixture, so the assertion read "not found" however the check behaved. The mirror image of the `go test | grep` trap already in the playbook. Output is captured to a variable first now, with the reason inline. Swept the kit for text belonging to other plugins while here, after cache found a comment in its copy describing security's tree: no NLSEC_/NLCAC_ references, and every path named in ci/ and tests/ either exists here or is WordPress core. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
kristianwindandClaude Opus 5
|
8121ca18dd |
Make the self-test run on Gitea, where it has been failing since it landed
ci/selftest.sh called `php` directly, four times. The Gitea runner image has no php — which is exactly why ci_php exists and says so in its own comment — so Static checks has been RED on Gitea since the self-test merged, while GitHub stayed green throughout. Found only by going to look at the other forge. Two of the four assertions failed there. The other two passed FOR THE WRONG REASON, and that is the worse half: they expect a non-zero exit, and `php: command not found` is a non-zero exit. A green tick for a check that never ran is the precise thing this script exists to prevent, reproduced inside it. Fixed by routing every call through ci_php, and by moving the fixtures inside the repo copy and staging them: ci_php's container gets its filesystem from `git ls-files`, so a fixture outside the repo or left unstaged is simply not there. selftest-empty/ carries a .keep file because git cannot track an empty directory, and an absent directory would make the check fail for a different reason than the one being asserted. Verified through the container path with php hidden from `command -v`, not just locally where php exists: all four verdicts identical either way, and the annotated path still resolves to includes/class-db.php. New 30th assertion: no ci/ script may call an interpreter directly. Its first version flagged five lines, all false: string literals mentioning php, and the calls inside the docker scripts in static.sh and site-test.sh, where php is correct because the container is a php image. Its second still conflated python3 with PyYAML. The runner HAS python3 and has NO php, and PyYAML is the only missing module — so bare python3 over the stdlib (build-zip.sh's zipfile, publish-release.sh's JSON readers) is correct and is not flagged. Measured, so the rule is not uniform. Controlled both ways: zero hits on this tree, and all four bare-php lines caught in the version that broke Gitea. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
kristianwindandClaude Opus 5
|
bd058566f5 |
Parse the workflows on Gitea too, and keep the compat matrix off its runner
Both found by the subscriptions session in its copy of this kit, and both reproduce here. They share the shape everything else in ci/ has had: the failure is invisible on GitHub, which is where anyone would be looking. 1. ci/workflows-in-sync.sh never parsed the workflow files on Gitea, and printed "Workflows: in sync." anyway. It skipped with a note when PyYAML was missing — and PyYAML is missing in exactly one place. Measured here rather than taken on report: `import yaml` on gitea/runner-images:ubuntu-latest raises ModuleNotFoundError. So the check was switched off on the single forge where an unparseable workflow produces no run at all, no failed run and no error to click on, which looks exactly like a runner that has not picked the job up. ci_python_yaml in ci/lib.sh now falls back to a container the way ci_php already did — alpine + py3-yaml, no network install, one apk call. There is no skip path left: workflows that cannot be parsed are a failure, not a note. Both routes verified to give byte-identical output. 2. compat.yml ran its five-cell matrix on Gitea for every PR touching ci/ or tests/. The `schedule` was gated to GitHub for exactly this reason and `pull_request` was forgotten. The subscriptions repo measured six jobs dying together with exit code 137 — SIGKILL, runner out of memory, two seconds in, without one error message in the log. Latent here only because this repo merges through GitHub and never opens a Gitea PR; one Gitea PR touching ci/ would have found it. New ci/check-workflows.py enforces both, called through ci_python_yaml. It parses every workflow and fails when a Gitea job with more than one matrix cell is not gated, and it is fatal when it parsed no file at all. 🔴 Its first version asked whether the `if:` CONTAINED the server_url test, and passed the very condition it exists to catch: `!= 'schedule' || server_url == ...` contains the gate while letting pull_request through. Substring matching is adjacent to the signal rather than the signal. It now compares the whole normalised condition against an allowlist, and was checked both ways in one run — red against the unfixed compat.yml naming the job and the cell count, green after. Three self-test assertions added, 29 total: an unparseable workflow fails, an ungated multi-cell matrix fails, and parsing zero files fails. Two notes from building it, both the same trap this kit keeps setting. The container path could not find check-workflows.py because ci_in_container tars `git ls-files` and the file was untracked. And the ungated-matrix fixture used | as a sed delimiter while the gate itself contains ||, so sed silently changed nothing; the assertion reported "could not stage the fixture" instead of passing, which is the only reason it was noticed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
kristianwindandClaude Opus 5
|
d0dcbd2c55 |
Assemble the self-test's fixture values instead of writing them literally
CI caught this on the previous commit's first push: the credential scan flagged
ci/selftest.sh itself, twice. The file is tracked, the scan reads every tracked
file, and the fixtures were credential-shaped literals sitting in it.
It passed locally because I ran ci/static.sh before `git add`, so selftest.sh
was still UNTRACKED and therefore invisible to a scan reading git ls-files.
That trap is documented four lines into the same file, about the fixtures it
plants. Knowing it and applying it to your own validation are two different
things, and the second is the one that counts.
Fixed by assembling every fixture value from pieces, which is stronger than any
exemption:
- A filename exemption would switch the scan off in ci/ and tests/, which is
exactly where a key gets pasted to test it. A sibling plugin reached the
same conclusion from the other side when its redactor fixture went red.
- A placeholder marker in the value would stop the fixture triggering the
check it exists to trigger.
- Split literals leave the WRITTEN file fully credential-shaped while the
SOURCE contains no matching string at all, so nothing needs excusing.
Verified both directions: the source no longer matches any pattern in the scan,
and each assembled value still does. static.sh green on the real tree with
selftest.sh tracked (45 files read, up from 44), self-test 26/26.
The pipeline worked as designed — static.sh runs before selftest.sh in the
static job, so the real tree was checked first and the failure named the file
and both rules. Nothing about this needed a human to notice.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
kristianwindandClaude Opus 5
|
0a58661864 |
Add ci/selftest.sh: prove the checks can still fail
Three rounds of fixes to this kit, and each round found the previous round's blind spot: a check that examined nothing, then a check that examined everything with a pattern that could not match, then a flag that made one rule skip binary files without a word. None of them were visible from a green run, and each was found by a sibling plugin rather than by anything here. So the battery stops being a shell loop somebody retypes and becomes a script CI runs. 26 assertions: 13 credential shapes that must be flagged, 6 lookalikes that must stay quiet, the direct-access count, the four text-domain verdicts, the annotated path being a real file, and a plugin.conf with its floors stripped failing to load. 🔴 It asserts on the thing that has CONSEQUENCES — the exit status of the script CI runs, and the specific failure message. Never a filename, a count, or a log line that merely sits near the signal. The previous ad-hoc battery asserted "the fixture's filename appears in the output", which passes on grep's own `binary file matches` diagnostic, because that diagnostic prints the filename in exactly the place a real failure would. A sibling plugin recorded two binary cases as PASS on that basis. The probe had the same defect as the check it was testing. The self-test was then checked to go red: making the AKIA pattern inert fails exactly one assertion, naming the shape. Its first run failed an assertion, and the fixture was at fault rather than the product. This script sources ci/lib.sh, so the floors are in the environment and a plain subshell inherits them — ci_load_conf then passed over a plugin.conf that does not define them. Fixed with an explicit unset, and the reason is inline so nobody weakens the assertion instead. A failing probe is evidence about the probe until you have looked at it. Wired into the static job in BOTH workflow roots, byte-identical: .gitea hides .github completely rather than merging it, so a step added to one root only would not run on Gitea and would look exactly like a runner that had not picked the job up. ci/workflows-in-sync.sh passes. Excluded from the release zip, and it plants nothing in the real tree — the fixtures go into a throwaway copy under mktemp, several being credential-shaped by definition. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |