CI / WordPress latest + WooCommerce latest on PHP 8.3 (pull_request) Successful in 1m18s
Compatibility / WP latest / WC latest / PHP 7.4 (pull_request) Skipped
Compatibility / WP latest / WC beta / PHP 8.3 (pull_request) Skipped
Compatibility / WP latest / WC latest / PHP 8.3 (pull_request) Skipped
Compatibility / WP nightly / WC latest / PHP 8.3 (pull_request) Skipped
Compatibility / WP latest / WC latest / PHP 8.4 (pull_request) Skipped
Compatibility / Keep the compatibility headers honest (pull_request) Skipped
CI / Static checks (pull_request) Successful in 4m38s
Takes woo-security-manager's current checker (sibling-hook boundary via FAMILY_PREFIXES, table-row scoping), keeping the floor in ci/plugin.conf. dist-files.sh now lists the same files in a git-less copy, which is what ci_php's php container is handed on the Gitea runner; before, check-docs failed there with exit 127. Co-Authored-By: Claude Code <noreply@anthropic.com>
619 lines
32 KiB
Bash
Executable File
619 lines
32 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Does ci/static.sh actually FAIL when it should?
|
|
#
|
|
# 🔴 Why this exists, and why it asserts what it asserts.
|
|
#
|
|
# Every check in this kit has now been wrong twice in the same way: it printed
|
|
# a reassuring line having examined nothing, and later it examined everything
|
|
# with a pattern that could not match. Both are invisible from the passing
|
|
# side, because a check that finds nothing and a check that CANNOT find
|
|
# anything produce identical output. The only defence is to plant a thing that
|
|
# must be found and watch the build go red.
|
|
#
|
|
# 🔴 And the probe itself has the same failure mode. An earlier version of this
|
|
# battery asserted "the fixture's filename appears somewhere in the output" —
|
|
# which passes on grep's own `binary file matches` diagnostic, because that
|
|
# diagnostic prints the filename in exactly the place a real failure would.
|
|
# A sibling plugin recorded two binary cases as PASS on that basis and was one
|
|
# message away from reporting the shape covered.
|
|
#
|
|
# So: assert on the thing that has CONSEQUENCES. The exit status of the script
|
|
# CI runs, and the specific failure message — never a filename, a count, or a
|
|
# log line that merely sits near the signal.
|
|
set -uo pipefail
|
|
cd "$(dirname "$0")/.."
|
|
|
|
# Loaded from the REAL tree, before the throwaway copy has its floors stripped
|
|
# by the last assertion below.
|
|
. ci/lib.sh
|
|
ci_load_conf || { echo "ci/plugin.conf will not load — fix that before running the self-test." >&2; exit 1; }
|
|
|
|
pass=0; fail=0
|
|
ok() { printf ' \033[32m✓\033[0m %s\n' "$1"; pass=$((pass + 1)); }
|
|
bad() { printf ' \033[31m✗\033[0m %s\n' "$1"; fail=$((fail + 1)); }
|
|
|
|
# A throwaway copy: these fixtures must never be committable to the real tree,
|
|
# and several of them are credential-shaped by definition.
|
|
work=$(mktemp -d -t wpci-selftest-XXXXXX)
|
|
trap 'rm -rf "$work"' EXIT
|
|
cp -a . "$work/repo"
|
|
cd "$work/repo"
|
|
|
|
# ---------------------------------------------------------------- credentials
|
|
# One fixture per shape the scan CLAIMS to catch, plus lookalikes that must
|
|
# stay quiet. An unfilled placeholder is not optional decoration: it is the one
|
|
# string a settings screen is guaranteed to contain, so a rule without that
|
|
# exclusion cries wolf on day one and gets commented out.
|
|
# 🔴 Every fixture value below is ASSEMBLED from pieces, never written as a
|
|
# literal. This script is itself a tracked file, so ci/static.sh scans it — and
|
|
# a credential-shaped literal here makes the real tree go red. It did: CI
|
|
# flagged ci/selftest.sh on this file's first push.
|
|
#
|
|
# Assembling is better than any exemption. A filename exemption would switch
|
|
# the scan off in ci/ and tests/, which is exactly where a key gets pasted to
|
|
# test it. A placeholder marker in the value would stop the fixture triggering
|
|
# the check it exists to trigger. Split literals leave the written file fully
|
|
# credential-shaped while the source contains no matching string at all, so
|
|
# nothing has to be excused.
|
|
akia="AKIA""IOSFODNN7EXAMPLE"
|
|
pem="-----BEGIN RSA PRIVATE"" KEY-----"; pem="${pem/ / }"
|
|
skant="sk-ant-""api03-AAAAAAAAAAAAAAAAAAAA"
|
|
ghp="ghp_""AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
|
|
gho="gho_""BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB"
|
|
pat="github_pat_""$(printf 'C%.0s' $(seq 1 60))"
|
|
sk="sk-""DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD"
|
|
opaque="h8Kd92MsLp01QwErTyUiOp"
|
|
|
|
printf 'aws = %s\n' "$akia" > probe-aws.txt
|
|
printf -- '%s\nMIIEfake\n' "$pem" > probe-pem.txt
|
|
printf 'k = %s\n' "$skant" > probe-skant.txt
|
|
printf 'k = %s\n' "$ghp" > probe-ghp.txt
|
|
printf 'k = %s\n' "$gho" > probe-gho.txt
|
|
printf 'k = %s\n' "$pat" > probe-pat.txt
|
|
printf 'k = %s\n' "$sk" > probe-sk.txt
|
|
printf "'api_key' => '%s',\n" "$opaque" > probe-array.txt
|
|
printf "define( 'WC_EDM_SECRET', '%s' );\n" "$opaque" > probe-define.txt
|
|
|
|
# Directory coverage. .distignore removes all three from the zip, and a scan
|
|
# reading the SHIPPED list instead of the tracked one therefore misses every CI
|
|
# script, workflow and test fixture — which is exactly where a key gets pasted
|
|
# to test it. A sibling plugin was missing 36 of 73 files this way.
|
|
mkdir -p ci tests .github/workflows
|
|
printf 'k = %s\n' "$skant" > ci/probe-dir.txt
|
|
printf 'k = %s\n' "$skant" > tests/probe-dir.txt
|
|
printf 'k = %s\n' "$skant" > .github/workflows/probe-dir.txt
|
|
|
|
# A key inside a file grep calls binary. `grep -I` skips these SILENTLY, so the
|
|
# value rule must not use it; the build going red is the right direction to be
|
|
# wrong in. This fixture exists because that flag was added by accident and
|
|
# nothing went red to announce it.
|
|
printf 'PNG\x00\x00\x00fake%s\nMIIE\n' "$pem" > probe-binary.bin
|
|
|
|
# Controls need no assembling: each is either a placeholder the scan excludes
|
|
# by design, or ordinary code that was never credential-shaped. If one of these
|
|
# ever makes the real tree go red, that is a false positive worth knowing about.
|
|
printf 'sha = 9f2b7c1e4a8d6b3f0e5c2a9d7b4f1e8c\nAKIA\n' > control-hash.txt
|
|
printf "'api_key' => 'REPLACE_WITH_YOUR_API_KEY',\n" > control-placeholder.txt
|
|
printf "'password' => 'xxxxxxxxxxxxxxxxxxxxxxxx',\n" > control-masked.txt
|
|
printf "'api_key' => '',\n'token' => 'short',\n" > control-empty.txt
|
|
printf "wp_die( 'token', 'a long sentence that follows a comma' );\n" > control-comma.txt
|
|
printf "define( 'WC_EDM_TOKEN', 'TODO_FILL_THIS_IN_LATER_XX' );\n" > control-tododefine.txt
|
|
|
|
# -N: an UNTRACKED fixture is invisible to a scan reading `git ls-files`, so it
|
|
# passes clean and looks exactly like the scan working.
|
|
git add -N probe-*.txt probe-binary.bin ci/probe-dir.txt tests/probe-dir.txt \
|
|
.github/workflows/probe-dir.txt control-*.txt >/dev/null
|
|
|
|
echo "→ ci/static.sh must FAIL with credentials planted"
|
|
out=$(./ci/static.sh 2>&1); code=$?
|
|
[ "$code" != 0 ] && ok "exit status is $code" || bad "exit status is 0 — the build would have gone green"
|
|
|
|
# The message, not the filename. grep's own "binary file matches" diagnostic
|
|
# names the file too, in the same stream, which is what made the weak version
|
|
# of this assertion pass on a check that never fired.
|
|
fired() { echo "$out" | grep -q "^.*$1 \(looks like it contains a credential\|assigns a long opaque value\)"; }
|
|
|
|
for f in probe-aws.txt probe-pem.txt probe-skant.txt probe-ghp.txt probe-gho.txt \
|
|
probe-pat.txt probe-sk.txt probe-array.txt probe-define.txt \
|
|
ci/probe-dir.txt tests/probe-dir.txt .github/workflows/probe-dir.txt \
|
|
probe-binary.bin; do
|
|
fired "$f" && ok "flagged $f" || bad "NOT FLAGGED: $f — the pattern is inert for this shape"
|
|
done
|
|
|
|
for f in control-hash.txt control-placeholder.txt control-masked.txt \
|
|
control-empty.txt control-comma.txt control-tododefine.txt; do
|
|
fired "$f" && bad "FALSE POSITIVE on $f — a check that cries wolf gets commented out" \
|
|
|| ok "ignored $f"
|
|
done
|
|
|
|
# The guard check must report a plausible number rather than a bare adjective.
|
|
echo "$out" | grep -qE '^ [0-9]+ shipped PHP files parsed' \
|
|
&& ok "the direct-access check reports a count" \
|
|
|| bad "the direct-access check printed no count — it cannot tell 13 files from 0"
|
|
|
|
# --------------------------------------------------- direct-access guard
|
|
# 🔴 Grade 2 evidence: a planted instance of exactly the failure the check
|
|
# describes, not an arbitrary trigger. The grep version of this check passed
|
|
# this fixture — the words are present, in a comment, and there is no guard.
|
|
# It reported "every one guarded" over a file that runs for anyone who fetches
|
|
# it directly.
|
|
echo
|
|
echo "→ the direct-access check reads structure, not text"
|
|
cat > includes/class-selftest-unguarded.php <<'FIXTURE'
|
|
<?php
|
|
// This file is safe because defined( 'ABSPATH' ) is checked elsewhere.
|
|
// It is not. There is no guard anywhere below this line.
|
|
echo 'reachable';
|
|
FIXTURE
|
|
git add -N includes/class-selftest-unguarded.php >/dev/null
|
|
# Captured first, NOT piped into grep. This script runs under `set -o
|
|
# pipefail`, and static.sh is expected to exit non-zero here — so
|
|
# `static.sh | grep -q` would report STATIC.SH's status, not grep's, and the
|
|
# assertion would read "not found" however the check behaved. The mirror image
|
|
# of the `go test | grep` trap, and it cost this assertion one red run.
|
|
guard_out=$(./ci/static.sh 2>&1 || true)
|
|
if printf '%s' "$guard_out" | grep -q 'class-selftest-unguarded.php,line=1::No guard'; then
|
|
ok "a file mentioning the guard only in a comment is still flagged"
|
|
else
|
|
bad "an unguarded file passed because the words appear in a comment"
|
|
fi
|
|
rm -f includes/class-selftest-unguarded.php
|
|
git rm -q --cached includes/class-selftest-unguarded.php >/dev/null 2>&1 || true
|
|
|
|
# ---------------------------------------------------------------- text domain
|
|
# 🔴 ci_php, never a bare `php`. The Gitea runner image has NO php — that is
|
|
# why ci_php exists and says so — and this section originally called `php`
|
|
# directly. Two of its four assertions then failed there, and the other two
|
|
# PASSED FOR THE WRONG REASON: they expect a non-zero exit, and `php: command
|
|
# not found` is a non-zero exit. A green tick for a check that never ran is the
|
|
# exact thing this whole script exists to prevent, reproduced inside it.
|
|
#
|
|
# ci_php falls back to a container whose filesystem comes from `git ls-files`,
|
|
# so the fixtures live INSIDE this copy of the repo and are staged. A fixture
|
|
# outside the repo, or an unstaged one, is simply not there when it matters.
|
|
cd "$work/repo"
|
|
echo
|
|
echo "→ ci/check-textdomain.php verdicts"
|
|
|
|
# Not a literally empty directory: git cannot track one, so it would not reach
|
|
# the container at all and the check would fail for a different reason than the
|
|
# one being asserted.
|
|
mkdir -p selftest-empty selftest-nostrings selftest-dotted/includes
|
|
printf 'no php in here\n' > selftest-empty/.keep
|
|
printf '<?php\n$x = 1;\n' > selftest-nostrings/plugin.php
|
|
printf "<?php\necho __( 'x' );\n" > selftest-dotted/includes/class-db.php
|
|
git add -N selftest-empty/.keep selftest-nostrings/plugin.php \
|
|
selftest-dotted/includes/class-db.php >/dev/null
|
|
|
|
ci_php ci/check-textdomain.php --min=0 "$TEXT_DOMAIN" selftest-empty >/dev/null 2>&1 \
|
|
&& bad "an empty scan root passed — the root verdict must not be configurable away" \
|
|
|| ok "an empty scan root is fatal even with --min=0"
|
|
|
|
ci_php ci/check-textdomain.php --min=999999 "$TEXT_DOMAIN" . >/dev/null 2>&1 \
|
|
&& bad "a --min above the real count passed" \
|
|
|| ok "a --min above the real count is fatal"
|
|
|
|
ci_php ci/check-textdomain.php "$TEXT_DOMAIN" selftest-nostrings >/dev/null 2>&1 \
|
|
&& ok "zero calls without --min warns rather than failing" \
|
|
|| bad "zero calls without --min failed — a small plugin may legitimately have none"
|
|
|
|
# The annotated path must be a file that exists. str_replace( $root, '', $path )
|
|
# with a root of "." strips every dot and yields "class-dbphp".
|
|
ann=$(ci_php ci/check-textdomain.php "$TEXT_DOMAIN" selftest-dotted 2>&1 \
|
|
| sed -n 's/.*file=\([^,]*\),.*/\1/p' | head -1)
|
|
[ -n "$ann" ] && [ -f "selftest-dotted/$ann" ] \
|
|
&& ok "the annotated path exists on disk: $ann" \
|
|
|| bad "the annotated path is not a file: '$ann'"
|
|
|
|
# ------------------------------------------------------- the clone's git identity
|
|
# 🔴 Grade 2 evidence: row 1 is the exact line ci/bump-tested-up-to.sh carried
|
|
# until 2026-09-25, not an arbitrary `git config`. That line writes .git/config,
|
|
# so every later commit in the clone — a release, by a person — is authored by
|
|
# the robot with no trailer. Six such commits in one sibling plugin, two in
|
|
# another. git warns about an identity it cannot FIND and never about a wrong
|
|
# one, so nothing in the repository ever says so.
|
|
#
|
|
# The controls are the larger half here, and three of them were live faults in
|
|
# earlier versions of this check: `--[a-z-]+` as the scope group reads `--unset`
|
|
# as a scope and flags the REPAIR, and without a trailing-value requirement a
|
|
# `$(git config user.name)` READ is a violation. A build that fails on its own
|
|
# remedy is how a correct check gets deleted — the failure accuses the check,
|
|
# and the next person believes it.
|
|
#
|
|
# 🔴 The fixtures are assembled through $gc rather than written out, because
|
|
# THIS file is a ci/*.sh file and the check reads every one of them. Spelled in
|
|
# full, its own rows match: the first version of this block reported six false
|
|
# positives and no count, while the same rows measured standalone were 16/16
|
|
# correct. A failing probe is evidence about the probe until you have looked at
|
|
# it — and the labels have to dodge it too, which is why they name the shape
|
|
# instead of quoting the line. Identical to the credential fixtures three
|
|
# hundred lines up, one file over.
|
|
#
|
|
# The plant goes into ci/build-zip.sh because nothing here SOURCES it: static.sh
|
|
# and this script source ci/lib.sh only. A plant in a sourced file is not inert
|
|
# text — it runs, sets the clone's identity for real, and the commit fixing the
|
|
# fault is authored by the plant. nolimit-seo lost a run to exactly that. So the
|
|
# last assertion checks the clone rather than the output.
|
|
echo
|
|
echo "→ ci/static.sh must catch a written identity and ignore reads and repairs"
|
|
cd "$work/repo"
|
|
cp ci/build-zip.sh "$work/build-zip.orig"
|
|
|
|
identity_row() { # $1 = RED|green (want), $2 = label, $3 = line to plant
|
|
cp "$work/build-zip.orig" ci/build-zip.sh
|
|
printf '%s\n' "$3" >> ci/build-zip.sh
|
|
# The plant must BE there before any verdict is believed. A printf that went
|
|
# somewhere else leaves the control state, and then every row passes while
|
|
# proving nothing — and the all-green rows are exactly the ones whose zeroes
|
|
# you want to believe.
|
|
if [ "$(grep -cF "$3" ci/build-zip.sh)" -lt 1 ]; then
|
|
bad "could not stage the git-identity fixture: $2"
|
|
return
|
|
fi
|
|
row_out=$(./ci/static.sh 2>&1 || true)
|
|
# The stable half of the message. The wider phrasing moved from "A ci/ script
|
|
# sets …" to "A tracked file writes …" when the scan was widened, and these
|
|
# rows all reported NOT FLAGGED while the check was working perfectly — an
|
|
# assertion on a log line is coupled to its wording.
|
|
if printf '%s' "$row_out" | grep -q 'git identity that outlives it'; then
|
|
got=RED
|
|
else
|
|
got=green
|
|
fi
|
|
if [ "$got" = "$1" ]; then
|
|
ok "$2 -> $got"
|
|
elif [ "$1" = RED ]; then
|
|
bad "NOT FLAGGED: $2 — the clone keeps the robot's name"
|
|
else
|
|
bad "FALSE POSITIVE: $2 — the check goes red on something it must allow"
|
|
fi
|
|
}
|
|
|
|
gc="git config" # never spelled out below: see the note above
|
|
identity_row RED 'WRITE no flag, quoted value' "$gc user.name \"compatibility check\""
|
|
identity_row RED 'WRITE --local scope' "$gc --local user.name \"x\""
|
|
identity_row RED 'WRITE --global scope' "$gc --global user.email \"x@y\""
|
|
identity_row RED 'WRITE --system scope' "$gc --system user.name \"x\""
|
|
identity_row RED 'WRITE unquoted value' "$gc user.email noreply@localhost"
|
|
identity_row green 'REPAIR --local --unset' "$gc --local --unset user.name"
|
|
identity_row green 'REPAIR --unset, no scope' "$gc --unset user.email"
|
|
identity_row green 'READ inside a command substitution' "echo \"\$($gc user.name)\""
|
|
identity_row green 'FIX git -c ... commit' 'git -c user.name="x" -c user.email=y commit -m z'
|
|
identity_row green 'other a non-identity key' "$gc core.autocrlf false"
|
|
identity_row green 'prose a commented-out mention' "# $gc user.name \"x\" is the fault"
|
|
cp "$work/build-zip.orig" ci/build-zip.sh
|
|
|
|
# A violation OUTSIDE ci/*.sh. This is the row the old scope missed entirely:
|
|
# a workflow step writes an identity into the runner's clone just as
|
|
# permanently, and the glob the check started with could not see it.
|
|
cp "$work/build-zip.orig" ci/build-zip.sh
|
|
wf=.github/workflows/selftest-identity.yml
|
|
printf 'name: x\njobs:\n y:\n steps:\n - run: %s user.email "x@y"\n' "$gc" > "$wf"
|
|
git add -N "$wf" >/dev/null # untracked files are invisible to git ls-files
|
|
wf_out=$(./ci/static.sh 2>&1 || true)
|
|
printf '%s' "$wf_out" | grep -q 'writes a git identity that outlives it' \
|
|
&& ok "flagged a write outside ci/ (a workflow step)" \
|
|
|| bad "NOT FLAGGED: a workflow step writing an identity — the scan is still scoped to ci/"
|
|
rm -f "$wf"
|
|
git rm --cached -q --ignore-unmatch "$wf" >/dev/null 2>&1
|
|
|
|
# 🔴 A drift check on the FORM, and only that: `if … | xargs -0 grep …; then`
|
|
# has a false green in it, because xargs exits 123 when any batch found nothing
|
|
# and `if` reads 123 as no-match. It is invisible while the list fits one exec,
|
|
# which is why nothing here can measure it — this tree's 48 files are one batch.
|
|
# So this asserts the shape is gone rather than that the bug is absent, and it
|
|
# cannot tell you the replacement is correct. scan_tracked greps per file and
|
|
# reads its verdict from the output, which is what makes batching irrelevant.
|
|
# ^[^#]* — the first version of this matched the COMMENT in static.sh that
|
|
# quotes the bad form in order to explain it, which is the stale-docblock trap
|
|
# in its own right: the paragraph documenting a rule is text the rule applies to.
|
|
if grep -nE '^[^#]*(if|elif).*\| *xargs.*grep' ci/static.sh; then
|
|
bad "a check above consumes xargs' exit status — 123 on a partial match reads as clean"
|
|
else
|
|
ok "no check in ci/static.sh consumes an xargs exit status"
|
|
fi
|
|
|
|
# And the passing line must carry a number: "every script is clean" reads the
|
|
# same over zero files as over eleven.
|
|
clean_out=$(./ci/static.sh 2>&1 || true)
|
|
printf '%s' "$clean_out" | grep -qE "^ [0-9]+ tracked files leave the clone's git identity alone" \
|
|
&& ok "the git-identity check reports how many files it read" \
|
|
|| bad "the git-identity check printed no count — it cannot tell 48 files from 0"
|
|
|
|
# The fixtures are lines that SET an identity. If any of them executed rather
|
|
# than being read, this clone now has one — which would also mean every verdict
|
|
# above was measured against a tree nobody intended.
|
|
if git config --local --get-regexp '^user\.' >/dev/null 2>&1; then
|
|
bad "a git-identity fixture RAN — the clone has a local identity and the rows above are void"
|
|
else
|
|
ok "no fixture executed: the clone still has no local identity"
|
|
fi
|
|
|
|
# ------------------------------------------------------ compat values in two files
|
|
# 🔴 The two currency checks read each value from the file that value belongs
|
|
# in — correct, and therefore blind to a SECOND copy in the other file. The
|
|
# plugin header said WC 11.1.2 and README.txt said 11.1.0 for four releases
|
|
# with that step printing green. Each reading was right; the pair was never
|
|
# compared. So these rows are about our own two copies agreeing, which is a
|
|
# fact about this tree and fatal, not about being up to date, which is advisory.
|
|
echo
|
|
echo "→ a compatibility value in two files must agree"
|
|
cd "$work/repo"
|
|
cp README.txt "$work/README.orig"
|
|
compat_row() { # $1 = field, $2 = value to plant in the readme
|
|
cp "$work/README.orig" README.txt
|
|
sed -i "s/^$1: .*/$1: $2/" README.txt
|
|
if [ "$(grep -c "^$1: $2\$" README.txt)" -lt 1 ]; then
|
|
bad "could not stage the compat fixture for '$1'"
|
|
return
|
|
fi
|
|
compat_out=$(./ci/static.sh 2>&1 || true)
|
|
if printf '%s' "$compat_out" | grep -q "'$1' is .* in the plugin header and"; then
|
|
ok "flagged a drifted '$1' ($2 in the readme)"
|
|
else
|
|
bad "NOT FLAGGED: '$1' drifted to $2 in the readme and nothing compared the two copies"
|
|
fi
|
|
}
|
|
compat_row 'WC tested up to' '11.0.0'
|
|
compat_row 'Requires PHP' '8.0'
|
|
compat_row 'Requires at least' '5.0'
|
|
cp "$work/README.orig" README.txt
|
|
|
|
# The pass must say how many pairs it compared. A tree where the readme carries
|
|
# none of these fields has nothing to disagree and prints the same silence as
|
|
# one where every pair matches.
|
|
compat_clean=$(./ci/static.sh 2>&1 || true)
|
|
pairs=$(printf '%s' "$compat_clean" | grep -c "^ both files say ")
|
|
if [ "$pairs" -ge 3 ]; then
|
|
ok "$pairs value pairs compared across the two files"
|
|
else
|
|
bad "only $pairs pairs compared — the agreement check has almost nothing to look at"
|
|
fi
|
|
|
|
# ------------------------------------------------------------------------ docs
|
|
# 🔴 ci/check-docs.php has four failure modes and one subtle exclusion, and the
|
|
# exclusion is the one that would regress unnoticed: it strips comments before
|
|
# reading the source. Without that, a CONTRACT.md example shown in a docblock
|
|
# reads as a real hook call — so a file documenting a contract is reported as
|
|
# breaking it. Nolimit Security proved all four by hand in one sitting; four
|
|
# commands in a commit message are not an assertion that runs again next month.
|
|
#
|
|
# Unlike the git-identity rows, these fixtures cannot trip the checker over
|
|
# themselves: it reads only what ci/dist-files.sh ships, and ci/ and tests/ are
|
|
# not in that list. Verified rather than assumed — dist-files.sh returns no
|
|
# ci/ path at all.
|
|
echo
|
|
echo "→ ci/check-docs.php must catch a document that stopped describing this tree"
|
|
cd "$work/repo"
|
|
docs_verdict() { ( . ci/lib.sh && ci_load_conf && ci_php ci/check-docs.php . ) 2>&1 || true; }
|
|
|
|
# 1. a hook in the source that no document mentions.
|
|
cp includes/class-helpers.php "$work/helpers.orig"
|
|
printf '\n// Added by ci/selftest.sh.\nfunction wc_edm_selftest_probe() {\n\treturn apply_filters( %s, true );\n}\n' \
|
|
"'wc_edm_selftest_undocumented'" >> includes/class-helpers.php
|
|
if [ "$(grep -c 'wc_edm_selftest_undocumented' includes/class-helpers.php)" -lt 1 ]; then
|
|
bad "could not stage the undocumented-hook fixture"
|
|
else
|
|
printf '%s' "$(docs_verdict)" | grep -q 'not in docs/CONTRACT.md: wc_edm_selftest_undocumented' \
|
|
&& ok "flagged a hook the source offers and CONTRACT.md does not mention" \
|
|
|| bad "NOT FLAGGED: an undocumented hook — a sibling cannot discover it without reading our source"
|
|
fi
|
|
cp "$work/helpers.orig" includes/class-helpers.php
|
|
|
|
# 1b. the same fault for a hook a SIBLING owns and we answer. Before
|
|
# FAMILY_PREFIXES this fell into "core, not ours to document" and stayed green.
|
|
printf '\n// Added by ci/selftest.sh.\nadd_filter( %s, %s );\n' \
|
|
"'nlsec_selftest_sibling'" "'__return_true'" >> includes/class-helpers.php
|
|
if [ "$(grep -c 'nlsec_selftest_sibling' includes/class-helpers.php)" -lt 1 ]; then
|
|
bad "could not stage the sibling-hook fixture"
|
|
else
|
|
printf '%s' "$(docs_verdict)" | grep -q 'not in docs/CONTRACT.md: .*nlsec_selftest_sibling' \
|
|
&& ok "flagged a sibling-owned hook we answer and CONTRACT.md does not mention" \
|
|
|| bad "NOT FLAGGED: an undocumented sibling hook — the whole boundary to another plugin is invisible"
|
|
fi
|
|
cp "$work/helpers.orig" includes/class-helpers.php
|
|
|
|
# 2. 🔴 the control that matters: the same name, in a COMMENT only. The checker
|
|
# must stay GREEN, and this asserts the specific success line rather than an
|
|
# exit code — a refusal, a missing file or a fatal all exit non-zero too, and
|
|
# any of them would read as "the comment was ignored".
|
|
printf '\n// apply_filters( %s, true ) — an example in prose, not a call.\n' \
|
|
"'wc_edm_selftest_in_a_comment'" >> includes/class-helpers.php
|
|
if [ "$(grep -c 'wc_edm_selftest_in_a_comment' includes/class-helpers.php)" -lt 1 ]; then
|
|
bad "could not stage the comment-only fixture"
|
|
else
|
|
comment_out=$(docs_verdict)
|
|
if printf '%s' "$comment_out" | grep -qE 'all [0-9]+ in docs/CONTRACT.md'; then
|
|
ok "a hook name in a comment is not a hook call (comment stripping still works)"
|
|
else
|
|
printf '%s\n' "$comment_out" | sed 's/^/ /'
|
|
bad "a hook name in a COMMENT was read as a call — a docblock documenting a contract now breaks the build"
|
|
fi
|
|
fi
|
|
cp "$work/helpers.orig" includes/class-helpers.php
|
|
|
|
# 3. the other direction: a row for a hook that no longer exists. Silent without
|
|
# this check, and the row reads as deliberate.
|
|
cp docs/CONTRACT.md "$work/contract.orig"
|
|
printf '\n| `wc_edm_selftest_renamed_away` | `bool $x` | `true` | nothing |\n' >> docs/CONTRACT.md
|
|
if [ "$(grep -c 'wc_edm_selftest_renamed_away' docs/CONTRACT.md)" -lt 1 ]; then
|
|
bad "could not stage the stale-row fixture"
|
|
else
|
|
printf '%s' "$(docs_verdict)" | grep -q 'wc_edm_selftest_renamed_away' \
|
|
&& ok "flagged a CONTRACT.md row whose hook is not in the source" \
|
|
|| bad "NOT FLAGGED: a stale CONTRACT.md row — the document describes a tree that is not this one"
|
|
fi
|
|
cp "$work/contract.orig" docs/CONTRACT.md
|
|
|
|
# 4. a document emptied to its heading. "Exists" passes over a stub.
|
|
cp docs/SETUP.md "$work/setup.orig"
|
|
printf '# Setting it up\n' > docs/SETUP.md
|
|
printf '%s' "$(docs_verdict)" | grep -q 'docs/SETUP.md has 1 lines' \
|
|
&& ok "flagged a document emptied to its heading" \
|
|
|| bad "NOT FLAGGED: a one-line SETUP.md — 'the file exists' is not 'the file says anything'"
|
|
cp "$work/setup.orig" docs/SETUP.md
|
|
|
|
# 5. the version drifting away from the header.
|
|
sed -i 's/^Version 1\.[0-9]*\.[0-9]*/Version 9.9.9/' docs/README.md
|
|
if [ "$(grep -c '^Version 9.9.9' docs/README.md)" -lt 1 ]; then
|
|
bad "could not stage the version-drift fixture"
|
|
else
|
|
printf '%s' "$(docs_verdict)" | grep -q 'does not mention version' \
|
|
&& ok "flagged docs naming a version the header does not" \
|
|
|| bad "NOT FLAGGED: docs at 9.9.9 against a different header — the docs describe another release"
|
|
fi
|
|
git checkout -q -- docs/README.md 2>/dev/null || cp "$work/README.md.orig" docs/README.md
|
|
|
|
# 5b. the file list must be the same without git. ci_php runs check-docs.php in
|
|
# a php container that is handed the tracked files and has no git; dist-files.sh
|
|
# used to exit 127 there, so the docs check failed on the Gitea runner alone.
|
|
nogit=$(mktemp -d)
|
|
git ls-files -z | tar --null -T - -cf - | tar -xf - -C "$nogit"
|
|
if [ -e "$nogit/.git" ]; then
|
|
bad "could not stage the git-less copy"
|
|
else
|
|
with_git=$(./ci/dist-files.sh)
|
|
without_git=$(cd "$nogit" && ./ci/dist-files.sh 2>&1 || echo "exit $?")
|
|
if [ -n "$with_git" ] && [ "$with_git" = "$without_git" ]; then
|
|
ok "dist-files.sh lists the same $(printf '%s\n' "$with_git" | wc -l | tr -d ' ') files with and without git"
|
|
else
|
|
bad "dist-files.sh differs without git — the docs check goes red on the Gitea runner only"
|
|
fi
|
|
fi
|
|
rm -rf "$nogit"
|
|
|
|
# 6. both conf values must refuse rather than default. An unset floor parses to
|
|
# 0 and every comparison against it passes, and a guessed prefix classifies
|
|
# every hook in the tree as somebody else's.
|
|
cp ci/plugin.conf "$work/conf.orig"
|
|
for key in HOOK_PREFIX MIN_SHIPPED_PHP_FILES FAMILY_PREFIXES; do
|
|
grep -v "^$key=" "$work/conf.orig" > ci/plugin.conf
|
|
if [ "$(grep -c "^$key=" ci/plugin.conf)" -ne 0 ]; then
|
|
bad "could not stage the missing-$key fixture"
|
|
continue
|
|
fi
|
|
printf '%s' "$(docs_verdict)" | grep -q "No .*$key" \
|
|
&& ok "refuses to run with no $key rather than guessing" \
|
|
|| bad "ran without $key — a guessed prefix or a zero floor passes over everything"
|
|
done
|
|
cp "$work/conf.orig" ci/plugin.conf
|
|
|
|
# And the pass must carry both numbers. "The documentation matches" over zero
|
|
# hooks in zero files is the same green as over nine in fourteen.
|
|
docs_clean=$(docs_verdict)
|
|
printf '%s' "$docs_clean" | grep -qE '^ [0-9]+ hooks in [0-9]+ shipped PHP files, all [0-9]+ in docs/CONTRACT.md' \
|
|
&& ok "the docs check reports how many hooks and how many files it read" \
|
|
|| bad "the docs check printed no counts — it cannot tell 9 hooks in 14 files from 0 in 0"
|
|
|
|
# ------------------------------------------------------------------- plugin.conf
|
|
echo
|
|
echo "→ the floors must be required, not defaulted"
|
|
cd "$work/repo"
|
|
grep -v '^MIN_' ci/plugin.conf > ci/plugin.conf.tmp && mv ci/plugin.conf.tmp ci/plugin.conf
|
|
# The `unset` is load-bearing and this assertion failed without it. This script
|
|
# sources ci/lib.sh itself, so the floors are already in the environment and a
|
|
# plain subshell INHERITS them — ci_load_conf then finds them set and passes
|
|
# over a plugin.conf that does not define them. The fixture was contaminated,
|
|
# not the check: a failing probe is evidence about the probe until you have
|
|
# looked at it. The real static.sh runs in a fresh shell where they are unset.
|
|
if ( unset MIN_SHIPPED_PHP_FILES MIN_TRANSLATION_CALLS
|
|
. ci/lib.sh && ci_load_conf ) >/dev/null 2>&1; then
|
|
bad "a plugin.conf with no floors loaded — an unset floor is a no-op comparison"
|
|
else
|
|
ok "a plugin.conf with no floors fails to load"
|
|
fi
|
|
|
|
# ------------------------------------------------------------------ workflows
|
|
# Both of these are silent where they happen. An unparseable workflow produces
|
|
# NO RUN on Gitea, and a multi-cell matrix that is not gated lands on one
|
|
# self-hosted runner as exit code 137 with an empty log. Neither shows up on
|
|
# GitHub, which is where anyone would be looking.
|
|
echo
|
|
echo "→ ci/workflows-in-sync.sh must catch what only breaks on Gitea"
|
|
cd "$work/repo"
|
|
cp -a .github/workflows "$work/wf-github-backup"
|
|
cp -a .gitea/workflows "$work/wf-gitea-backup"
|
|
restore_workflows() {
|
|
rm -rf .github/workflows .gitea/workflows
|
|
cp -a "$work/wf-github-backup" .github/workflows
|
|
cp -a "$work/wf-gitea-backup" .gitea/workflows
|
|
}
|
|
|
|
# Corrupt BOTH roots identically, so this fails on the PARSE rather than on the
|
|
# byte-identity rule that would otherwise fire first.
|
|
printf '\n this: [is not\n' >> .github/workflows/ci.yml
|
|
printf '\n this: [is not\n' >> .gitea/workflows/ci.yml
|
|
./ci/workflows-in-sync.sh >/dev/null 2>&1 \
|
|
&& bad "an unparseable workflow passed — on Gitea it would simply never run" \
|
|
|| ok "an unparseable workflow fails the build"
|
|
restore_workflows
|
|
|
|
# The gate that keeps a five-cell matrix off the single self-hosted runner.
|
|
# @ as the delimiter, not |: the gate itself contains `||`, and with | as the
|
|
# delimiter sed parses the fixture into nonsense and silently changes nothing —
|
|
# which is how this assertion first ran against an unmodified file. It reported
|
|
# "could not stage the fixture" rather than passing, which is the whole reason
|
|
# the staging is checked before the assertion is made.
|
|
sed -i "s@^ if: github.event_name == 'workflow_dispatch'.*@ if: always()@" \
|
|
.github/workflows/compat.yml .gitea/workflows/compat.yml
|
|
if grep -q 'if: always()' .gitea/workflows/compat.yml; then
|
|
./ci/workflows-in-sync.sh >/dev/null 2>&1 \
|
|
&& bad "an ungated multi-cell matrix passed — it lands on the Gitea runner as exit 137" \
|
|
|| ok "an ungated multi-cell matrix fails the build"
|
|
else
|
|
bad "could not stage the ungated-matrix fixture — the gate line did not match"
|
|
fi
|
|
restore_workflows
|
|
|
|
# Parsing nothing is never a legitimate result.
|
|
ci_python_yaml ci/check-workflows.py >/dev/null 2>&1 \
|
|
&& bad "check-workflows.py passed having parsed no file at all" \
|
|
|| ok "parsing zero workflow files is fatal"
|
|
|
|
# ------------------------------------------------- no bare interpreter calls
|
|
# The bug that cost a green Gitea build: this script called `php` directly.
|
|
# The Gitea runner image has no php and no PyYAML, so a bare call there is
|
|
# "command not found" — which is a NON-ZERO exit, and therefore indistinguishable
|
|
# from the failure two of these assertions were written to expect. They went
|
|
# green having run nothing.
|
|
#
|
|
# ci_php and ci_python_yaml exist for this and say so in their comments. lib.sh
|
|
# is excluded because it is where the real calls live.
|
|
#
|
|
# Measured, so the rule is not uniform: the runner HAS python3 and has NO php,
|
|
# and PyYAML is the only missing module. A bare python3 over the stdlib is
|
|
# therefore correct and is not flagged.
|
|
echo
|
|
echo "→ no ci/ script may call php or python3 directly"
|
|
# Narrow on purpose. A broad "the word php appears" fires on every string
|
|
# literal that mentions php, and on the calls INSIDE the docker scripts in
|
|
# static.sh and site-test.sh, where php is exactly right because the container
|
|
# is a php image. A check that cries wolf gets commented out, which is worse
|
|
# than not having it. These two shapes are what the real bug looked like: the
|
|
# interpreter as the first word of a command, or inside $( ).
|
|
bare=$(cd "$work/repo" && for f in ci/*.sh; do
|
|
[ "$f" = ci/lib.sh ] && continue
|
|
sed 's/#.*//' "$f" \
|
|
| grep -nE '^[[:space:]]*php[[:space:]]|\$\((cd [^)]*&&[[:space:]]*)?php[[:space:]]' \
|
|
| sed "s|^|$f:|"
|
|
# python3 itself IS on the Gitea runner; only PyYAML is missing. So a bare
|
|
# python3 is fine for anything stdlib (build-zip.sh's zipfile, the JSON
|
|
# readers in publish-release.sh) and wrong only when yaml is involved.
|
|
sed 's/#.*//' "$f" \
|
|
| grep -nE '^[[:space:]]*python3[[:space:]].*(yaml|check-workflows)' \
|
|
| sed "s|^|$f:|"
|
|
done)
|
|
if [ -n "$bare" ]; then
|
|
printf '%s\n' "$bare" | sed 's/^/ /'
|
|
bad "a ci/ script calls an interpreter directly instead of ci_php / ci_python_yaml"
|
|
else
|
|
ok "every interpreter call goes through the container-capable wrapper"
|
|
fi
|
|
|
|
echo
|
|
if [ "$fail" != 0 ]; then
|
|
echo "Self-test: $fail of $((pass + fail)) assertions FAILED."
|
|
exit 1
|
|
fi
|
|
echo "Self-test: all $pass assertions passed."
|