Make the CI checks say how many files they actually read

Four checks in ci/ printed a reassuring line having examined nothing. Found
independently by the woo-ai-manager and woo-cache-manager sessions in their own
copies of this kit; all four reproduce here.

A check that examined nothing prints the same sentence as one that examined
everything, and only the passing side can tell them apart. That is why these
survived being verified: a pattern that is too broad fails loudly the first time
it meets real code, while one that finds nothing passes forever. So every
success line now carries a count — the old text domain line could not tell 142
from 0.

- static.sh, direct-access guard: counts what the loop examined and fails below
  MIN_SHIPPED_PHP_FILES. shipped_files() swallows a dist-files.sh failure with
  `|| true`, so an empty list is one broken .distignore away, and this is the
  check standing between a PHP file and being requested straight from a browser
  outside every capability check the code assumes has happened. The floor is
  above 1 because a pattern that wrongly excludes includes/ collapses the list
  without emptying it.

- static.sh, credential scan: counts what it read, fatal on zero. A pushed key
  is compromised even after the commit is deleted, so a silent no-op here is not
  a weak check but the absence of one wearing a green tick. The floor is 1 and
  is not configurable: this list has no filter that could go partially wrong, so
  the only way it misreports is by coming out empty.

- check-textdomain.php: three zeros, three verdicts. No PHP files at all is
  always fatal and deliberately not caller-configurable — it says the scan root
  is wrong, not that the plugin is small. Below --min is fatal. Zero calls with
  no --min is a warning, since a small plugin may legitimately have no
  translatable strings, but it is never silent.

- check-textdomain.php: fix the annotated path. str_replace( $root, '', $path )
  with the default root of "." removed every dot anywhere in the path, so a real
  error in includes/class-db.php was reported on "includes/class-dbphp" — not a
  file, not clickable, and not greppable. Every error this script printed with
  the default root was mangled that way.

The floors live in ci/plugin.conf, the only file under ci/ that knows anything
about this plugin, rather than hardcoded in static.sh: static.sh is copied
verbatim between plugins too, so a number in it would make this plugin's shape
everybody's constraint. ci_load_conf now requires them — an unset floor turns
its comparison into a no-op and silently restores the bug.

Each floor was broken once and watched to fail before being believed: empty
shipped list, wrong scan root, --min above the real count, zero calls without
--min, empty credential list, and a plugin.conf missing both floors. Static
checks green, and now report 13 shipped PHP files, 142 translation calls across
17 files, 44 tracked files read.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kristianwind
2026-09-17 00:14:40 +02:00
co-authored by Claude Opus 5
parent 7f3c8a36a2
commit 9b0b0ab475
5 changed files with 174 additions and 13 deletions
+19 -1
View File
@@ -20,7 +20,7 @@ ci/ every script; plugin.conf is the only one you edit
.distignore
```
**2. Rewrite `ci/plugin.conf`** — five values, all commented in the file:
**2. Rewrite `ci/plugin.conf`** — seven values, all commented in the file:
```sh
PLUGIN_SLUG=your-plugin
@@ -28,8 +28,26 @@ PLUGIN_MAIN=your-plugin.php
TEXT_DOMAIN=your-plugin
NEEDS_WOOCOMMERCE=yes # no, for a plugin that does not need it
SITE_TESTS="tests/fixtures.php tests/smoke.php"
MIN_SHIPPED_PHP_FILES=5 # floors: see below
MIN_TRANSLATION_CALLS=60 # 0 for a plugin with no translatable strings
```
`ci_load_conf` fails if any of them is unset, the two floors included — an
empty floor turns its comparison into a no-op and quietly restores the bug the
floors exist for.
Set each floor well below the plugin's real count. They are not assertions
about size: a tight number goes red on ordinary churn and gets raised until it
means nothing. They assert that the scan still finds the plugin at all, because
**a check that examined nothing prints the same reassuring line as one that
examined everything**. Count the real numbers first (`./ci/dist-files.sh | grep
-c '\.php$'`, and the tail of a `ci/static.sh` run), then take roughly a third.
Then break each floor once and watch it fail before believing it. A pattern
that is too broad fails loudly the first time it meets real code; one that is
too narrow is indistinguishable from one that finds nothing, and passes
forever — which is why every success line in this kit now carries a count.
**3. Adjust `.distignore`** for that plugin's internal documents. Everything
left in `wp-content/plugins/` can in principle be fetched over HTTP from a live
shop, so notes and tooling stay out of the zip.