kristianwindandClaude Opus 5
a1f6e54f9d
Four fixes, two of them serious, three of them found by the session porting
this into subscriptions.
1. THE SMOKE TEST COULD NOT FAIL. `wp eval-file` runs the file through eval()
INSIDE A METHOD, so the top level is function scope. `$wc_edm_failures = 0`
at the top and `global $wc_edm_failures` in the helper were two different
variables: the helpers incremented the real global, the verdict read the
local, and the local was always zero.
Measured rather than reasoned. A deliberately false assertion added to the
unmodified file:
FAIL DELIBERATE FAILURE — this run must go red
All assertions passed on WordPress 7.1 with WooCommerce 11.1.0.
EXIT=0
and with the fix, the same forced failure:
FAIL DELIBERATE FAILURE — this run must go red
1 assertion(s) failed on WordPress 7.1.
EXIT=1
The FAIL lines always printed, so failures found by reading the output were
real finds. What was broken is the CI gate: a regression on either forge
would have gone green. Four lines at the top now prove the counter reaches
the verdict before anything relies on it — this is the one file whose whole
job is to be able to go red, so it checks that it can.
2. `git ls-files` QUOTES NON-ASCII PATHS. "TESTMILJØ.md" comes back as the
literal string "TESTMILJ\303\230.md", which matches no pattern in
.distignore, so the file is kept — an internal document walking into the
release zip with nothing said. Fixed in ci/dist-files.sh and in
ci/static.sh's trailing-?> check, which had the same bug and would have
silently skipped the file it exists to check. Verified by adding such a
filename and watching it be excluded. `done < <(...)` rather than a pipe, or
the loop runs in a subshell and the failure counter stops incrementing.
3. THE FEED CACHE WAS SIX HOURS, and Kristian hit it: a release published and
his shop shown nothing. The reasoning ("WordPress checks twice a day") was
wrong. WordPress gates its own check by screen in wp_update_plugins() — one
minute on Dashboard → Updates, one hour on Plugins, 2–12 hours in the
background — so WordPress is already the rate limiter and a longer cache
only hides releases underneath it. Fifteen minutes now, pinned by an
assertion.
4. The wp_update_plugins() assertion made a real HTTP request, because the
force-check tests had just deleted the transient. It passed only because
this feed repo is public, which made it measure the environment rather than
the code. Re-primed first.
ci_in_container() replaces the inline docker plumbing in ci_php(), taken from
the subscriptions port: the script goes in as a file rather than interpolated
into `sh -c`, and --entrypoint sh matters for images whose entrypoint swallows
its arguments.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
264 lines
11 KiB
Markdown
264 lines
11 KiB
Markdown
# Testing, releases and automatic updates
|
||
|
||
Three questions, answered by three things in this repository.
|
||
|
||
| Question | Answer |
|
||
| --- | --- |
|
||
| Does it still work on the newest WordPress and WooCommerce? | `ci/site-test.sh`, run weekly by `compat.yml` against `latest`, `nightly` and `beta` |
|
||
| Can updates be pushed out automatically? | Yes — `includes/class-updater.php` plus `release.yml` |
|
||
| What happens when *I* press update in wp-admin? | Nothing surprising, because the release was tested against those versions before it was published |
|
||
|
||
---
|
||
|
||
## Running the tests
|
||
|
||
```sh
|
||
ci/static.sh # syntax, hygiene, versions, headers — seconds
|
||
ci/site-test.sh # a real WordPress + WooCommerce — a few minutes
|
||
ci/build-zip.sh # what a shop actually installs
|
||
FROM_VERSION=1.4.0 \
|
||
ci/update-smoke.sh # let WordPress update itself from the live feed
|
||
```
|
||
|
||
`ci/site-test.sh` takes three knobs:
|
||
|
||
```sh
|
||
WP_VERSION=latest # or nightly, or an exact version like 7.0
|
||
WC_VERSION=latest # or beta, or an exact version like 11.1.0
|
||
PHP_VERSION=8.3 # any tag of the wordpress:cli image
|
||
```
|
||
|
||
It builds a throwaway site in Docker containers, installs WordPress and
|
||
WooCommerce at those versions, copies in **exactly the files the release zip
|
||
will contain**, activates the plugin, and runs `tests/fixtures.php` and
|
||
`tests/smoke.php` inside it with `wp eval-file`.
|
||
|
||
The tests use a real cart, real coupons and real `WC_Shipping_Rate` objects.
|
||
That is the point. A mocked cart keeps passing after WooCommerce has changed
|
||
underneath it, which is the one failure this whole directory exists to catch.
|
||
|
||
Everything is torn down afterwards — containers, volumes and the network — so
|
||
running it twice leaves nothing behind.
|
||
|
||
### Deprecations count as failures
|
||
|
||
The site test switches on `WP_DEBUG` and fails the build on any notice logged
|
||
from this plugin's directory. That is how a WordPress or PHP upgrade actually
|
||
breaks a plugin: nothing fails, a notice appears, and two releases later it is
|
||
a fatal. Notices from WordPress core and WooCommerce are ignored — failing on
|
||
those would make the build red for something nobody here can fix, and a build
|
||
that is always red is a build that gets ignored.
|
||
|
||
---
|
||
|
||
## The weekly compatibility run
|
||
|
||
`compat.yml` runs the matrix every Monday at 06:00 and on demand:
|
||
|
||
| WordPress | WooCommerce | PHP | What it tells you |
|
||
| --- | --- | --- | --- |
|
||
| latest | latest | 8.3 | today's shop |
|
||
| **nightly** | latest | 8.3 | the next WordPress, weeks early |
|
||
| latest | **beta** | 8.3 | the next WooCommerce, weeks early |
|
||
| latest | latest | 7.4 | the floor the plugin header promises |
|
||
| latest | latest | 8.4 | where deprecations appear first |
|
||
|
||
The `beta` cell falls back to the stable release when there is no open beta, so
|
||
it is never silently skipped.
|
||
|
||
When the whole matrix passes, a second job compares `Tested up to:` and
|
||
`WC tested up to:` against the current releases and opens a pull request if
|
||
they are behind. It runs only after the matrix is green, because those headers
|
||
are a claim — WooCommerce shows `WC tested up to` to the shop owner as an
|
||
untested-plugin banner, and that banner is what they read before deciding
|
||
whether to press update.
|
||
|
||
The schedule fires on both forges; the matrix job skips the scheduled run on
|
||
Gitea so the single self-hosted runner is not asked to repeat GitHub's work.
|
||
`workflow_dispatch` still works there.
|
||
|
||
---
|
||
|
||
## Cutting a release
|
||
|
||
1. Bump `Version:` in the plugin header. `ci/static.sh` checks that
|
||
`WC_EDM_VERSION` and the readme's `Stable tag:` agree with it — WordPress
|
||
compares the update feed against the header and nothing else, so a
|
||
disagreement means the update is offered again immediately after it is
|
||
installed, forever.
|
||
2. Add a changelog section to `README.txt`.
|
||
3. Merge to `main`.
|
||
4. Tag it **annotated**, because the message is the release notes:
|
||
```sh
|
||
git tag -a v1.4.0 -m "..."
|
||
git push origin v1.4.0 # origin pushes to GitHub and Gitea both
|
||
```
|
||
|
||
`release.yml` then runs the static checks, installs the plugin into a real
|
||
WordPress + WooCommerce and runs the full test suite, builds the zip, publishes
|
||
the release, and finally **reads the feed back anonymously** to prove a shop
|
||
can actually download it.
|
||
|
||
Two things in there are guards against bugs that are invisible without them:
|
||
|
||
- **The tag message is re-fetched before it is read.** `actions/checkout`
|
||
fetches every tag correctly and then overwrites the triggering one with the
|
||
commit SHA, turning an annotated tag into a lightweight one. Reading it then
|
||
returns the *commit* message — on a merge commit, "Merge pull request #12
|
||
from …". Nothing fails; the release is just published with the wrong text.
|
||
- **The anonymous read-back sends no credentials.** The two ways the feed
|
||
breaks — a private repository and a release with no zip attached — both
|
||
leave a release page that looks perfectly fine to anyone logged in.
|
||
|
||
---
|
||
|
||
## Proving an update actually arrives
|
||
|
||
`ci/update-smoke.sh` installs the **previous** version into a throwaway shop
|
||
and then gets out of the way: WordPress's own update check finds the new one,
|
||
downloads it and installs it.
|
||
|
||
```sh
|
||
FROM_VERSION=1.4.0 ci/update-smoke.sh
|
||
```
|
||
|
||
Nothing else can stand in for it. Every piece of the chain can be individually
|
||
correct and the update still not arrive:
|
||
|
||
- a package URL on an address the shop cannot reach,
|
||
- a zip whose top-level directory is not the plugin slug — which installs a
|
||
*second* copy beside the first and leaves the old one active,
|
||
- a version WordPress does not consider newer than what is installed.
|
||
|
||
It checks all three, plus that the plugin is still active afterwards and that
|
||
WordPress stops offering the update once it is applied.
|
||
|
||
Run it after a release.
|
||
|
||
---
|
||
|
||
## How the automatic updates work
|
||
|
||
The plugin header carries:
|
||
|
||
```
|
||
Update URI: https://gitea.nolimit.dk/kw/woocommerce-email-discount-manager
|
||
```
|
||
|
||
`includes/class-updater.php` hooks `update_plugins_gitea.nolimit.dk`, the
|
||
filter WordPress fires for plugins whose `Update URI` points at that host. It
|
||
reads `/api/v1/repos/kw/<repo>/releases/latest`, takes the tag as the version
|
||
and the attached `.zip` as the package, and hands both to WordPress. WordPress
|
||
does the version comparison, the download, the unpacking and the rollback.
|
||
|
||
That means all of this works with no extra code:
|
||
|
||
- the update notice on **Plugins** and **Dashboard → Updates**
|
||
- the **update now** link
|
||
- the per-plugin **Enable auto-updates** link
|
||
- `wp plugin update woocommerce-email-discount-manager` over WP-CLI
|
||
|
||
### Turning on automatic updates
|
||
|
||
Plugins screen → the **Automatic updates** column on the right → **Enable
|
||
auto-updates**. WordPress then installs new versions by itself on its
|
||
twice-daily check.
|
||
|
||
That link is not free for a plugin outside wordpress.org.
|
||
`WP_Plugins_List_Table` marks a plugin as `update-supported` only if it finds
|
||
it in the update transient's `response` list (an update is waiting) or its
|
||
`no_update` list (we were asked, there is none):
|
||
|
||
```php
|
||
// wp-admin/includes/class-wp-plugins-list-table.php
|
||
$plugin_data = array_merge( (array) $plugin_info->response[ $plugin_file ], array( 'update-supported' => true ), $plugin_data );
|
||
$plugin_data = array_merge( (array) $plugin_info->no_update[ $plugin_file ], array( 'update-supported' => true ), $plugin_data );
|
||
} elseif ( empty( $plugin_data['update-supported'] ) ) {
|
||
$plugin_data['update-supported'] = false;
|
||
```
|
||
|
||
A plugin in neither list gets "Auto-updates are not available for this plugin"
|
||
and no toggle — while looking entirely healthy otherwise. That is why
|
||
`offer_update()` answers even when the installed version is already current:
|
||
the answer lands in `no_update`, and the link stays. `tests/smoke.php` builds
|
||
the real list table and asserts the flag rather than trusting the reasoning.
|
||
|
||
### "Check again" really checks again
|
||
|
||
The feed is cached for fifteen minutes, and the cache is dropped entirely when
|
||
wp-admin is loaded with `force-check`.
|
||
|
||
It was six hours, on the reasoning that WordPress checks twice a day anyway.
|
||
That reasoning was wrong. WordPress gates its own check **by screen**:
|
||
|
||
| Screen | How often WordPress re-checks |
|
||
| --- | --- |
|
||
| Dashboard → Updates | every minute |
|
||
| Plugins | every hour |
|
||
| Background / cron | every 2–12 hours |
|
||
|
||
This filter only runs when that check runs, so WordPress is already the rate
|
||
limiter. A cache longer than its cadence does nothing but hide new releases
|
||
underneath it — a release published, and a shop shown nothing.
|
||
|
||
From the command line, `wp transient delete wc_edm_release_feed` does the same.
|
||
|
||
### The icon
|
||
|
||
`assets/icon.svg`, handed to WordPress in the offer's `icons` key. Without it
|
||
Dashboard → Updates shows a generic grey plug. WordPress takes the first of
|
||
`svg`, `2x`, `1x`, `default` that is set.
|
||
|
||
It is served from the shop's own copy of the plugin, not from the release host:
|
||
one fewer thing that has to be reachable for wp-admin to render correctly, and
|
||
it cannot go mixed-content on an HTTPS shop. The mark is drawn as paths rather
|
||
than `<text>`, because an SVG that leans on a font renders differently
|
||
everywhere — and this one is shown at about 40px, which is the size where that
|
||
matters.
|
||
|
||
`Update URI` earns its place a second way: since WordPress 5.8, a plugin that
|
||
declares one is **never asked about on wordpress.org**. Without it, a .org
|
||
plugin that happened to share this slug could be pushed onto a shop running
|
||
this one.
|
||
|
||
### One-time setup
|
||
|
||
The repository on Gitea must be **public**, because every shop reads that feed
|
||
without credentials. Settings → make it public. Nothing in a GPL plugin zip is
|
||
secret, and the release workflow fails loudly with this exact instruction if it
|
||
is not done.
|
||
|
||
If you would rather keep the source private, create a separate public repo that
|
||
holds only releases and point `Update URI` at that instead — the updater
|
||
derives everything else from the header, so that is a one-line change.
|
||
|
||
### The download address is rebuilt, not taken
|
||
|
||
The updater ignores the `browser_download_url` the API hands back and builds
|
||
the address from the `Update URI` header instead.
|
||
|
||
That field is filled from the forge's own configured root URL, which is not
|
||
necessarily the address the outside world reaches it on. `gitea.nolimit.dk`
|
||
returns `http://57.129.89.117:25001/...` there — plain HTTP, a raw IP, a
|
||
non-standard port, and around the Cloudflare tunnel in front of it. A shop
|
||
handed that would be downloading plugin code over an unencrypted connection to
|
||
an address nothing guarantees will keep working.
|
||
|
||
The paths Gitea reports are correct; only the host is wrong. So the host comes
|
||
from the header — the one place that already had to be right for any of this
|
||
to work — and the path is kept.
|
||
|
||
This is worth knowing because of *how* it hides: a CI job on the same network
|
||
reaches that raw IP perfectly well, so a verification step that follows the
|
||
reported URL goes green. Only a real client outside fails. The release check
|
||
therefore downloads the address **the plugin builds**, and opens the zip to
|
||
confirm it is the plugin rather than an error page with a 200 on it.
|
||
|
||
### It reports nothing
|
||
|
||
The update check sends a plain unauthenticated GET for a file every visitor to
|
||
that repository can read. The user agent is overridden on purpose: WordPress's
|
||
default one is `WordPress/7.1; https://the-shop.example`, which would tell the
|
||
update host which shops are running this plugin. Nothing else about the site
|
||
leaves it, ever.
|