Integration evidence and limits

Isolated integration fixture

Current candidate: 1.26.0-rc.1. After the owner’s later test authorization, the fresh ocp-check-bf3a22fd1247 run on 2026-10-04 passed all 37 PHP regression invocations, seven real integration scripts and the lifecycle suite. It used PHP 8.3.35, WordPress 6.8.3, WooCommerce 10.2.2, PhpRedis 6.1.0 and Redis 7.4.2; 187 installed runtime files and the drop-in matched the selected source. The owned containers, network and volumes were removed. Evidence: results/ocp-check-bf3a22fd1247/run.json and prefetch_membership_validation in status.json. The new prefetch scenario passed 50 real Redis checks for caps, seeded equal-count metadata inconsistencies, serialization, TTL and site/data isolation. The run used base commit 5055471 plus the recorded prefetch correction; it does not certify an exported ZIP. Real server partial-EXEC failure injection and performance costs remain unmeasured. This default run did not select --topologies, --acceptance or --commerce, and is not production acceptance. Historical results later in this file apply only to their original revision.

The subsequent error-buffer reporting update passed 38 host PHP unit cases and 152 assertions, including 24 focused checks of caps, logger delivery, shared request totals, reset behavior and rendered PHP templates. See log_bounds_validation in status.json. The Docker run above predates this update; its new admin reporting has not been checked in a real WordPress/browser request.

This is test infrastructure, not a production deployment. HTTP is published only on 127.0.0.1 (port 8088 for manual commands; a random port for the fresh runner). The test credentials and metrics mu-plugin must never be deployed. The fixture uses WordPress 6.8.3, WooCommerce 10.2.2, PHP 8.3, PhpRedis 6.1.0, Redis 7.4.2 and MariaDB 11.4. These are comparison fixtures, not claims about the latest supported releases. Image IDs and actual runtime versions are recorded.

From the repository root:

docker compose -f tests/integration/compose.yaml up -d --build
docker compose -f tests/integration/compose.yaml exec -T wordpress integration-setup
docker compose -f tests/integration/compose.yaml exec -T wordpress wp eval-file /opt/plugin/tests/integration/contracts.php --allow-root
python tests/integration/benchmark.py
python tests/integration/compare.py tests/integration/results/baseline.json tests/integration/results/candidate.json

Do not use these commands against a shared Redis or production database. The benchmark flushes only database 4 in this project’s isolated Redis container. Seed operations are idempotent. Rebuild and rerun setup to test a changed plugin. Setup refreshes the copied plugin and drop-in; it does not silently download a different WooCommerce version.

The benchmark records cold/warm p50/p95, PHP peak memory, SQL queries, cache operation counts, Redis wait, server command counters, network byte counters, redirects, runtime versions and image IDs. Byte counters include the INFO measurement probes. Cold samples are first requests after cache flush. Warm requests use an independent cookie jar per run. Checkout GET timings are not a checkout submission test; full purchase/session isolation, 20-process concurrency, Sentinel, Cluster, TLS, Relay and million-key tests remain separate release gates and must not be claimed as passing from this fixture alone.

The comparator rejects missing or mismatched scenarios, fewer than five repetitions, different runtime versions or request counts, and warm p95/peak-memory regressions over 5%. WordPress image IDs differ between plugin builds, so identity is retained as evidence but is not compared for equality. Review dependency image IDs when comparing runs.

Measurement notes (recorded in each result’s notes):

• Redis INFO stats probes run through one persistent docker compose exec session that executes redis-cli INFO stats per probe inside the container. Spawning a Docker CLI process per probe cost about 1.3 s each; Redis-side command and byte counters keep the same semantics (one probe connection and command per sample).
• localhost is resolved to 127.0.0.1 inside the benchmark process while the URL and Host header stay unchanged. On Windows localhost resolves to ::1 first and each connection waited about 2 s for the IPv6 attempt to fail inside response_ms. Results recorded before this fix are not comparable with newer results.

Comparable fixtures: seed.php assigns each fixture product a reserved post ID (1000 + n for fixture-00n, through wp_insert_post’s import_id) and fails if a reserved ID is taken or an earlier seed left a product at another ID. Auto-increment IDs were not reproducible: a 1.25.6 baseline fixture skipped one ID per product from the seventh product, so fixture_sha256 differed although the fixture was otherwise identical. Products are inserted in descending order (fixture-030 first): an explicit-ID insert raises the posts AUTO_INCREMENT counter to id + 1, so the first insert lifts it past the whole reserved range and side-effect posts created during later saves land at 1031 and above. Ascending order would hand those posts the next reserved ID and fail the collision check on exactly that baseline. WooCommerce pages are created before the products so their IDs stay below the reserved range. Recreate the disposable fixture (down --volumes, then setup) after this change. identity.php, metrics.php, seed.php and setup.sh are hashed into provenance, so baseline and candidate must both be measured with the same harness bytes; the comparator still requires identical fixture_sha256.

Configuration must match exactly, except for options listed in ABSENT_OPTION_EQUIVALENTS in compare.py. A baseline release that predates such an option reports null; that is accepted only when the baseline plugin version is older than the version that introduced the option, the candidate is not, and the candidate value is the documented equivalent of the old behavior. Currently only negative_cache (introduced in 1.26.0, equivalent false) is listed. Accepted equivalences are reported in the comparator’s configuration_equivalences output.

To stop while keeping test data:

docker compose -f tests/integration/compose.yaml down

down --volumes deletes this project’s fixture data. It is not part of the setup script.

Implementation gate

Before changing the production core, run the baseline tag’s plugin through this fixture and archive the results. Baseline tag: baseline/pre-modernization. No measured result is present for the v2 baseline/candidate performance comparison; the successful default integration run does not supply benchmark measurements.

Bounded PhpRedis Cluster regression

With the existing objectcache-integration WordPress fixture running and the official redis:7.4.2 image already available locally, run:

python tests/integration/cluster-probe.py
php -n tests/regression/cluster-false-read.php

The probe creates one disposable container containing three masters, no replicas, volumes, or published ports. It shares the test WordPress network namespace and binds Redis only to loopback with protected mode enabled. Occupied probe ports abort the run. Cleanup checks its own container label before removing it; existing fixture containers remain running. This does not rebuild the fixture or refresh its copied plugin, so apply the code under test to the fixture first.

The fixture’s PHP 8.3 / PhpRedis 6.1.0 path is tested with negative cache off/on and a key on each master: stored false/miss, external writer visibility, force, mutation invalidation, bulk values/order, atomic WRONGTYPE and normal recovery. Native GET/EXISTS order is also checked: Cluster clears the GET error when a following EXISTS succeeds; running GET last retains it for the cache error path. The JSON report is saved under tests/integration/results/cluster-compatibility.json (or the path supplied as the first argument). Assertion failures exit nonzero. This test does not establish Relay, replicas, failover, TLS, or other version coverage.

Sentinel failover and concurrency soak

With the fixture running and redis:7.4.2 available locally:

python tests/integration/sentinel-probe.py
python tests/integration/soak.py --seconds 300

The Sentinel probe starts one disposable labelled container (primary, replica and three Sentinels, loopback only, no published ports), runs real SENTINEL FAILOVERs and checks negative cache off/on with single-first, bulk-first and bulk-only writes after failover, stored false, misses, forced reads and new-request reads. Cleanup runs even when assertions or report writing fail. It does not cover TLS, Relay, network partitions or replica lag beyond an acknowledged WAIT.

The soak runs concurrent HTTP clients against the fixture, a parallel add() race (exactly one winner expected) and fails on new debug.log lines. It is concurrency evidence on one Docker Desktop host, not a production capacity measurement.

Fresh Docker fixture in one command

From the repository root, run python3 tests/integration/run.py (on Windows, python tests/integration/run.py). Add --topologies for the separate Cluster and Sentinel probes. Add the opt-in --valkey to swap the single-node redis service for pinned Valkey via compose.valkey.yaml (never run yet; see VALKEY.md). Each run builds the current checkout, creates a unique Compose project and fresh volumes, assigns a random localhost HTTP port, installs the fixture, and runs every PHP regression plus the integration contracts, negative-cache and recovery checks. It removes only its own containers/network/volumes even on failure. Reports and actual component versions are saved under ignored tests/integration/results/ocp-check-*/.

Prerequisites: Python 3.10+, a Docker engine running Linux containers, Docker Compose v2 with up --wait, and internet access for images, WordPress and WooCommerce packages. Docker must be accessible to the invoking user. This is a Docker test workflow; Codex Cloud requires a provider environment with an accessible compatible Docker engine. Provider/cloud execution is not implied or verified by local validation.

Use --keep only to retain the fresh fixture for debugging; its report includes the exact cleanup command. The default suite excludes benchmarks and soak tests. The JavaScript DOM regression tests/regression/tools.cjs requires separate Node.js, Playwright and browser installation and is not run by this PHP fixture.

The runner compares installed runtime/drop-in bytes with the selected checkout and records fixture, configuration and harness identity before running contracts. --acceptance additionally selects the heavy scenarios (never selected by default):

• alloptions-concurrency.php: 20 processes, five repetitions of add, replace, missing replace, competing field updates and ordered stale-snapshot field merges. A barrier forces WATCH conflicts; rejected writers must forget local values.
• cleanup-scale.php: 100,000 and 1,000,000 keys, group cleanup and streaming query pruning, five repetitions each. It records peak PHP memory and deletion batches, rejects batches above 500, verifies site/group/prefix fences and applies the declared 8 MiB memory-growth tolerance across the tenfold key-count change.

These scripts require OCP_ACCEPTANCE_FIXTURE=1 inside the disposable fixture; the runner supplies it only for the selected scenarios. Scenario source hashes must match the container. Reports are stored in run.json as structured evidence. The source definitions are not successful test results. They do not establish checkout/customer-session isolation, real Relay support, topology/TLS fault coverage, or the five-repetition HTTP baseline/candidate performance gate.

The counter contract includes real second-connection writes/deletions between WATCH and EXEC. The Sentinel failover contract expects a failed ordinary write to return false without replay, then checks recovery on a later explicit call. Pinned pipelines retain their no-replay behavior; known READONLY errors in this fixture must not be confused with uncertain lost replies in production.

--commerce adds a separate authenticated-customer HTTP scenario. It uses two independent cookie jars, verifies the logged-in customer identities, adds shared and customer-specific products, submits two Store API checkouts through the offline BACS gateway, and checks cart isolation after each purchase and after a new login. A third anonymous session must remain empty. Order/customer/item ownership is checked from a fresh CLI process; stock is read through WooCommerce and directly from the database before and after purchase, without manually flushing product caches. The final shared product must be out of stock.

The commerce runner creates two virtual products and records owned order IDs, including drafts observed while their line items are saved. It cleans tracked data and restores changed gateway/checkout options. Failures remain failures if cleanup cannot be confirmed; the outer runner removes its owned fixture unless --keep was requested. An authenticated nonce bootstrap and order-tracking mu-plugin is copied into that disposable container only during the scenario and removed in the runner’s finally block. Outbound WordPress mail is suppressed while it is installed. No card processor or real transfer is contacted. Cookies, nonces, passwords and order keys are excluded from commerce.json.

This source scenario targets the pinned WooCommerce 10.2.2 fixture, using its Store API checkout contract and processed-order hook. It has not been executed for v2. It does not establish shipping/tax extensions, external payment gateways or every WooCommerce storage-mode combination.

Without an accessible Docker daemon, the standalone PHP regressions can still run if PHP CLI is installed:

set -eu
for test in tests/regression/*.php; do php -n "$test"; done
php -n tests/regression/sentinel-tls.php legacy

These use stubs and require no Composer install, WordPress, Redis or network. They do not validate real Redis behavior.

Local validation on 2026-10-02: the fresh runner with --topologies exited 0 on Docker Desktop’s Linux engine (Compose 5.5.1). All PHP regressions and three real Redis integration scripts passed; Cluster passed six scenarios (three masters, no replicas), and Sentinel passed six failover scenarios (one primary, one replica, three sentinels). Runtime versions: PHP 8.3.35, PhpRedis 6.1.0, WordPress 6.8.3, WooCommerce 10.2.2 and Redis 7.4.2. Owned probe containers and fixture containers/network/volumes were removed. The old alloptions stub emits undefined negative_cache property warnings despite passing. Node/browser tests, benchmarks, soak runs and Codex Cloud execution were not validated by this run.

Plugin and drop-in lifecycle gate

The fresh python3 tests/integration/run.py runner also executes lifecycle.sh in its disposable fixture. It checks enable/disable, file-modification denial, upgrade event filtering, valid outdated drop-in replacement, foreign/absent drop-in preservation, and network deactivation/reactivation across fresh WP-CLI processes. The final process verifies a real Redis write and stored-false hit after re-enabling. This exercises the WordPress upgrade completion hook with an old version header; it does not download a historical plugin release or test an external update server. The upgrade checks preserve the plugin’s actual registered hook callback while excluding unrelated core language-pack and remote-update callbacks from the simulated event. The test requires explicit OCP_LIFECYCLE_FIXTURE opt-in and restores the active plugin/drop-in after failure. Never run it outside the disposable fixture.

Minimal WordPress core evidence

When package downloads prevent installing WooCommerce, an independently prepared real WordPress multisite fixture with this plugin/drop-in and the /fixture-site/ secondary site can run only the core contracts explicitly:

docker compose -f tests/integration/compose.yaml exec -T \
  -e OCP_CORE_ONLY_CONTRACTS=1 wordpress wp eval-file \
  /opt/plugin/tests/integration/contracts.php --allow-root

This mode still tests the real cache’s stored false/null, add/replace, arithmetic, TTL, bulk reads/writes, group flush and multisite isolation. It prints a core PASS only after these checks and an explicit SKIP stating that WooCommerce product invalidation coverage is untested. It does not install or prepare the fixture. The default contracts.php invocation requires WooCommerce; missing WooCommerce is a failure. The fresh runner does not enable core-only mode automatically. Archive the exact command and its output with runtime versions, and describe such a result as WordPress core cache evidence, never as a completed WooCommerce suite.

Primary-only Sentinel and split-alloptions bulk ADD

The normal runner includes bulk-alloptions-add.php; --topologies also runs sentinel-primary-only-probe.py. The latter creates one uniquely labelled Redis container with one primary, zero replicas and three Sentinels, sharing only the owned WordPress fixture’s network namespace and binding to loopback. It checks SCAN/HSCAN/SSCAN/ZSCAN iterator references and MATCH/count arguments, listKeys, false/null reads and incremental group flush isolation. It deletes its enumerated owned keys and waits briefly for Docker’s asynchronous container removal.

To run only this probe against an already refreshed disposable fixture, set OCP_FIXTURE_CONTAINER, OCP_FIXTURE_PROJECT and OCP_FIXTURE_URL to that fixture, then run python3 tests/integration/sentinel-primary-only-probe.py <report.json>. This is primary-only behavior evidence, not failover, TLS or licensed Relay proof.

Minimum supported runtime

The same fresh, isolated runner can build the minimum base explicitly:

python tests/integration/run.py --php-version=8.2 --phpredis-version=6.0.0

Defaults remain PHP 8.3/PhpRedis 6.1.0. Requested versions are recorded separately from actual runtime versions in each report. The options select existing Docker build arguments, with the same contracts, identity checks and owned cleanup. A selectable environment is not evidence that its tests passed.

The default real-Redis sequence also includes ttl-history.php: separate metadata bounds/expiry, actual second-client WATCH conflicts, old-format and site isolation, completed-hour reports and sampled cache writes. See TTL-HISTORY.md.

TTL history browser acceptance

ttl-history-browser.cjs uses the existing Playwright browser stack against a retained disposable fixture only. Set OCP_BROWSER_FIXTURE=1 and OCP_FIXTURE_URL=http://localhost:<fixture-port>, expose the installed Playwright package on NODE_PATH, then run:

node tests/integration/ttl-history-browser.cjs build/ttl-history-browser.json

It defaults to installed Edge; BROWSER_CHANNEL=chrome selects installed Chrome. It uses only the seeded fixture admin, rejects non-local targets, blocks external browser requests, and never changes cache configuration. Reports exclude cookies, nonces and passwords. Real HTTP checks cover explicit loading, disabled/empty history, authentication/REST nonce requirements and no-store responses. Rendering and HTTP-error fixtures are synthetic and labelled separately; they do not prove real Redis nonempty/error behavior. The registered-route scenario ttl-history-api.php separately covers retained records and multisite permissions.