Service operation and backups

Operating and recovering the Go licensing backend

This guide covers the current Linux, loopback-only Go service and an isolated recovery rehearsal. It does not deploy a public service or provision domains, TLS, billing accounts or production credentials. The executable, HTTP contract and persistence format remain those described in README.md.

Build and configuration

Build a reviewed revision with Go 1.24+ (tested with Go 1.26.1) and retain its commit ID and binary checksum with the operator’s configuration record:

cd product/licensing/go
CGO_ENABLED=0 go build -buildvcs=false -o /tmp/original-licensing .
go test -race ./...
go vet ./...

The static binary runs without Go, Python, SQLite or an OpenSSL executable on the server. Key generation and backup utilities are separate operator tooling. Run under one dedicated Unix identity on a local filesystem that supports flock, atomic rename, file fsync and directory fsync. Network filesystems are unsupported. Use private directories with mode 0700 and state, lock, private-key, HMAC configuration and backup files with mode 0600. Set the shell’s umask 077 before redirects or file creation; the binary’s own umask cannot protect files opened earlier by the shell. Keep the runtime tree outside a web/document root.

Keep the HMAC secret, RSA private key and their custody records on the server and in restricted backups. PHP receives only trusted public keys plus its own license/installation identities. The server state stores identity hashes, not recoverable bearer credentials: restoring it cannot reconstruct a site’s lost installation key. Back up WordPress identity options and its private entitlement cache through the separate WordPress recovery process.

The service has no hot reload. HMAC material, RSA signer and ZIP bytes are loaded at startup, and the ZIP becomes an immutable in-memory snapshot. Deploy new configuration/artifacts through an operator-controlled restart. Do not replace these files in place while treating their disk contents as evidence of what an older running process loaded. Record the exact startup files, version, public-key fingerprint, binary revision and checksums together.

A future HTTPS proxy must use an operator-selected trusted HTTPS origin, forward only to the local listener, avoid caching entitlement/download responses and avoid logging bodies or Authorization headers. PHP transport verifies TLS, disallows redirects and uses fixed routes; no arbitrary download URL is accepted. Do not configure production PHP to use the HTTP loopback fixture exception. Proxy/TLS configuration, access controls and public deployment remain separate work. WordPress admin configuration describes the current trusted-origin and public-key interface; use the installed plugin’s current configuration instructions when its bridge is enabled.

Local startup and operator actions

The following shell example uses operator-prepared absolute paths and a trusted, mode-0600 signing.env file containing an exported LICENSING_SIGNING_KEY. The file is shell code; load only the operator’s own configuration. The example does not generate or provision production secrets.

set -eu
umask 077
LIC_RUNTIME_DIR=/srv/original-licensing
LIC_BIN="$LIC_RUNTIME_DIR/original-licensing"
LIC_STATE="$LIC_RUNTIME_DIR/state/licenses.json"
LIC_CONFIG_DIR="$LIC_RUNTIME_DIR/config"
. "$LIC_CONFIG_DIR/signing.env"
LIC_SIGNING_KEY_ID=${LIC_SIGNING_KEY_ID:-fixture-v1}
# Set this from the operator's reviewed ZIP/version record.
: "${LIC_PACKAGE_VERSION:?Set the configured original ZIP version}"
# Existing deployments must investigate missing state rather than issue replacements.
test -s "$LIC_STATE"
"$LIC_BIN" --database "$LIC_STATE" serve --port 8090 --signing-key-id "$LIC_SIGNING_KEY_ID" \
  --private-key "$LIC_CONFIG_DIR/private.pem" \
  --original-package "$LIC_CONFIG_DIR/original.zip" --version "$LIC_PACKAGE_VERSION" &
LIC_SERVER_PID=$!

For a deliberately new database, issue a license with a future Unix expiry and a site limit of 1..1000. Redirect the issued bearer key to a private file; do not put it in a shell argument, log or support transcript. Origin approval requires independent ownership evidence; the backend does not fetch user-supplied URLs. Local CLI commands are the trusted operator interface, not public HTTP admin endpoints:

: "${LIC_NEW_EXPIRY:?Set a future Unix timestamp}"
"$LIC_BIN" --database "$LIC_STATE" issue --expires "$LIC_NEW_EXPIRY" --sites 1 \
  > "$LIC_CONFIG_DIR/new-license.txt"
"$LIC_BIN" --database "$LIC_STATE" approve-origin --key-stdin \
  --site https://approved.example.test < "$LIC_CONFIG_DIR/new-license.txt"
"$LIC_BIN" --database "$LIC_STATE" revoke --key-stdin \
  < "$LIC_CONFIG_DIR/new-license.txt"

Normal application activation then sends the existing three JSON fields: license_key, installation_key, site. The first activation binds installation ownership. Deactivation frees a slot while retaining that ownership; there is no operator installation-replacement/recovery command. The same state path must be used by the running service and operator commands, with the same fixed .lock file. Do not unlink/recreate the lock while any user of it is running.

Use an existing supervisor for process ownership/restarts, or keep the PID of the specific child launched above. The binary handles SIGTERM/SIGINT and allows up to 10 seconds for HTTP shutdown. In that same shell, stop it with:

kill -TERM "$LIC_SERVER_PID"
wait "$LIC_SERVER_PID"

There is no unauthenticated health route. A listening socket is not proof of a valid entitlement, signer, package or state. A controlled operator fixture can check /check, /entitlement, /updates and ZIP digest verification using a known approved installation; do not expose bearer material in monitoring logs.

Consistent private backup

The simplest coherent backup is made while the service is stopped and operator mutations/configuration changes are paused. Preserve the JSON, RSA private/public PEMs, exact HMAC configuration, original ZIP/version and binary/configuration record as a set. Restrict and encrypt any off-host copy using the operator’s existing custody process; this guide does not create a storage account.

A live JSON backup is also consistent when it takes the same fixed licenses.json.lock used by the backend. Take that lock before opening the JSON pathname. The backend replaces JSON via atomic rename, so locking the JSON file/inode itself does not serialize with it. A plain JSON copy might be a complete snapshot, but its ordering relative to a concurrent revocation is unknown. Locking makes that ordering explicit.

This local Linux example requires flock and GNU coreutils. Create a unique snapshot directory; keep keys/artifacts/configuration immutable and pause their rotation for the entire backup. All paths below refer to the operator’s own private directories:

set -eu
umask 077
LIC_BACKUP_ROOT=/srv/original-licensing-backups
LIC_BACKUP_DIR=$(mktemp -d "$LIC_BACKUP_ROOT/snapshot.XXXXXX")
chmod 700 "$LIC_BACKUP_DIR"
(
  flock -x -w 30 9 || exit 1
  cp -- "$LIC_STATE" "$LIC_BACKUP_DIR/licenses.json"
  chmod 600 "$LIC_BACKUP_DIR/licenses.json"
  sync -f "$LIC_BACKUP_DIR/licenses.json"
) 9<>"$LIC_STATE.lock"
for LIC_BACKUP_NAME in private.pem public.pem signing.env original.zip; do
  cp -- "$LIC_CONFIG_DIR/$LIC_BACKUP_NAME" "$LIC_BACKUP_DIR/$LIC_BACKUP_NAME"
done
cp -- "$LIC_BIN" "$LIC_BACKUP_DIR/original-licensing"
# Add the reviewed revision, startup flags, ZIP version and backup timestamp
# to a private configuration record alongside this snapshot.
chmod 600 "$LIC_BACKUP_DIR"/*
(
  cd "$LIC_BACKUP_DIR"
  sha256sum licenses.json private.pem public.pem signing.env original.zip \
    original-licensing > SHA256SUMS
)
sync -f "$LIC_BACKUP_DIR"

If no original ZIP is configured, omit that file and record that fact instead. The checksum manifest detects accidental copy corruption, not malicious changes by someone who can replace both files and manifest. Verify it against a trusted custody record when recovering. Copying keys under the state lock alone cannot make concurrent key rotation consistent; use immutable versioned configuration or a stopped-service backup. Do not copy .lock, temporary .licenses-* files, or a stale process PID as restore state. A new lock is created for a new path.

The service fsyncs its own state writes before rename and syncs the parent directory afterward. A storage failure after rename can still report failure although the state committed. Investigate the persisted state before assuming an operator mutation failed or retrying issuance. Capacity is bounded to 16 MiB JSON, 10,000 licenses, 100,000 approved origins and a 64 MiB startup ZIP; snapshot rewrites and reads serialize, so deployment capacity requires separate measurement.

Restore into a new isolated path

Keep a restored backend disconnected from production clients/proxies until its state is reconciled. An old backup can forget later revocations, deactivations, activations, approved-origin changes or newly issued licenses. Retaining signing keys can make the forgotten revocation especially misleading: an old download token can become acceptable again while it remains within its deadline. Backup integrity and successful startup do not establish current authorization state.

Recover into a new private directory, never over the live/default database. Keep the original source and its latest state unchanged for investigation. Verify the manifest and configuration record, then copy the selected state and material into a freshly created directory:

set -eu
umask 077
LIC_RESTORE_ROOT=/srv/original-licensing-recovery
LIC_RESTORE_DIR=$(mktemp -d "$LIC_RESTORE_ROOT/recovery.XXXXXX")
chmod 700 "$LIC_RESTORE_DIR"
(cd "$LIC_BACKUP_DIR" && sha256sum -c SHA256SUMS)
for LIC_RESTORE_NAME in licenses.json private.pem public.pem signing.env original.zip; do
  cp -- "$LIC_BACKUP_DIR/$LIC_RESTORE_NAME" "$LIC_RESTORE_DIR/$LIC_RESTORE_NAME"
done
chmod 600 "$LIC_RESTORE_DIR"/*
sync -f "$LIC_RESTORE_DIR"
# Remain OFF here pending reconciliation with the latest trusted authorization state.

The conservative reconciliation API accepts an explicitly acknowledged authoritative latest Go JSON path and its exact digest. It removes rights without importing newer grants or replacing customer/installation identity. It cannot discover freshness, automate authority verification, migrate ownership or reconstruct an operator audit journal. Use the latest independently trusted state and records to reconcile all mutations after the snapshot; if that evidence is missing, leave the recovered service off rather than assume old approvals and revocation flags are current. Prefer another new recovered path when selecting a newer trusted snapshot. Do not manually discard retained installation bindings to “fix” an activation error. Protect copies of operator-held bearer credentials; state hashes alone cannot be passed to the current revoke CLI.

After reconciliation, validate the selected files and permissions in a separate local fixture/network namespace with no production proxy or client attached. Start the reviewed binary with the recovered explicit state/key/package paths and a separate loopback port. Check known approved ownership, activation limits, signed active/denial states, license expiry and ZIP digest. Stop that rehearsal process before any approved cutover. For that isolated, reconciled rehearsal, use the recovered files rather than the live/default paths:

. "$LIC_RESTORE_DIR/signing.env"
# Select the identifier recorded with this recovered private PEM/public-key map.
LIC_SIGNING_KEY_ID=${LIC_SIGNING_KEY_ID:-fixture-v1}
"$LIC_BIN" --database "$LIC_RESTORE_DIR/licenses.json" serve --port 18090 --signing-key-id "$LIC_SIGNING_KEY_ID" \
  --private-key "$LIC_RESTORE_DIR/private.pem" \
  --original-package "$LIC_RESTORE_DIR/original.zip" --version "$LIC_PACKAGE_VERSION" &
LIC_REHEARSAL_PID=$!
# Run the controlled ownership/expiry/signature/download checks here.
kill -TERM "$LIC_REHEARSAL_PID"
wait "$LIC_REHEARSAL_PID"

Production routing and restart remain a separate operator change; this document does not perform it.

Schema corruption/invalid JSON causes startup to fail. Corruption encountered by a running process produces unsigned 503 and does not grant access. Do not delete corrupt state and let an empty path masquerade as recovery: preserve the original and investigate or restore into a new path. A missing state path is treated as a new empty store, so operational preflight must distinguish initial issuance from an unexpected loss. New process startup reloads state and signer/package files; each later request reloads live JSON while holding the state lock.

Signing identity, rotation and outages

Restoring the same RSA key, HMAC secret, current reconciled state and exact ZIP preserves the service’s signing identity. Previously issued download tokens can still be cryptographically valid until their five-minute/license deadline, but live revocation, deactivation, ownership and package-digest checks remain decisive. Restoring the same state with a changed HMAC secret rejects old download tokens. That change requires a service restart and fresh update metadata; it does not invalidate RSA-signed entitlements already cached by PHP.

RSA replacement requires trusted PHP public-key distribution. Configure an explicit signer ID (default fixture-v1) and distribute overlapping old/new public-key mappings before switching the loaded private PEM at restart. Do not treat an in-place PEM swap as graceful rotation. Clients configured with only the old public key will reject new signatures and retain only their prior verified deadlines. See rotation and recovery for the implemented API and the external trust/custody requirements. Neither key rotation nor a restored database can remotely erase an offline client’s previously verified cache.

During an outage, the PHP client retains verified validity/grace deadlines and never extends them because of transport/503 failures. Active entitlements refresh within 300 seconds, expire within 3600 seconds and have at most another 3600 seconds of grace, all capped by license expiry. A verified signed inactive, revoked or expired denial can invalidate cached premium access immediately. An offline client can retain access only until its existing deadline; an unsigned error is not a signed revocation. Package delivery is always a live server check: a revoked/deactivated installation cannot use a previously issued token after the server observes that state, and expired tokens/licenses are rejected. An already in-flight download checked before a mutation is not recalled by the mutation.

Isolated recovery evidence

Run the self-contained fixture from this directory:

go test -race -run '^TestBackupRestore' -v ./...

backup_restore_test.go creates only its own /tmp/original-licensing-restore-* directory, temporary RSA/HMAC material and synthetic original ZIP. It uses the fixed state lock for the snapshot, revokes the live synthetic source, restores to new paths, then starts fresh Go subprocesses with the real HTTP handler. Its clock injection exists only in the test helper. It confirms all of the following:

• The source’s revocation blocks its old token.
• An older backup restores ownership/site limits and RSA identity, but also accepts that same unexpired token again: revocation rollback is real.
• Changing HMAC material rejects the old token while retaining RSA entitlement identity.
• Restoring the latest revoked state yields a signed denial and rejects the token.
• A fresh process at license expiry rejects activation, checks, updates and download, and returns a signed expired denial.
• Corrupt restore JSON fails startup; private files stay mode 0600; recovery leaves the source unchanged and removes its subprocesses/temporary directory.

This demonstrates local snapshot/restore mechanics and their authorization ambiguity. It does not validate off-host backup availability, machine/disk failure recovery, operator audit completeness, a reconciliation process, public TLS or a paid artifact. Those remain separate operational gates.