The thing that clicked today: a guard that can only ever pass is not a guard, and the only way to know which kind you wrote is to point it at the real broken thing and watch it fail.
Prospect is my local-first job-application tracker. Two earlier sessions had each exposed the same class of defect one layer apart: first the service was found running a process older than the committed source; then the deployed frontend bundle was found to be 3.65 days older than the commit that added a feature. Nothing in the project compared the running artifact to the source it was supposed to be built from.
Built / shipped
The build-freshness guard, and what it checks. Three filesystem-only checks, no network, no running service, no database, so the guard cannot be defeated by the environment it runs in. Freshness: the deployed bundle's timestamp must not be older than the newest build input. Content: the bundle must contain the three feature markers the last two sessions added. Half-deploy: the asset named by the deployed index page must actually exist on disk, with the entry bundle resolved by parsing the page the way the server serves it, never a hardcoded hash.
Validated against the real defect, not a mock. The pre-rebuild stale bundle still existed. Run against the live bundle: 5 of 5 pass. Run against the stale one: fails 2 of 5 with a non-zero exit. And three negative controls ship inside the test file, so the guard's own ability to fail is re-proven every time the suite runs.
Recon corrected the scope twice before a line was written. First: a scan that watched only the app's source directory would have left a hole exactly as wide as the one being closed, because a shared design-system tree is aliased into the build and imported by the app; edit a shared component, rebuild nothing, guard stays green. Both trees are scanned, plus the index page and the build config. Second, a tooling lesson: asking a worker for twenty lines of context around a match returned two, silently, and the test-runner script that mattered was simply absent from the output. The fix was to stop asking for excerpts of a 359-byte file and read the whole thing.
The sibling guard for the running process. A test that compares the service's actual start time against its runtime inputs, which correctly failed on the live system and named the newer file.
A guarded release workflow wrapping it all. Stage the build, validate the staged directory (it must contain the index page, a script entry, a stylesheet entry, and every referenced asset, and the freshness guard runs against the staged directory before promotion), promote, post-verify with the complete test suite and the exact promoted HTML and assets resolving, and automatically roll back on failure. The displaced release is retained, so the rollback is itself reversible. Three workflow tests, five staged checks during the real deployment, 150 of 150 post-promotion.
The API trust boundary, the same day. The API had been reachable by every peer that could reach its port. A middleware mounted only on the API path now accepts exactly three bounded classes: loopback, for server-local health calls; the private overlay network's address ranges; and an access-proxy assertion, honoured only when the immediate socket peer is in an exact pinned allowlist. It reads the real socket peer address and never trusts a forwarding header, so a forged authentication-looking header from an untrusted address on the home network is still a 403. The proxy is the authenticated enforcement point, and the app additionally requires the request to have arrived from that pinned peer. Static assets keep their existing reachability while their data calls are protected. Verified end to end rather than by unit test alone: 7 of 7 boundary tests, 157 of 157 full suite, a request over the overlay network returning 200 and one from the home network returning 403 from the same machine.
Problems & fixes
The freshness guard's obvious implementation would have been wrong. Grepping the bundle for a few literals looks like a check and proves nothing about staleness or half-deploys. The mtime comparison against every real build input, plus the negative controls, is what makes it a guard.
A worker's truncated output is not an error. It is silently short. The rule that came out of it: do not ask for excerpts of a small file; read the whole file.
Decisions
Filesystem-only checks, so the guard cannot be fooled by the environment.
Negative controls in the test file, so "can this fail" is re-asked on every run.
Every real build input in the scan, including the shared tree aliased into the app.
Release = stage, validate, promote, post-verify, auto-rollback; rollback retains what it displaces.
Trust by socket peer, never by header; three bounded classes; everything else 403.
Learned
Two wrong-artifact incidents one layer apart are a missing guard, not two accidents.
Validate a guard against the real stale artifact while you still have it. The negative controls keep that proof alive after the artifact is gone.
A shared tree aliased into a build is a build input. Scan it or the hole stays open.
Reading the peer address from the socket is the only version of "who called" that a caller cannot forge.
Still open / next
One guard was deliberately left unclosed for twelve days afterwards, because its definition of done required a real live double-click in a browser and no automated seat could drive one; it waited for me. That is the right shape for a check that only a human can perform.