The thing that clicked today: a monitor that can fix things is an actor, and a fleet already has enough actors; the monitor's job is to be the one thing that only ever reports.
The fleet had a dozen conventions and a growing number of machines, profiles and timers that were supposed to match them. Nothing checked that they did on a schedule.
Built / shipped
What it checks. Global contract parity across machines; every directed pair of machine-to-machine access, including the laptop; required services and timers per machine; required repository and convention paths. A dashboard surface shows the current state per check.
Three design choices. It writes atomic current state plus a bounded history of a thousand samples, and it preserves last-known-good evidence, so a probe outage cannot erase what was true. It classifies drift separately from probe failure, which is the distinction most naive monitors collapse and the reason most dashboards cry wolf: "the contract differs" and "I could not reach the machine to check" are different facts with different responses. And it never auto-repairs, deliberately, so the monitor can never become an unreviewed actor on the fleet.
Honest about its first run. The 319-test suite passed. The first real scheduled cycle surfaced two genuine corrections before the final cycle came back 26 of 26 OK: a wrong unit name for one exporter, and an interaction between the remote shell client and the service manager's mount namespaces that made a check fail for a reason unrelated to the thing being checked.
A follow-up pass added durability to the signal and rewrote both the compact module and the full page to explain the monitor in the same plain terms a human had needed during a walkthrough, without changing the collector or its machine-readable output.
Problems & fixes
The monitor's first two findings were about itself. That is the right order: a monitor that found drift on its first run before finding its own unit-name bug would have reported a drift that was a probe failure.
Decisions
Read-only, forever. Repairs are proposed to a human, never executed.
Drift and probe failure are separate states with separate colours.
Keep last-known-good, bounded, so a blind spell is visible as a blind spell and not as a fresh green.
Learned
Most dashboards that lie are collapsing "different" and "unknown." Splitting them is most of the value.
A monitor's authority should be the narrowest thing on the fleet. The moment it can act, its bugs become outages.
Run the monitor against itself first; its own unit name and shell interaction were the first two bugs it found.
Still open / next
The per-check thresholds and debounce behaviour took another pass two weeks later, once the monitor had enough history to show which of its alarms were real.