← All posts

July 1 - a spare radio, a 0.06-second failover, and two interfaces that renamed themselves

The thing that clicked today: on modern Linux, adding a card can rename interfaces you did not touch, and every config that hardcodes an interface name is a bet that the bus never changes.

The GPU box needed a faster wireless uplink. One new card went in.

Built / shipped

The cascade. The new card took a name the onboard radio used to have, the onboard radio moved to a new name, and an unrelated wired port shifted as collateral. That mattered because the service providing network address translation for the isolated storage subnet hardcoded the old radio's name in a kernel-parameter call: the missing interface made that call fail, the failed call failed the whole unit at boot, and the failed unit meant the DHCP server for that subnet never launched. The subnet had no internet and no addressing, while the box itself looked perfectly fine.

The rebuild. The two radios became a primary and hot spare pair through route metrics, with traffic going out through both, so failover needs no service restart. The per-radio kernel parameters were marked ignore-failure, so a renamed card can never take the unit down again. A cleanup step runs before start, deleting firewall rules if present and killing any stale DHCP server by its pid file, so a restart no longer duplicates rules or double-launches the server: the unit is idempotent on restart.

Measured, not assumed. A scripted link-down test on the primary: the default route flipped to the spare in 0.062 seconds. Bringing the primary back: about 19 seconds of reassociation and address lease, then traffic returned to the primary automatically. Throughput went from a 185 to 239 megabit baseline on the old radio to 291.76 down and 223.17 up on the new one. Reboot survival verified without rebooting, by checking the generated network units carry the metrics.

Problems & fixes

The original plan was to hand the whole job to an AI coding tool running on the box itself, which failed immediately: the box had no working uplink, and the tool needs the network to reach its model. A bootstrap chicken-and-egg that made the work a human-at-the-keyboard job until the first radio was up.

The laptop cannot verify the NAT path from where it sits, by design: its own default route goes out its own wireless. The verification had to run from a machine on the storage subnet.

Decisions

Primary and hot spare by route metric, no restart required to fail over.

Ignore-failure on every per-interface kernel parameter, so a rename is a degraded radio, not a dead gateway.

Idempotent start: clean up before you add.

Audit every file that hardcodes an interface name after any card change, not just the one you expected to change.

Learned

One stale interface name can cascade to no DHCP. Dependency chains in service units do not announce themselves.

A failover number you have measured (0.062 s) is worth more than a design you have described.

The tool that would do the work needs the thing you are fixing. Bring the first link up by hand.

Still open / next

The mesh's storage subnet now survives either radio failing, and the service survives a card being renamed; what it does not yet survive is the box itself going away, which is the standby work that came later.