The thing that clicked today: a search index built on one machine and queried on another is only correct if both compute exactly the same numbers for the same input, and "exactly" has to be proven, not assumed.
Loupe runs on a small always-on box. Heavy machine learning runs on the GPU box. So searching photos by meaning ("dog on the couch", "beach at sunset") has to split in two: turn every image into a vector on the GPU box once, then turn each query into a vector on the small box at search time, and compare. If the two sides disagree by even a little, search quietly gets worse with no error to tell you.
Built / shipped
A runtime decision made on the merits. The GPU path through the usual deep-learning framework was proven open with a smoke test first, so choosing the lighter inference runtime instead was a real choice, not a workaround: one runtime across the whole stack (faces and other models already used it), portability to machines without a GPU, and deterministic output.
Plumbing proven on a small warm model before pulling the real one. A 512-dimension model that was already on disk validated the whole pipeline and the symmetry check on a 600-image sample with zero failures and zero downloads. Only then did the production model come in: a 1152-dimension image-and-text model, exported and pinned.
The recipe, pinned as a hard requirement. Model, image preprocessing, tokenizer, pooling, normalization and dimension have to be byte-identical on the index side and the query side. Written once as a small module that both machines import, so there is no second copy to drift.
The full pass. 61,971 images in about three hours twenty-six minutes, starting around 15 per second on cached files and settling to 9 to 10 per second through a mixed HEIC and raw corpus, run under a CPU quota so the box stayed usable. Result: 61,946 embedded, 25 failed (21 corrupt, 4 missing), zero silent skips, a 290.8 MB index. Transferred to the small box with matching checksums on both ends.
Cross-machine validation. Two reference queries embedded on the small box's CPU reproduced the GPU box's top-5 results exactly: same assets, same order, distances agreeing to about one part in a hundred thousand. That is the whole point of the pinned recipe, and it held.
The query side, then the feature. A search module on the small box that imports the same recipe, loads the text model on the first query rather than at startup (it is 2.7 GB in memory on a box with 7.6 GB), and hands the nearest neighbors to the existing item renderer. Behind a flag, off by default, then on, then a search box in the app with an owner-mode toggle.
Problems & fixes
The premise was false. The plan said "supersede the existing search backend." A read-only look at the live server found no search route, no search UI, and no dependency on the service the plan assumed. There was no existing search feature. What was planned as a risky migration was a zero-risk addition of something new. Always read the running code before trusting the plan's description of it.
The similarity function was not the one I thought. The vector store's nearest-neighbor match defaults to plain Euclidean distance, not cosine similarity. For normalized vectors the two give the same ranking but different values, which is easy to misread as recipe drift when comparing numbers across machines. Noted, and the query path re-ranks a candidate pool by cosine explicitly.
Twenty-five failures that were one failure. The 25 failed images looked like corruption. A follow-up found every single Canon raw file in the corpus (20 of 20) failed, because the image library cannot open Canon's container format while other raw formats worked. A raw-decode fallback fixed 24 of 25; one genuinely corrupt file stays failed. The fallback touched only the image-loading step, so query symmetry was preserved, and a re-embedded raw file's nearest neighbors turned out to be its own burst siblings, which is the right answer. The same library limitation affected thumbnails and face detection, so the fallback went there too.
Memory on the small box. The service went from about 175 MB to 2.28 GB after the first query loaded the text model. Comfortable alone, with a note to watch it if a heavy job lands alongside.
Decisions
One inference runtime across the stack, chosen after the alternative was proven to work, for portability and determinism.
Pin the recipe in one module both machines import. A second copy is a drift waiting to happen.
Load the text model lazily on first query, not at startup, because 2.7 GB resident on a 7.6 GB machine is a cost worth deferring.
Ship behind a flag, default off, then flip. Verified at each step.
Learned
Recipe drift is the silent failure mode of split-machine search. Two reference queries reproducing the other machine's results exactly is the test that matters, and it is cheap.
Read the live code before the plan. The premise was wrong and the work was easier than described.
A default distance metric is a decision someone else made. Know which one you are ranking by.
A cluster of failures with one cause looks like scattered corruption until you group it.
Still open / next
The index is on the small box and search is live. Next is the scoring work that sits on top of these same vectors: replacing a score that only exists for my library with one anyone can run.