Build log

Everything we have written about Build log.

Latest

Five Near-Matches

Build logSeptember 23, 20266 min read

Five near-matches kept the exact product away from the model

The exact product was already in Shopify, but five broad search results kept it out of the model's context. We moved the hard limit after ranking instead of before it.

More from the team

The AI started talking again. The human request stayed open.

A handoff nobody picked up used to silence the assistant forever. We now let the AI resume after staff silence without clearing the human request, because helping again is not the same as resolving it.

Build logSeptember 23, 20265 min read

Twenty chunks did not mean we had the whole document

A source with twenty stored chunks could mean either complete or truncated. We started recording the pre-cap count so cut, complete and unknown became three measurable states instead of one guess.

Build logSeptember 23, 20266 min read

Semantic retrieval was off in production, and the fallback hid it

Semantic retrieval had been failing with 401s while keyword fallback kept the assistant looking healthy. The first embeddings backfill exposed that production was authenticating with a literal environment-variable name.

Build logSeptember 23, 20265 min read

Right Match Wrong Image

The product match was right; the card still made it look wrong

A shopper asked for white, and we found a product that really had a white variant. But its main card image was black. We changed the rule so a variant-only match must be stated in the reply or the card stays off the rail.

Build logSeptember 22, 20264 min read

Nine tied rows were enough to make the same question retrieve different sources

Nine tied policy rows produced two different top-threes on consecutive calls. The fix was not a better ranking — it was making the ranking we already had give the same answer twice.

Build logSeptember 21, 20264 min read

The model only gets to paint inside three ellipses

One uploaded drawing becomes a face that blinks and speaks. Identity holds because everything outside three marked ellipses is the merchant's own bytes, not because the model was asked nicely.

Build logSeptember 21, 20264 min read

The metafields that crowded out the shop policies

Six policies against two and a half thousand metafields, one shared limit, and a refund policy the assistant never saw. Nothing was broken, and nothing reported it.

Build logSeptember 21, 20263 min read

What verified means when a source is switched off

A control that appears to work and does not is worse than one that visibly fails. Why retrieval reads the switch rather than the rows, and why the inspector is the same code path.

Build logSeptember 21, 20264 min read

Four records, and what the assistant is actually shown

Everything a store has written, narrowed to about two thousand characters for one question. The budget, the two ranking lanes, and the floor below which nothing counts.

Build logSeptember 21, 20264 min read

108 unread

We published 108 articles nobody read

They were written by a model, posted through our own API, and made public on arrival. One of them had no text in it. Here is the count, and what we changed.

Build logSeptember 21, 20264 min read

Draft vs live

A draft should never take the live answer down

Asking for a draft of an FAQ entry quietly unpublished the one the store was already giving. A write that sets a status is a visibility change.

Build logSeptember 21, 20264 min read

When every score ties, the ranking is a coin flip

Keyword scores tie constantly, and a tie handed the order to whatever the database returned. The fix is deliberately not an improvement.

Build logSeptember 21, 20263 min read

One store's reply

A turn id is whatever the client says it is

A retry handed one store the reply written for another. The key was a string the browser chose, used across a collection that spans every tenant.

Build logSeptember 21, 20264 min read

What it keeps

Three windows, and what the assistant is allowed to keep

A transcript for a day, the subjects for a month, personal data anonymised at ninety. Three numbers that used to disagree, and why there will not be a fourth.

Build logSeptember 21, 20264 min read