Everything a store has written, narrowed to about two thousand characters for one question. The budget, the two ranking lanes, and the floor below which nothing counts.
September 21, 20264 min readSaytu team
Ask a storefront assistant whether a jacket runs small and something has to decide, in the moment, which parts of that store's own writing the model is allowed to see. Not all of it. A shop with two hundred products has far more text about itself than fits in a single request, and most of it has nothing to do with sizing.
Here is the whole budget, which is smaller than people expect.
Whatever a store has written — its policies, its pages, its FAQ entries, its collection copy, its product metafields — at most four pieces of it reach the model for any one question, and each is trimmed to 520 characters.
That is roughly two thousand characters. About one page of a paperback.
The number is not an efficiency exercise. The model's reply is capped at 1,600 tokens, and this block is additive: the same request already carries the shopper's cart, the product they are looking at, their order if we have it, and the conversation so far. Knowledge has to fit in what is left, and the reply has to fit after that. Every character spent on a policy is a character not spent answering.
Once you accept four, the interesting question stops being "how much can we send" and becomes "which four", which is the rest of the pipeline.
Candidates are ranked twice, by two different methods, and the two are merged.
The first lane is semantic. Pages are chunked and embedded, the shopper's question is embedded, and chunks are scored by cosine similarity. Anything below 0.25 is treated as unrelated and dropped outright, however well it scores against the others — a floor, not a ranking. At most three of the four slots can be filled this way.
The second lane is keyword. The question is tokenised, stopwords removed, and candidates score by the terms they actually contain.
We keep both on purpose. Semantic search finds the page that answers the question without using any of its words, which is most of what shoppers ask. Keyword search finds the exact product name, the SKU, the one term a shopper typed because they read it off a box, which embeddings routinely blur into "something like it". Neither is reliably better. They fail differently, which is the useful property.
The ranking only sees candidates that were fetched, and the fetch has its own limits: 200 candidates in total, drawn from seven source types, each with its own quota.
Those quotas exist because of a specific failure, and it is worth its own post. The short version: one shared limit across all types let sheer row volume decide who reached the ranking at all, and a store's six shop policies lost to two and a half thousand product metafields before a single thing was scored.
The ordering of the fetch matters as much as its size, for a reason that took us a while to see: tied scores used to hand the choice to the database, so the same question could fill these four slots differently on two consecutive days.
A shopper asks about returns on a sale item. The question embeds close to a chunk of the store's refund policy, which clears the floor and takes a slot. The words "sale" and "final" hit a line in an FAQ entry, which takes another. Two slots are left and nothing else scores, so two slots go unused, because there is no rule that says four must be filled.
The model gets roughly a page of the store's own words, written by the merchant, about exactly the thing that was asked. Then it answers in the store's voice, and everything it said can be traced to a row somebody at that store wrote.
That last part is the whole reason for the machinery. Not that the assistant knows things. That somebody can check where it got them.
