The exact product was already in Shopify, but five broad search results kept it out of the model's context. We moved the hard limit after ranking instead of before it.
September 23, 20266 min readSaytu team
Five Near-Matches
The product was in the catalogue, and the shopper had named it closely enough that its title matched all the important terms. The model still never saw it.
A shopper wrote "i looking for Alien stage T-shirt". Our product search turned that into an OR query: Alien OR stage OR T-shirt. Shopify returned five rows, and we passed those rows on as the products the assistant could reason about. In a store full of Alien Stage merchandise, that was enough to fail.
Five blind boxes matched two of the three terms. They filled all five places before the T-shirt whose title matched all three ever reached the model. The model then gave an accurate answer about the data we had shown it: it could see blind boxes.
The wrong answer started before generation.
The part that made this failure difficult to spot was that search had not returned nothing. It returned five products, so our fallback never opened the broader catalogue path. From the outside, that made the bug look like model behaviour.
The shopper asked for a T-shirt and the assistant talked about blind boxes. It was easy to conclude that the model had ignored a clear request, but it had not. We had given it a candidate set that excluded the product the shopper actually named.
Once that boundary had been crossed, the model had no way to recover the missing shirt. A product that never enters the context cannot be rescued by a better prompt, a larger model, or a more confident answer. The five-row limit had quietly become part of our ranking policy.
The OR query was there for a reasonable reason. Product search has to survive imperfect language: shoppers omit words, change phrasing, and rarely type catalogue titles exactly. A broad query gives related products a chance to appear when a strict match would return nothing.
The mistake was treating broad retrieval and final selection as the same step. With only five rows, Shopify's ranking decided which candidates the model was allowed to know existed. We were asking a broad search for recall and then immediately treating its first few results as precision.
Those are different jobs. The Alien Stage store made the difference visible because there were enough related products to crowd the result set. A blind box containing "Alien Stage" was relevant enough to enter the OR results even when the shopper had explicitly included "T-shirt".
Near-matches were not bad results. They were bad replacements for an exact match.
We changed product search into three passes. The first pass is precise: when there are at least two usable terms, we try them ANDed together. A shopper who names a product now gets the chance to hit that product before related merchandise competes for space.
If that precise search finds something, we return it. That is what the original search failed to do: the T-shirt matched Alien, stage and T-shirt together, so five products matching only part of the request should never have been able to hide it.
The broad search still exists because we still need it. If the precise pass finds nothing, we run the OR search, but we no longer take the first five rows as the answer. We read twenty-five candidates first, then rank them ourselves before cutting the set down.
Increasing the candidate window was not about showing twenty-five product cards. The shopper still needs a small result set; the larger window exists so ranking has something to rank.
A term appearing in the product title counts twice. A term found in the product type, vendor, tags or description counts once. That gives a product whose name closely follows the shopper's request more weight than something related through surrounding metadata.
Only after that ranking did we cut the list to five. Before, the first five rows decided what existed. After the change, twenty-five rows competed before five were chosen, so the model still got a bounded product set without letting the bound itself decide relevance.
The next day, the cap itself moved. Five had been doing a second job: the same number gated the prompt list, so a shop with six matching products could not show the sixth, and the model never learned it existed. Once the cards tag could narrow each answer on its own, the number only had to be a ceiling, and we raised it to eight. The rail shows about two and a half cards at 375px, so a shopper swipes to the eighth and never reaches a twelfth.
The precise AND pass introduced a different failure if we fed every leftover word into it. The shopper's message began with "i looking for" and, after query extraction, a lone "i" could survive as a search term. One dead word in an AND query is enough to empty the whole result.
So a single Latin letter is no longer treated as a term. We wanted "Alien stage T-shirt" to narrow the search; we did not want "i" to veto it. Stricter retrieval is only useful when the strictness comes from shopper intent rather than conversational filler.
The third pass is the catalogue shelf. That path still matters because search syntax cannot solve every product-discovery problem. A shopper and a merchant can describe the same item using different languages or different vocabulary, and sometimes there is no useful lexical match at all.
But the Alien Stage failure did not need a smarter fallback. The exact product was already findable; we were simply preventing it from reaching the part of the system capable of using it. That was why changing the model would have been the wrong fix.
AI shopping is moving towards agents that do more than answer product questions. They can discover items, compare them and increasingly carry the shopper through checkout. That makes the retrieval boundary more important, not less.
A checkout agent cannot choose correctly from a product it was never allowed to see. Structured catalogue access solves one part of the problem: the product is available to the system. It does not guarantee that the right product survives search, ranking and candidate limits long enough to reach the model making the recommendation.
Our shirt was in Shopify the whole time. The search returned results and the model answered faithfully from those results, but the shopper still could not get the product they had effectively named.
The fix was not to make the AI more persuasive. We stopped letting five near-matches hide the exact product before the AI ever had a chance to choose it.
