We matched a shopper's request for white to a product that really had a white variant, then showed its black main photograph. The reply was technically right and the experience was wrong. We changed the rule so product cards have to support the answer a shopper actually sees.
September 24, 20265 min readSaytu team
An AI shop assistant can give a factually correct answer and still leave the shopper convinced that it got the product wrong. We found that out with a colour question where the retrieval worked, the variant match was real, and the reply was accurate. The failure was the product card sitting directly underneath it.
The product was a Black/White tee. One of its variants was white, so when the shopper asked for white, the product legitimately matched. The assistant could therefore talk about the white option without inventing a colour or reaching outside the catalogue.
Then the interface showed the product's main photograph. That photograph was black. The shopper's next message was: "ทำไมส่งสีดำมา" - why did you send me the black one?
It is tempting to classify this as a presentation bug. The retrieval found a valid product. The catalogue data supported the reply. The language model did not hallucinate a colour. From the text pipeline's point of view, several important things had gone right.
But that is not how a shopper experiences an AI shop assistant. The shopper does not inspect retrieval traces and then grade the text independently from the interface. They see a message and the products immediately attached to it. If the message says white and the first visual evidence says black, those two pieces form one contradictory answer.
That distinction matters more as shopping interfaces become richer. Product cards are not decoration added after the AI has finished answering. They carry information of their own: the product name, image, price, link and the visual identity of the item the shopper is being invited to consider.
Once we treated the card as part of the answer, the bug became much easier to describe. The question was no longer simply whether we retrieved a product with a white variant. It became whether what the shopper sees supports what we just told them. In this case, it did not.
The awkward part was that neither side was individually false. The catalogue really did contain the white variant. The product's main image really was the black version. Showing either fact by itself was defensible.
Showing them together without explaining the difference was not. This is a common shape of problem in commerce because a product record and a product card do not necessarily represent the same level of detail. A catalogue can describe several variants under one product while the card has one primary photograph. Matching the product record therefore does not guarantee that the photograph represents the attribute that caused the match.
For an ordinary search-results page, a shopper may tolerate that distinction. They can open the product, inspect the variants and work out why it appeared. A conversational interface makes a stronger promise. The assistant has just responded to a specific request. The cards underneath look like evidence for that response. The closer those cards sit to the sentence, the more strongly they inherit its meaning.
That made our rule straightforward: when a product matches only because of a variant, we cannot silently rely on the product's main image to explain the match.
The change was deliberately narrow. A product that matches only through a variant now has two acceptable outcomes. If the reply explicitly explains the relevant variant, the product can remain. The shopper has enough information to understand why the card belongs there even if the primary photograph represents another variant.
Otherwise, the product stays off the card rail. We preferred that to pretending the main image could communicate information it does not contain. We also preferred it to treating the whole product as a bad match, because the catalogue had answered the shopper's request correctly. The problem was not the existence of the white variant. The problem was presenting a black photograph as though it visually confirmed the white answer.
That difference is small in code and large in the conversation. It changes the standard from "the product technically matches" to "the recommendation makes sense in the form the shopper actually receives it."
We later found the same problem while arranging a demo reply. The sentence introduced what was in stock in white, size M, while the product row led with a black tee. Again, the individual pieces could be justified. Together, they told two different stories.
The useful design rule was not to ban the black product. The reply itself had room for another item in the shopper's size. What mattered was correspondence between the sentence and the order of the cards: the products answering "white, size M" had to lead, while the additional option belonged after them.
A product row has grammar even though it contains no sentence. First position says "this is the thing I meant first". A group directly under a reply says "these are the products that answer what I just said". Reversing that hierarchy can make accurate data look inaccurate.
We started with a bug that looked almost absurd: the assistant correctly answered a request for white and immediately showed black. The useful lesson was not that colour matching is difficult. The catalogue had already matched the colour. The lesson was that correctness in a shopping assistant extends beyond the generated sentence.
Retrieval has to find the right product. The reply has to describe it truthfully. The variant that caused the match has to remain understandable. And the card presented as evidence has to agree with the sentence above it, or make the difference explicit.
A system can pass the first three checks and still fail the shopper on the fourth. That is why we no longer treat product cards as an output that comes after the answer. For a shopper, they are part of the answer. If the words say white and the product underneath looks black, the technically correct explanation does not rescue the experience.
The shopper already told us the result more clearly than any internal metric could: "Why did you send me the black one?"
