AI to Answer Product Questions: Why Generic Chatbots Get Specs Wrong
AI to answer product questions needs grounding in your real catalog and site content, not general model knowledge. How it works, and where chatbots fail.
An AI that answers product questions well is doing something narrower than it sounds: matching a shopper's specific question, "will this fit a 2019 model," "is this machine washable," "what's the difference between the two finishes," against your store's actual product data and site content, and saying only what that data supports. Get the grounding right and it looks like magic. Get it wrong and the bot invents a compatible size, a material the product doesn't use, or a feature that doesn't exist, confidently enough that the shopper believes it until the return request comes in.
This post covers the three kinds of questions that actually stall a purchase, why a chatbot running on general model knowledge gets them wrong, what the current crop of Shopify Q&A apps does and doesn't cover, and what grounding actually requires before you trust any of it with your catalog.
The three questions that stall a sale
Pre-purchase product questions cluster into a small set of shapes, and each one needs a different kind of correct answer.
Spec questions ask for a fact that's either on the product page or isn't: dimensions, materials, weight, capacity, ingredients, care instructions. These look easy because the answer is usually written down somewhere. The failure mode is a bot that paraphrases instead of quoting, and turns "hand wash cold, lay flat to dry" into "machine washable," which is a wrong answer with real consequences.
Compatibility questions ask whether one product works with another, or with something the shopper already owns: "does this fit a 2021 Tacoma," "will this charger work with the Pro model or just the base one." These require cross-referencing two pieces of catalog data instead of retrieving one product's page, and it's the category most likely to expose a bot that's guessing from general knowledge of "how products like this usually work" instead of your specific SKUs.
How-to questions ask what to do with the product after buying it, or how to decide between two similar ones: "how do I season this pan the first time," "what's actually different between the medium-firm and firm mattress." These depend on content that often lives outside the strict spec sheet, in a buying guide, an FAQ, a blog post, or a care page, which means the AI needs more than the product database to answer them; it needs whatever the store has published about the product anywhere on the site.
All three show up constantly. Baymard Institute's research on ecommerce product pages found up to 95% of shoppers relying on reviews to fill in exactly this kind of gap when the product page itself doesn't answer it, and separately found that most leading sites still ship "mediocre" or worse product-page UX, meaning the information a shopper needs is often technically present somewhere on the site but not where or how they need it in the moment they're asking.
Why generic chatbots get this wrong
A general-purpose LLM chatbot answers from its training data and whatever general knowledge it has about products like yours, not from your specific catalog, unless something explicitly feeds it your data first. Ask it whether your jacket comes in a men's large and it will give a plausible, fluent answer based on how jackets like yours are usually sized, which has no connection to whether your specific SKU is currently in stock in that size.
This is retrieval-augmented generation's whole reason for existing: a RAG-grounded assistant retrieves the actual, current product record and site content before answering, rather than answering from memory. Recent research on e-commerce question answering documents this directly, building retrieval systems specifically because plain LLM answers on product questions drift from the actual catalog when there's no retrieval step forcing the model back to real data (see the arXiv paper on contextually aware e-commerce product Q&A and the related graph-enhanced retrieval work for e-commerce support). Grounding doesn't make hallucination impossible, since a model can still misread a retrieved document, but it removes the biggest source of error: making something up because nothing was retrieved to check against in the first place.
The practical tell is speed of failure. A well-grounded assistant is slower to convince and faster to be right, because it's constrained to quoting or paraphrasing something real. An ungrounded one answers instantly and sounds equally confident whether it's right or completely wrong, which is the actual danger: a customer can't tell the difference from the outside, and finds out only after the product arrives and doesn't fit.
What's already on the shelf: Q&A widgets
Shopify's app store has a growing set of AI Q&A tools built specifically for this problem, and it's worth knowing what they cover before assuming you need something bigger. Apps like Answer It and InstantQ add an AI-answered Q&A section to individual product pages, drawing on that product's own listing to draft answers, with the option to require review before an answer publishes. Similar tools like eFAQ and StoreFAQ generate FAQ content per product or per store from the same underlying data.
These apps solve the spec-question case well: they're grounded in the one product page the shopper is already looking at, which is the easiest version of this problem. Where they run out of runway is the other two categories. A Q&A widget scoped to a single product page usually can't cross-reference a second product for a compatibility question, and it typically doesn't reach beyond the product listing into a buying guide, size chart, or policy page for a how-to question that spans more than one source. It's also page-bound by design: the shopper has to be on that exact product page to ask, not mid-conversation somewhere else on the site.
None of that is a knock on the category. A dedicated Q&A widget is often the right, cheap first step, and several of them are free or close to it. It's a narrower tool solving a narrower slice of the problem than a conversational agent with catalog-wide access, and knowing which slice you actually need is the first decision to make.
What real grounding requires
For an AI to answer the compatibility and how-to questions on top of the single-product spec questions, it needs two things a page-scoped widget doesn't have by default: live access to the whole catalog, so it can check one product's specs against another's, or current stock and variants rather than a stale export, and ingestion of the store's broader content beyond the product listing: size charts, care pages, buying guides, and FAQs, so a "what's the difference between" question can pull from wherever the actual answer lives.
Freshness matters as much as breadth. A catalog snapshot from three weeks ago will confidently answer a compatibility question about a product that's since been discontinued or changed. The gap between "grounded" and "grounded and current" is where a lot of otherwise well-built tools quietly fail; ask any vendor how often the underlying data refreshes before trusting the answers it gives.
Where Ernest fits
Ernest is one AI agent that plays three roles for a Shopify or WooCommerce store, and answering product questions is its product-agent side. It's grounded in the store's live product catalog and everything ingested from the store's own site: descriptions, specs, FAQs, size charts, automatically on connect, and it isn't scoped to a single product page, so a compatibility question that spans two SKUs or a how-to question that lives on a buying guide can be answered in the same conversation as a spec question about the product in front of the shopper.
That grounding is also its boundary. Ernest answers from what the store has actually published; it doesn't fill a gap in your product content with a plausible guess, which means the honest fix for an unanswerable question is usually adding the missing detail to the product page or FAQ, not expecting the agent to know it anyway.
Other limits worth knowing before comparing it to a page-scoped Q&A widget:
- No manual review queue for answers. Unlike some Q&A apps that let a merchant approve each AI answer before it publishes, Ernest answers live in the conversation from what it's ingested; the control point is what content the store gives it, not a per-answer approval step.
- Channels are the storefront chat widget and email, with escalation to a human by email when a question needs a person. No SMS, voice, or social channels.
- No proactive outreach. It answers when a shopper asks; it doesn't pop open based on browsing behavior or send follow-up messages.
- No medical, legal, or safety advice beyond what the product content states. For anything with real ambiguity or stakes, a grounded agent should point to a human, not stretch its source material to cover it.
The same agent handles the two jobs that pair naturally with product questions: pre-purchase sizing, shipping, and stock questions on the sales-agent side, and guided product discovery for the shopper who doesn't know what they want yet, covered in our breakdown of AI shopping assistants for ecommerce. It's a different job again from algorithmic recommendations, the static-widget comparison covers that distinction if a shopper's actual need is "what should I look at" rather than "does this specific thing work."
Pricing is conversation-based: a free plan with 100 conversations that never expire, a real free plan rather than a trial, then $49/month for 500 conversations, $149 for 2,000, and $299 for 10,000. AI is included at every tier, with no separate per-seat or per-resolution charge.
How to check before you trust any of it
Whatever tool you're evaluating, test it against your own hardest cases before your catalog's reputation is on the line. Pull your ten most common product questions from support history, split evenly across specs, compatibility, and how-to, and run each one against the actual candidate tool on your actual catalog, not a demo store. A spec question answered wrong is a factual error you can point to. A compatibility question answered wrong is a return you're paying shipping on both ways. Score the tool on how it handles the one it doesn't know: a grounded assistant should say so and hand off, not guess with confidence.