AI search has spawned a fast-growing measurement market, but most prompt tracking tools still cannot answer the one question leadership keeps asking: did this change make the business better? Ecommerce teams have a structural advantage in answering that question, because products still need to be made and shipped. That makes large retail sites an ideal environment for running controlled GEO experiments on the same scalable templates that already power SEO testing.<\/p>\n
This guide breaks down how to move past prompt tracking dashboards toward real A\/B testing for generative engine optimisation, from writing a testable hypothesis to reporting net business impact.<\/p>\n
Prompt tracking tools sample brand mentions, citations, AI visibility, and share of voice across ChatGPT, Perplexity, Gemini, AI Overviews, AI Mode, and other surfaces. Some of that data is useful for debugging and surfacing early warnings about how a brand is described. The problem is that prompt tracking is being asked to do too much.<\/p>\n
Prompts are personal. They are often long, specific, and shaped by previous conversations. The same user may ask a follow-up that changes the whole context, and different users may receive different answers. The universe of possible prompts is effectively infinite, which is why the situation has been described as a “search volume one” world. A dashboard that samples a small portion of reality cannot prove that a website change improved business outcomes.<\/p>\n
That is the gap between visibility and impact. Prompt tracking can help form hypotheses and notice issues, but it should not be the main evidence that a GEO programme is working.<\/p>\n
People still need a manufacturer or retailer to make and ship products. ChatGPT can help with research, comparison, and decision-making, but the commercial question remains familiar: will the customer buy from you?<\/p>\n
That reality makes ecommerce different from media or publishing models where an AI summary can more directly substitute for the content itself. On retail and transactional sites, product pages and category pages are scalable templates that can be tested. PDPs, PLPs, category pages, internal search pages, faceted pages, buying guides, and related content blocks can all be split into variant and control groups, changed, and measured. The discovery interface may look like a mix of search engine and chatbot, and the process may be more conversational, but the testing mechanics look familiar.<\/p>\n
AI discovery relies on two broad information sources. The first is training data, which shapes a model’s understanding of language, entities, relationships, associations, and brand context. Training data is largely fixed until the next training run, which is a slow timescale and a poor basis for a strategy that amounts to waiting for the next model to like you more.<\/p>\n
The second source is retrieval. When a user asks a question, the model may bring in fresh information through retrieval-augmented generation (RAG). In practice, the model may open dozens of background tabs across the web and synthesise an answer from them.<\/p>\n
For ecommerce, retrieval is essential because product recommendations need freshness. AI systems need up-to-date information on current stock levels, today’s price, active discounts, latest reviews, delivery options, local availability, and new launches. That often means retrieving pages, feeds, or search results from the live web, which is why product feeds, structured data, PDPs, pricing, availability, and product attributes can all become part of how machines understand and recommend products.<\/p>\n
A user may type one long prompt into an AI system, but the model may break that task into many background searches: product comparisons, reviews, pricing, availability, best options for a use case, brand reputation, delivery details, and other supporting information. The user sees one answer, but behind it there may have been many searches.<\/p>\n
The old keyword-first model asked which keyword you were targeting, where you ranked, and what the search result looked like. In AI discovery, the hidden fan-out queries may be where the real retrieval happens. Teams usually cannot see all of those fan-out queries, and they may only get clues rather than a clean list. That is one reason testing becomes more important. A team can make a change and measure whether it improved LLM referrals, Google organic traffic, or the net business outcome, without needing perfect visibility into every hidden query.<\/p>\n
In traditional SEO, a successful test usually works through one of three mechanisms: targeting new keywords, improving rankings for existing keywords, or changing the search result appearance so more searchers click through. GEO has analogues, but the language changes.<\/p>\n
A GEO hypothesis might aim to target new fan-out queries, improve visibility for existing fan-out queries, influence the summary returned by an LLM, or make a page, product, or brand easier for the model to recommend. The fourth mechanism is especially interesting because it feels a little like conversion rate optimisation, except the “converter” is partly the machine. The question becomes whether the page has provided the information the model needs to confidently recommend the product. That could involve product detail, comparison language, reviews, freshness, structured data, key features, FAQs, delivery information, stock information, or buying guidance.<\/p>\n
A weak GEO hypothesis says, “This might help AI visibility.” A stronger hypothesis says, “Adding clearer product suitability information to PDPs may help models retrieve and recommend these products for more specific fan-out queries, while also improving confidence in the AI-generated summary.” That framing gives the team something testable.<\/p>\n
For large ecommerce sites, GEO testing happens on the same scalable surfaces that SEO testing already uses. That includes product detail pages, product listing pages, category templates, buying guide modules, comparison content, FAQs, review summaries, key feature summaries, internal linking modules, structured data freshness indicators, product feed-aligned content, and availability and delivery information.<\/p>\n
The mechanics look similar to SEO A\/B testing. A team makes a change to a variant group of pages, compares performance with a control group, and measures the result. What changes is the journey being measured. In traditional search, a user might open several tabs, compare sources, read reviews, check products, and then come to the site. In AI discovery, more of that research may happen inside the conversation. The model reads, compares, summarises, and narrows options before the user arrives, and the site may only see the final click. That makes the click more valuable in some cases, but harder to interpret, since a lower number of clicks can still represent more qualified visitors when more research happens before the visit.<\/p>\n
Many GEO changes could plausibly help SEO too. More useful content, better structure, fresher product information, clearer summaries, stronger internal links, and better structured data can all carry SEO hypotheses. But that overlap does not mean every GEO-positive change is SEO-positive.<\/p>\n
Most practical GEO work today still reaches AI systems through search-related retrieval, which creates overlap with SEO, but overlap is not the same as sameness. A change can help an LLM understand and summarise a page while hurting Google organic performance. A change can make a page richer for AI retrieval while making it bloated, duplicative, or less effective in traditional search.<\/p>\n
This is where single-channel measurement becomes dangerous. A team could look only at LLM referrals, see a positive result, and roll the change out, but if Google organic traffic falls by more in absolute terms, the business loses. That is the bigger risk with guessing what works in GEO: a visible win in one channel can hide a larger loss elsewhere.<\/p>\n
The clearest public example of this dynamic comes from SearchPilot’s work with Omio, a travel comparison platform. SearchPilot has published the full Omio GEO A\/B testing story, and the lessons go beyond confirming that GEO can be tested.<\/p>\n
In one test, adding brand USPs increased LLM traffic by +18%. In another, adding structured key takeaways performed positively for LLM-driven traffic but would likely have hurt Google organic sessions by -6.5%. Omio chose not to roll out that second change and developed follow-up iterations instead. The AI result looked positive on its own, but the business result was not. Without measuring Google organic performance at the same time, the team could have shipped a net-negative change, which is the strongest argument against treating GEO as a checklist. A tactic can be directionally plausible and still wrong for a specific site, page type, or business goal.<\/p>\n
Executive attention to AI search has spiked, and the questions keep coming: “Are we ready?”, “Are we doing the right things?”, “How do we go faster?” Those are good questions, and they need better answers than another dashboard of AI visibility scores.<\/p>\n
When reporting GEO work to leadership, frame the result as a net business outcome rather than a single-channel win. Show the LLM referral change alongside the Google organic change and any movement in downstream commercial metrics. Make clear which changes were rolled out, which were held back, and why. The point is to replace guesswork with evidence that connects each GEO test to the broader search and traffic picture, so leadership can see whether the programme is actually working.<\/p>\n
GEO stands for generative engine optimisation, the practice of making product and category pages easier for large language models to retrieve, understand, and recommend. For ecommerce, it centres on product detail pages, product listing pages, structured data, product feeds, and the freshness signals that AI systems use when answering shopping questions.<\/p>\n
Prompt tracking samples a small portion of an effectively infinite universe of personalised prompts, and it cannot link a website change to a business outcome. It can help with debugging and early warnings, but it cannot prove that a change should be rolled out across a large ecommerce site.<\/p>\n
Yes. SearchPilot’s work with Omio showed a change that lifted LLM-driven traffic while projecting a -6.5% hit to Google organic sessions, which is why Omio held the change back. That is the core reason GEO and SEO should be measured together rather than in isolation.<\/p>\n