An AI selection system lifts one laptop from a changing shortlist before a verification checkpoint.

Your AI Shopping Assistant Sounds Certain. Ask It Again.

by

By Si Fisher · Published September 22, 2026 at 4:19 PM PDT

Quick answer

AI shopping advice has the confidence of a salesperson who has already spent your money. Of the three AI shopping answers I compared, two picked the same Dell XPS 13 configuration but quoted it $50 apart; the third picked a MacBook Air.

My advice: let AI build a shortlist, then Repeat, Trace, and Verify: ask again, find what drove the choice, and check the exact model and offer with the manufacturer and retailer.

The same Dell, quoted $50 apart

On September 22, I compared three AI shopping answers. Using the labels attached to the responses, here is what each picked:

  • Gemini Pro: 13-inch M4 MacBook Air, 16GB/256GB, quoted at $999.99 with a general Best Buy search link.
  • Grok 4.6 Thinking: Dell XPS 13 DX13260, 16GB/512GB, quoted at Best Buy for $849.99.
  • GPT medium: Dell XPS 13, 16GB/512GB, exact model LDX13260-5960SKY-PUS, quoted at Best Buy for $899.99.

The Dell answers disagreed on price. One response also showed $806.99 FinanceMyCart cards, while the retailer snapshot contained inconsistent pricing and expired-deal cues. A precise dollar sign can make shaky information look checkout-ready. It is not.

Gemini also described 15–18 hours of real-world battery life. Apple lists up to 15 hours of wireless web browsing or 18 hours of video streaming under different test workloads. That does not promise the same result with Zoom and business apps. Nor did the evidence support its promise that the laptop would never stall or lag.

This is not a controlled test. I did not verify identical prompts, fresh chats, or the systems behind the displayed labels. The quoted prices are not verified current offers; checkout gets the final word.

What happened when researchers asked again?

How much weight should you give this? This is an early research paper, posted September 16, 2026; its record does not list acceptance after peer review. Treat it as a useful warning, not the final word on every AI shopper. Check the paper’s publication record.

The researchers asked 117 broad shopping questions three times in five versions of ChatGPT, Gemini, and Google Search AI Overviews, including both website and developer access: 1,755 attempts and 1,536 responses.

The repeated lists overlapped surprisingly little. On a scale from 0 for no shared products to 1 for identical lists, the averages were 0.178 for ChatGPT, 0.287 for Gemini, and 0.421 for AI Overviews. The researchers did not combine differently written names for the same product, so the overlap may be higher.

Keep the limits in view: the preprint used broad questions, mostly technology-related questions, access from the Netherlands, and one period of data collection. It did not show that ads caused a recommendation or identify the most accurate assistant. Read the study and its limits.

If a purchase matters, ask twice and inspect the evidence. Until you open them, citations are just footnotes wearing a tiny trust costume.

Repeat, trace, verify: three checks between an AI shopping recommendation and a purchase.
Repeat the request, trace the recommendation to its sources, then verify the exact offer. Editorial illustration.

Use the Repeat, Trace, Verify check

This check cannot prove an answer is correct, but it can expose where the answer is weak.

  1. Repeat: Put the same specific request into a fresh chat. Keep the budget, country, required features, and intended use unchanged. If the pick changes, compare the reasons before comparing the products.
  2. Trace: Note the products, decisive claims, and sources. Ask, “What exact condition would make your strongest alternative better?” Now you have something testable instead of a vague “best.”
  3. Verify: Check the exact configuration, price, stock, warranty, returns, and compatibility on manufacturer and retailer pages. The assistant does not get to grade its own homework.

A changed answer is not automatically bad. Several products may fit, prices move, and your request may omit a preference. What matters is whether the reason survives a check against the manufacturer, retailer, or original source.

Try it with a specific laptop request

Want to try this yourself? Copy this prompt, then change the budget and work details to match yours:

I need a laptop for a small consulting business in the United States. My budget is $1,000 before tax. I mainly use Microsoft 365, browser-based business software, Zoom, and occasional light photo editing. I carry it between home and client meetings, so portability and battery life matter. Which one laptop should I buy, and why? Give the exact model and configuration, its current price, and links supporting your recommendation.

Run that wording again in a fresh chat. Save the model, configuration, price, reasons, and links. If the pick changes, ask what fact changed the choice. Skip the robot prizefight.

Confirm the processor, memory, storage, display, weight, warranty, returns, and price. The right laptop family with the wrong configuration is still the wrong box at your door.

Let AI build the shortlist, not close the sale

I still find AI useful here. It can turn a messy wish list into tradeoffs and alternatives worth checking. Take the final decision outside the chat window.

  • Confirm the exact model and configuration with the manufacturer.
  • Confirm the variant, price, stock, shipping, and taxes at the retailer.
  • Read the actual return and warranty terms, especially for marketplace sellers.
  • Check compatibility claims in the relevant support document.
  • Save the shortlist and decisive sources.

Frequently asked questions

Does this mean AI shopping recommendations are random?

No. Answers changed substantially, but some products remained and a different choice is not necessarily wrong. The study supports checking consistency and evidence; it does not make every recommendation a coin toss.

Should I ask the same shopping question more than once?

For an expensive purchase, or one that must work with equipment you already own, yes. Repeat the same specific request in a fresh chat. If the pick changes, compare the reasons and verify decisive claims.

Do citations make an AI recommendation reliable?

They make the recommendation easier to inspect. They do not prove the source supports the exact model, configuration, price, or conclusion. Open the links and check the facts that matter.

Is my three-assistant comparison the same as the study?

No. I compared one answer from each of three assistants. I have not established that they all received identical prompts in fresh chats, or verified their underlying model versions. The paper repeated questions across interface, API, and search settings. My comparison simply shows why products, prices, and sources deserve a second look.

Sources and method

I summarized the paper’s design and main finding, then created the Repeat, Trace, Verify check. I wrote the laptop prompt before I collected the responses, but the evidence does not establish that each assistant received identical text. The paper is licensed CC BY-SA 4.0; I paraphrased it and linked the original. For corrections, email hello@theprotoolkit.com.

Leave a Comment