On this page
The short version
Same question, different AI answers. Every morning for 32 days, we asked five AI systems the same 16 buying questions: 1,814 answers in all. Then we checked whether they agreed with each other on the same day, and with themselves the next day.
- Same morning, all systems: they agreed 1 time in 5. On 89 of 416 question-days, the AIs that answered named a business in common, or all named none. (The headline)
- When all five answered, they never named a business in common. Not once in 201 question-days. They only agreed when they all named nobody.
- Some pairs overlap far more than others, from 40% (Google AI Mode and Perplexity) to 63% (Google's two AI features). (Pairs)
- Each AI disagreed with itself about a third of the time. Its answer shared no business with the day before on 29% to 37% of days. (Day to day)
- Checking once tells you very little. Here's what to do instead. (What it means)
Last week we looked at the big picture: over a whole month, five AI systems picked the same favourite business for just 1 of 35 questions. People asked a fair follow-up. Surely on any given day they roughly agree? This is the answer, day by day.
The headline number
The short answer: on the same morning, asked the same question, the AI systems agreed about 1 time in 5.
| What happened that morning | Question-days | Share |
|---|---|---|
| Named at least one business in common | 37 | 9% |
| All named no business | 52 | 13% |
| Disagreed | 327 | 79% |
| Agreed in total | 89 | 21% |
The stricter test is worse. On the 201 question-days when all five systems answered (Perplexity was only added on 23 September, and Google sometimes shows no AI Overview), they never once named a business in common. All their agreement those days was agreement that nobody should be named.
I sell an AI visibility tracker, so weigh my interest in this. The business is our public demo, so every answer is there to check.
Want to see which questions your customers ask AI? The free question finder writes them from your website.
How we counted
The short answer: one real business, 16 questions, five AI systems, every morning, compared answer by answer.
The business is Kato Electrical, a Vancouver-area electrician. Every morning from 7 September to 8 October 2026, our tool, LLM Visibility Lens, asked his 16 buying questions in ChatGPT, Gemini, Google AI Mode, Google AI Overviews and Perplexity. It asked without logging in, from his city, and recorded every business each answer named. That's 1,814 answers, or about 113 per question.
What "agreed" means here. For each question and each morning, the systems that answered agreed if there was at least one business that every one of them named, or if none of them named any business. Spelling variants of a company were merged ("Intel Power Electric" and "Intel Power Electric Ltd."). Days when Google showed no AI Overview count as "no answer" for AI Overviews, not as a disagreement. If a system answered twice that day, both answers were combined.
What it doesn't cover. It's one business and one trade. It uses anonymous, logged-out answers; a logged-in ChatGPT user with saved memories can see different ones, as OpenAI's memory FAQ explains. The method behind the counts is on the how it works page.
"At least one business in common" is a generous test. Two answers that each list four electricians and share just one still count as agreeing. Even so, most mornings they didn't.
Which pairs agree on the same day
The short answer: no two AI systems overlapped on more than about two days in three.
Google's systems overlap most. AI Mode and AI Overviews shared a business (or both named none) on 63% of question-days, as did Gemini and AI Overviews. That fits what we found in Google AI Mode for local businesses: both lean on Google's own data.
ChatGPT and Perplexity are the surprise. Last week's month-level study found they rarely picked the same favourite business. On a given morning, though, they shared at least one name 62% of the time. They often list some of the same companies but put different ones first. Their long-run favourites differ, while their daily lists overlap.
Perplexity and Google's systems overlap least, at 40% to 42%. Perplexity reads very different sources, as we showed in Perplexity vs ChatGPT.
Does an AI agree with itself tomorrow?
The short answer: roughly two days in three. The rest of the time, its answer shared nothing with the day before.
| AI system | Shared a business with yesterday (or both none) | Business named one day, not the next (or back) |
|---|---|---|
| ChatGPT | 67% | 10% |
| Gemini | 64% | 12% |
| Google AI Mode | 65% | 17% |
| AI Overviews | 71% | 11% |
| Perplexity | 63% | 9% |
Two things stand out.
No AI is steady. On about a third of consecutive days, each system's answer to the same question had no business in common with its own answer the day before. Nothing changed on Kato's website in between. This isn't a fault in any one product. Language models produce slightly different text each run, and they may search different pages, as Thinking Machines Lab explained. SparkToro's study of AI recommendations found the same list of brands came back less than 1% of the time, as Search Engine Journal reported.
The system that likes you most can also be the jumpiest. Google AI Mode named Kato more often than any other system over the month. It also flipped him in or out most often, on 17% of consecutive days. Being named often and being named reliably aren't the same thing.
One question, one month
Here's what all of that looks like for a single question.
"Who is the best electrician in Vancouver BC?" Read it row by row:
- AI Mode named Kato on most days, but not all.
- ChatGPT mostly named other electricians for three weeks. Then, from late September, it switched between naming Kato and naming no business at all.
- Gemini and AI Overviews switched between Kato, other firms and nobody, sometimes from one day to the next.
- Perplexity, once added, named Kato most days.
Now picture checking this question once, on a random morning, in one AI. You could have come away with almost any story. That's why we track daily and report rates over weeks rather than single answers. The full argument is in why there's no rank #1 in ChatGPT.
You can open any of these mornings in the live demo and read every answer.
Why the answers move
Three reasons, and none of them is a bug.
Each answer is generated fresh. The model picks its words with some randomness. Even the order of businesses in a list can change, and with it which ones make the cut.
The sources change. When an AI searches the web before answering, it may read different pages on different days. In our ChatGPT citations data, ChatGPT drew on 92 different websites for one business's questions.
Google decides when to answer at all. For some local searches Google shows a map instead of an AI Overview, which we covered in why near me searches show no AI Overview. Google's own page on AI features and your website explains that these only appear when Google thinks they help.
What it means for how you check
One answer from one AI, on one day, is close to a coin toss. Here's what works instead.
- Use rates, not snapshots. "Named in 18 of 31 answers" means something. "ChatGPT mentioned us this morning" doesn't. Our guide to what one free check can and can't do shows how many answers you need.
- Check every AI separately. On the same morning they disagree four times in five. A blended score hides that.
- Compare like with like. Same questions, same days, same systems. Change the question set and the numbers stop being comparable.
- Don't panic over one bad day. Each AI flipped Kato in or out on 9% to 17% of consecutive days, often reversing itself the next morning. Judge a change over weeks.
- Look for questions where they all agree on nobody. Those 52 question-days were the only real consensus in the data, and they point to open doors. See We asked AI who to hire 80 times.
For the content side of getting named more often, start with how to rank in ChatGPT, how to rank in AI Overviews, spoken questions and whether your Google Business Profile helps.
How to track it
Ask the same questions in each AI, every day, logged out, from your customers' location, and record who gets named. A few weeks of that gives you rates you can trust. Doing it by hand across five systems takes about half an hour a day.
LLM Visibility Lens does it every morning. It asks ChatGPT, Gemini, Google AI Mode and AI Overviews daily, and Perplexity on the Pro and Agency plans. It keeps every answer and shows each system's rate with its range, so one odd morning doesn't move your score. The plans start free, and agencies can send the reports under their own name. See the agency plan.
There's also a plain comparison with Semrush's AI visibility tool, the changelog, the FAQ page and the security page. If you'd like me to look at how consistent the AI answers are for your business, get in touch.
Questions people ask
Do AI chatbots give the same answer every time?
No. In our data each AI system's answer to the same question shared no business with its own previous day's answer about a third of the time, and the tracked business flipped between named and not named on 9% to 17% of consecutive days, with nothing changed on its website.
Do ChatGPT, Gemini and Perplexity agree with each other?
Rarely on the same day. Across 416 question-days, the systems that answered agreed only 21% of the time, counting days when they all named no business. On the 201 days all five answered, they never named a business in common.
Why does ChatGPT give different answers to the same question?
Each answer is generated fresh with some randomness, and ChatGPT may search different sources each time. Logged-in users can also get answers shaped by saved memories and past chats. That's why a single answer is a poor guide to how often ChatGPT recommends you.
Which AI systems agree with each other most?
On the same day, Google's AI Mode and AI Overviews overlapped most, at 63% of question-days, as did Gemini and AI Overviews. ChatGPT and Perplexity overlapped on 62%. Perplexity and Google's systems overlapped least, at 40% to 42%.
How often should I check whether AI recommends my business?
Daily, for several weeks, on the same set of questions in each AI system. Because answers change from day to day, the useful number is how often you're named over that period, with a range around it, not what any one answer said.
Is this different from the month-level study?
Yes. Our earlier study compared each AI's favourite business over a whole month and found five systems agreed on a favourite for 1 of 35 questions. This one compares answers on the same morning, and each AI against itself from one day to the next.
Harjeet Singh Bedi Founder, LLM Visibility Lens · SEO consultant, Bedi SEO
Harjeet has worked in SEO since 2010, first in agencies and now running Bedi SEO for local businesses, online stores and agencies. He built LLM Visibility Lens after clients kept asking whether they showed up in ChatGPT and other AI tools and nobody could give them an honest, measured answer. Every figure in this article comes from answers his tool collected.