In October 2026, Vectara’s public hallucination leaderboard showed the best large language model inventing unsupported claims in 1.8% of document summaries. The same table showed OpenAI’s o3-pro doing it in 23.3%. Both numbers are correct, and both come from the same test of more than 7,700 articles. That spread is the first thing to understand about AI hallucination statistics: the model matters, the task matters more, and a number without a date and a method attached is close to useless.
This page collects the figures that hold up, with the source and the date for each. At the end we lay out the four numbers we track for clients who care how AI systems describe their brand.
What Does a Hallucination Rate Measure?
A hallucination is output that sounds confident and is not supported by the facts available to the model. Researchers measure it in at least three different ways, and the results are not comparable across methods.
The first is grounded summarization. The model gets a document and a request to summarize it. Any claim in the summary that the document does not support counts as a hallucination. Vectara’s leaderboard uses this method, and it produces the lowest numbers, because the answer is sitting in front of the model.
The second is closed-book factual recall. The model answers a short question with no document and, often, no web access. OpenAI’s SimpleQA and PersonQA benchmarks work this way, as does Artificial Analysis’s AA-Omniscience test. These produce much higher numbers, because the model has to rely on what it absorbed in training.
The third is real-world task evaluation, where experts grade answers to the kinds of questions people ask: legal research, news, medicine. These studies are slower and smaller, but they are the closest to what your customers experience.
When someone tells you “AI is wrong 30% of the time,” ask which of the three they mean. Several analysts have pointed out that the round 30% figure circulating in media coverage does not trace back to any single formal measurement.
Summarization Numbers From the Vectara Leaderboard
Vectara has run its leaderboard since 2023, scoring models with its own hallucination evaluation model. The current version, last updated October 22, 2026, uses a dataset of more than 7,700 articles across news, science, medicine, law, sports, business, and education. The dataset is kept private so that model makers cannot train on it.
The best results at the October update were Ant Group’s Finix S1 32B at 1.8%, OpenAI’s GPT-5.4 nano at 3.1%, and Google’s Gemini 2.5 Flash Lite at 3.3%. Further down, OpenAI’s GPT-5.4 pro scored 8.3%, its o4-mini models scored 18.6%, and o3-pro scored 23.3%. Small models from Mistral sat above 20%.
Two patterns stand out. The first is that bigger and more capable does not mean more faithful. Several flagship reasoning models score worse at summarization than smaller, cheaper ones, because they add inferences and background knowledge that the source text never stated. The second is that no model scores zero. Even at 1.8%, a company summarizing 10,000 documents a month would produce about 180 summaries containing something the source did not say.
Open Questions: Where Models Guess
Take the document away and the numbers climb fast.
OpenAI’s April 2025 system card for o3 and o4-mini reported hallucination rates on PersonQA, a set of questions about public facts on real people. The older o1 model hallucinated on 16% of questions. The newer o3 hit 33%, and o4-mini reached 48%. OpenAI’s explanation was that the reasoning models make more claims overall, right and wrong. It was a rare case of a lab publishing evidence that its newer model was worse on a key safety metric.
The GPT-5 system card, released in August 2025, reported that gpt-5-thinking hallucinated on 40% of SimpleQA questions without web access, against 52% for GPT-4o. OpenAI also said GPT-5 with thinking was about 80% less likely than o3 to make a factual error on real-world traffic prompts.
On the AA-Omniscience benchmark, which tests 6,000 hard factual questions across domains, the 2026 Stanford AI Index reported hallucination rates from 22% to 94% across 26 models. The spread there reflects a design choice. Some models refuse when unsure, and some always answer. A model that refuses more looks better on hallucination rate and worse on helpfulness.
For a brand, the closed-book number is the one that matters most. When a buyer asks a chatbot “Is this company legitimate?” or “Who founded it?”, the model often answers from memory, not from a fresh search.
Why Do Models Hallucinate at All?
In September 2025, OpenAI researchers published a paper titled “Why Language Models Hallucinate” that put the cause in plain terms. Language models are trained to predict likely text, and the benchmarks used to grade them have mostly rewarded answering over abstaining. A model that guesses on a question it does not know will sometimes be right and score points. A model that says “I don’t know” scores nothing. Over many rounds of training and evaluation, that incentive pushes models toward confident guessing.
The paper argued that the fix is partly in how labs grade their models: penalize confident errors more than honest uncertainty, and give credit for abstaining. Several newer benchmarks now report a refusal or answer rate alongside the hallucination rate for exactly this reason. Vectara’s leaderboard lists an answer rate for every model, and some models with low hallucination rates also decline more often.
For anyone reading AI hallucination statistics, this explains a pattern that otherwise looks strange. A newer model can be more capable and more confident at the same time, which means more correct answers and more wrong ones. The hallucination rate alone does not tell you which effect is winning. You need the refusal rate next to it.
Legal AI: The Most Studied Failure
No field has documented hallucinations more carefully than law, because the consequences are public. In the 2023 case Mata v. Avianca, a federal judge in New York sanctioned two lawyers and their firm $5,000 after they filed a brief citing cases that ChatGPT had invented.
Stanford’s RegLab and Human-Centered AI institute then ran a preregistered study of the legal research tools sold as the fix. Published in 2024 and in the Journal of Empirical Legal Studies in 2025, it found that Lexis+ AI and Thomson Reuters’s tools each hallucinated more than 17% of the time, with Westlaw’s AI-Assisted Research above 33%. Earlier Stanford work found general-purpose chatbots hallucinating on 69% to 88% of specific legal queries.
The lesson for anyone outside law is about retrieval. These tools use retrieval-augmented generation, meaning they search a database before answering, and vendors had marketed that as eliminating hallucinations. It reduced them. It did not remove them, because the model can still misread, misattribute, or stitch together sources that say different things.
News and Current Events
In February 2025, the BBC published a study in which its journalists reviewed 362 answers from ChatGPT, Copilot, Gemini, and Perplexity to questions about BBC news stories. More than half had significant issues.
In October 2025, the European Broadcasting Union and the BBC released a much larger follow-up involving 22 public service media organizations in 18 countries and 14 languages. Journalists evaluated more than 3,000 responses. The headline findings: 45% of answers had at least one significant issue, 31% had serious sourcing problems, and 20% had major accuracy issues, including invented details and outdated facts. Gemini performed worst, with significant issues in 76% of its responses. The BBC’s own before-and-after comparison showed improvement, but error levels stayed high.
NewsGuard’s monthly audits point the same way. Its August 2025 report found leading chatbots repeating false news claims 35% of the time, up from 18% a year earlier, as the bots stopped declining to answer.
A February 2026 Stanford-led study tested six commercial chatbots daily for two weeks on fresh news. On adversarial questions built on false premises, GPT-5 answered correctly only 19% of the time, producing answers that went along with the false premise.
How to Lower the Rate in Your Own Use
The studies above also show what reduces errors in practice, and most of it is within reach of any team using these tools.
Give the model a source. The gap between summarization rates on Vectara’s leaderboard, often under 10%, and closed-book rates on PersonQA or SimpleQA, often above 30%, is the strongest evidence in this whole field. Pasting the document, the policy, or the dataset into the prompt changes the task from recall to reading, and reading is what models do best.
Ask for quotes and locations. Instructing a model to quote the passage that supports each claim, and to say which section it came from, makes unsupported claims easier to spot. It does not eliminate errors, but it moves them into the open.
Allow uncertainty. Telling the model that “I don’t know” is an acceptable answer lowers the pressure to guess. Pair it with a request to flag any claim it is less sure about.
Check the claims that carry risk. Names, numbers, dates, citations, and anything legal or medical deserve a human check every time. The legal sanctions cases all share one feature: nobody opened the cited cases before filing.
What These AI Hallucination Statistics Mean for Your Brand
Most coverage of AI hallucination statistics treats them as a problem for AI companies. For a business, they are a reputation problem you can partly control. A model that hallucinates on 33% of questions about people will also guess about your founder, your funding, and your products when it lacks clear sources.
We use a simple tracking model with clients that we call the Four-Number Brand Watch. Each month, run the same 20 buyer-style questions about your company through ChatGPT, Gemini, Perplexity, and Claude, and record four numbers. The first is the fact error rate: the share of answers containing a claim about you that is false. The second is the citation rate: how often the answer links to a source you control or a credible third party. The third is the competitor bleed rate: how often a competitor’s name or product appears in an answer about you. The fourth is the refusal rate: how often the model says it does not know.
A rising refusal rate with a falling error rate means the models are getting cautious about you, which usually signals thin coverage. A falling citation rate with a stable error rate means the models are answering from memory, so fresh press and structured facts have not reached them yet. Either way, the fix is the same: publish consistent facts on pages AI systems read, earn coverage in outlets they trust, and correct errors where they start.
If you ran those 20 questions about your company today, how many answers would you be comfortable with a buyer reading?