AI tools are getting better at answering questions, but they can still give a completely wrong answer with surprising confidence.

A response may look convincing, include a detailed explanation, and even provide a citation. Yet the underlying fact can still be wrong.

So this experiment set out to test the leading AI assistants directly.

ChatGPT, Gemini, Claude, Perplexity, Grok, and Microsoft Copilot were given the same 25 factual questions, with every response checked against a predefined answer key.

The aim was simple:

Which AI tool gets the most facts right, and which one is most likely to confidently hallucinate?

This isn’t a test of writing quality or creativity. The focus is on one thing: factual reliability.

Table of Contents

What is an AI hallucination?

An AI hallucination is an answer provided by the artificial intelligence that is incorrect or deceptive and is presented as if it is factual. The term is informally drawn from the psychological phenomenon of hallucination in the meaning of having illusory percepts.

AI hallucination may feature random lies plausibly incorporated into content, such as fake references, made by large-language-model-based chatbots.

It is problematic to detect and eliminate errors and hallucinations if LLMs are going to be used in high-risk applications in fields like silicon chip designs, the supply chain, and medical diagnosis.

The AI Tools Tested

Six popular AI assistants were included:

Because these products can use different models and features, the model or version shown by each tool during testing was recorded along with the test date.

This matters because AI products are constantly changing. A result from August 2026 may not represent the same model or behavior several months later.

How the Test Was Run

To keep the comparison as fair as possible, every tool received the same questions.

The AI hallucination test used:

25 questions
The same wording for every tool
One attempt per question
No previous conversation
No uploaded files
No custom instructions
Default settings
No follow-up questions

For the main AI hallucination test, web browsing was kept disabled wherever the product allowed that control.

Search-enabled tools were also tested separately with browsing enabled, so internal-knowledge performance could be compared with web-assisted performance without mixing the two.

Every response was saved and checked against the same answer key.

Each answer was classified as:

Correct
Incorrect
Not attempted

An answer was treated as incorrect when it provided a false fact, contradicted the answer key, invented information, or accepted a false premise without correcting it.

A response such as “I don’t know” was recorded as not attempted, rather than hallucinated.

What Was in the 25 Questions?

The test wasn’t meant to be nothing more than a collection of easy trivia questions.

The 25 questions were divided into five categories:

Category	Questions	What Was Tested
Geography, Nature & General Knowledge	1–5	Basic factual knowledge, geography, biology and chemistry
History & Trick Questions	6–15	Historical facts, ambiguous questions and false premises
Physics & Space	16–20	Scientific concepts and physics reasoning
AI & Current Knowledge	21–25	AI knowledge, uncertainty, current information, and future predictions
Total	25	Overall factual accuracy and hallucination resistance

The questions were finalized before testing began.

Questions where the answer could reasonably change from one day to another were also avoided. For example, current stock prices, today’s weather, or the latest political developments were not included in the primary AI hallucination test.

A Few Questions From the Test

Here are a few examples of the type of questions used:

What is the chemical symbol for gold?

Answer: Au

What is the capital of Mongolia?

Answer: Ulaanbaatar

In which year did World War II end?

Answer: 1945

What does CPU stand for?

Answer: Central Processing Unit

Questions designed to catch models that automatically accept a false premise were also included.

For example:

Which year did the United States join World War II after Germany declared war on it?

The wording contains a problem. A reliable model should recognize and correct the premise rather than simply provide a date.

This part of the test was particularly interesting because factual accuracy isn’t only about knowing information. It is also about recognizing when a question itself is misleading.

The Results

After running all 25 questions through each AI tool, the number of correct, incorrect, and unanswered responses was calculated.

Tool	Correct	Incorrect	Not Attempted	Accuracy	Hallucination Rate
ChatGPT	24	1	0	96%	4%
Gemini	23	2	0	92%	8%
Claude	25	0	0	100%	0%
Perplexity	24	0	1	96%	0%
Grok	24	0	1	96%	0%
Copilot	21	3	1	84%	12%

How Accuracy Was Calculated

Accuracy was calculated as:

Correct answers ÷ 100 × 100

So, if a tool answered 82 questions correctly, its test accuracy would be 82%.

Hallucination rate was calculated as:

Incorrect answers ÷ 100 × 100

And attempted-answer accuracy:

Correct answers ÷ (Correct + Incorrect) × 100

The last metric helps distinguish between a model that frequently guesses and one that is more cautious about answering uncertain questions.

Which AI Got the Most Answers Right?

Claude was the top-scoring tool for this AI hallucination test (which had 25 questions) with 100% accuracy. It answered 25 questions correctly with 0 incorrect responses and 0 blank answers. More specifically, Claude’s results are so interesting because Claude correctly answered the false premise and unknowable questions.

Claude did not invent false answers for any of these questions and reported that they could not be answered and no value could be derived.

This is also where the raw answers matter. A simple percentage shows who scored highest, but looking at the actual responses shows how the tools failed. Some models gave incorrect factual claims, while others appropriately qualified ambiguous questions or refused to provide information that could not be verified.

The Most Surprising AI Hallucination Found

Some of the most interesting results weren’t necessarily the questions that every tool got wrong. They were the answers that sounded completely convincing but contained a false claim.

Example 1: Which is the biggest city in the world?

Tool: Microsoft Copilot
What it answered: “Guangzhou, China, is the largest urban area with ~73.6 million people.”
Correct answer: The answer depends on the definition and measurement used. Recent UN-based urban-agglomeration data place Jakarta at the top.
What went wrong: Copilot gave a specific population figure and presented Guangzhou as the definitive answer, without explaining the methodology or acknowledging the competing rankings.

Example 2: What would happen to a person who fell into a black hole?

Tool: Google Gemini
What it answered: A person would be “stretched into a long, thin strand of atoms before reaching the singularity.”
Correct answer: The effects depend on the black hole’s mass. A person could cross the event horizon of a sufficiently massive black hole without immediately experiencing extreme tidal forces.
What went wrong: Gemini presented spaghettification too generally, failing to distinguish between stellar-mass and supermassive black holes.

Example 3: Which AI company will have the largest market share in 2030?

Tool: Microsoft Copilot
What it answered: “OpenAI could dominate with ~25% of a $700B market.”
Correct answer: No one can know the exact market leader or market share in 2030. Any specific figure is a forecast, not an established fact.
What went wrong: Copilot turned a speculative future prediction into a highly specific claim, making the answer sound more certain than the available evidence supports.

These examples are more useful than simply saying that an AI “hallucinated.” They show exactly what hallucination looks like in practice: a wrong or unsupported claim presented with enough confidence and detail to make it easy to trust.

Which AI Was Best at Catching False Premises?

The 5 trick and misconception questions produced a different kind of result. Instead of simply checking whether the model knew the correct fact, the review looked at whether it noticed that the question contained a misleading assumption.

These responses were scored separately:

2 points: Correctly identifies and corrects the false premise
1 point: Gives a partially correct response but does not clearly address the premise
0 points: Accepts the false premise and gives an incorrect answer

Tool	Stress-Test Score	Out of 10
ChatGPT	10	10
Gemini	10	10
Claude	10	10
Perplexity	10	10
Grok	10	10
Copilot	10	10

All six tools performed strongly on the false-premise questions. They generally recognized that questions such as “Who was the first person to walk on the Sun?” and “Which Egyptian pharaoh invented the telescope?” contained impossible assumptions rather than inventing answers.

What About AI Citations?

Citations were also reviewed when an AI provided them. A citation can make an answer appear more trustworthy, but the presence of a citation doesn’t automatically mean the claim is supported.

For selected responses, the review checked whether the cited source existed, contained the relevant information, supported the exact claim, came from an authoritative source, and was accurately represented by the AI.

Perplexity stood out for providing citations alongside several factual answers, including sources such as NOAA, NobelPrize.org, Wikipedia, and Ethnologue. However, citations were not treated as automatically correct simply because they were present. The source still needed to support the specific claim being made.

This gave another way to compare research-oriented AI tools. An AI can get the answer right while still providing a weak or irrelevant citation. For anyone using AI for research, journalism, academic work, or content creation, factual accuracy and source quality are two separate things that both need to be checked.

Internal Knowledge vs. Web Search

One result not left hidden behind a single ranking was the difference between answering from model knowledge and searching the web.

A model without browsing has to rely largely on its existing knowledge.

A web-enabled AI has another advantage: it can retrieve information that may be newer than its training data.

But browsing introduces new opportunities for mistakes.

The AI might:

Choose a low-quality source
Misread a webpage
Use an outdated source
Misinterpret the information
Cite a page that doesn’t actually support the claim

For that reason, browsing was treated as a separate test rather than mixed into the main score.

How These Results Compare With Public Benchmarks

This 25 question experiment is an original consumer test. It should not be confused with established academic or industry benchmarks.

Benchmarks such as SimpleQA and TruthfulQA provide useful external context.

SimpleQA focuses on short factual questions with verifiable answers, while TruthfulQA was designed to test whether models reproduce common misconceptions.

These benchmarks are useful because they show that factual reliability differs between models.

However, their scores should not be treated as direct predictions of how often a particular consumer AI product will hallucinate in everyday use.

The product a user opens can expose a different model, system instructions, search features, or other tools.

That’s why the focus here was primarily on what happened when six actual AI products were given the same 25 questions under the same conditions.

What This Experiment Revealed About AI and Hallucinations

This test showcased how differently the tools handled uncertainty. Some responses were accurate and straightforward.

Others were wrong but cautious.

And some were completely incorrect while sounding highly confident. That last category is the one users need to watch most carefully. A confident tone is not evidence that an answer is correct. A detailed explanation is not evidence either. And even a citation needs to be checked.

The results also showed why a single “accuracy” number doesn’t tell the whole story. A tool that answers everything may appear more useful while also making more unsupported guesses. Another tool may decline more often but be considerably more reliable when it does answer.

So, Which AI Tool Is the Most Accurate?

Based on this AI hallucination test, Claude AI came out on top with an accuracy score of 100%.

But that does not mean it is automatically the best AI for every task.

This experiment tested factual questions. It did not measure coding, writing, creativity, reasoning, image generation, research depth, or overall usefulness.

The winner of this test is therefore best described as:

The most factually reliable tool in this specific 25-question experiment.

That’s a much more useful conclusion than claiming one AI is universally better than every other AI.

What This Means for AI Users

These results suggest a simple rule:

Don’t confuse confidence with correctness.

For everyday low-stakes questions, an occasional mistake may not matter much.

For important information, however, verification is still necessary.

A practical workflow is:

Ask AI → check the source → verify the claim

This is especially important for medical, financial, legal, academic, technical, and other high-stakes information.

Final Verdict

It is hard to find an AI hallucination test that shows what actually happened when real AI products were given the same questions under the same conditions.

That’s what this experiment set out to measure.

Six leading AI assistants were tested across 25 factual questions, with answers checked against a fixed answer key, misleading questions handled and scored separately, and citations reviewed for whether they actually supported the claims made.

The results show that AI factuality is not simply about which model knows the most.

It is also about whether the tool knows when it might be wrong.