Why AI models make things up, and what actually helps

What a hallucination is

A hallucination is a fluent, plausible answer that is false. The model is not lying, because it has no intent, and it is not broken, because the same model answers most questions correctly. It produced the most plausible continuation of your question, and for a fact it never reliably learned, a plausible answer and a true one can be different things.

Why they happen: models are taught to guess

In September 2025, researchers from OpenAI and Georgia Tech published a clear account of where hallucinations come from. Their argument is that language models hallucinate "because the training and evaluation procedures reward guessing over acknowledging uncertainty".¹ Most benchmarks grade an answer as right or wrong. A model that says it is not sure scores zero, exactly like a model that is wrong, so a model that always guesses scores higher on average. Developers optimise for those scores, and the habit sticks.

The same paper traces the problem back to pre-training. Some facts, like a single person's birthday, appear too rarely in training data for any pattern to predict them. The model has nothing to generalise from, and when it is asked anyway, it fills the gap with something plausible.¹ The authors' proposed fix is to change how the main benchmarks are scored, so that a confident wrong answer costs more than an honest "I am not sure".

How hallucination is measured

One widely used test is OpenAI's SimpleQA: 4,326 short, fact-seeking questions written so that "there exists only a single, indisputable answer".² Because each question has exactly one correct answer, grading is unambiguous, and the benchmark can separate three outcomes: correct, incorrect, and declined to answer. That third column matters. A model that declines more often and is wrong less often is usually the more useful one, even if its raw accuracy looks no better.

What actually reduces hallucinations

• Ground the answer in sources. Retrieval-augmented generation fetches relevant documents and has the model answer from them. The original RAG paper found that, for language generation tasks, these models "generate more specific, diverse and factual language" than a model working from memory alone.³ Web search is the everyday version of this.
• Show the sources. An answer with links can be checked in seconds; one without them has to be trusted. See why AI answers should cite their sources.
• Use a stronger model for hard questions. Larger, newer models know more facts and are better calibrated about the ones they do not know. Paying for them on every message is wasteful, but on a hard or high-stakes question they are worth it.
• Let the model think. Reasoning before answering lets a model check its own work on multi-step problems. See how hard an AI should think.

What Nivana does about it

Nivana cannot make any model incapable of hallucinating, and no one honestly can. What a router can do is put each question in front of the conditions that make a correct answer most likely:

• Current facts go through search. Before routing, Nivana checks whether a good answer depends on information from the web, such as prices, news, release dates or what a particular site says. If it does, the answering model searches first, answers from what it found, says how recent the information is when that matters, and links its sources inline with a short list at the end.
• Hard questions go to strong models. As difficulty rises, the router weighs measured quality much more heavily than price, so expert-level and high-stakes questions go to the strongest model for that task.
• Effort is set per question. A proof or a tricky analysis gets a deeper think; a greeting does not.
• Widgets don't make up data. When an answer includes a chart or a contact card, the model is told to use only real or clearly stated numbers and details, and never to invent them.
• You can see who answered. Every reply shows which model wrote it, so you can tell which model to blame for a mistake.

What you can do

• Ask for sources on anything factual you plan to rely on, and open at least one.
• Say "if you are not sure, say so". It does not fix the incentive, but it gives the model permission to decline.
• Be most careful with rare specifics: exact quotes, citations, dates, small-company details and people who are not famous. These are the gaps the research describes.