Hallucination
A hallucination is output from a language model that is fluent, confident and wrong: an invented citation, a function that does not exist in the library, a plausible date for an event that never happened. It is a normal mode of operation, not an error state.
Why it happens at all
A language model is trained to produce text that is likely, and likely text is usually true text — but not always, and the model has no separate mechanism for telling the two apart. There is no fact table to consult and no internal flag that fires when knowledge runs out. Asked for a source, the model produces the shape of a source: plausible authors, a real-sounding journal, a volume number in the right range.
This is why hallucinations cluster in predictable places:
- Specifics that are rare in training data: exact citations, statistics, dates, case law, phone numbers.
- Library and API surfaces, especially anything that changed after the training cutoff. Invented method names are extremely common.
- Questions with a false premise. Ask why a thing happened and the model will usually explain why, rather than say it did not happen.
- Anything just past the edge of what it knows, where the correct answer would be "I am not sure".
What to watch out for
Confidence carries no information about accuracy. The tone of a wrong answer is identical to the tone of a right one, because tone comes from style, not from verification. Readers calibrate on confidence and are therefore systematically misled.
Two more misconceptions worth naming. Setting temperature to zero does not remove hallucination — it removes variation, so you get the same wrong answer every time, which can be worse because it looks stable. And retrieval reduces but does not eliminate it: grounding the model in real documents removes a large class of invented facts, but the model can still misread a source, blend two of them, or answer confidently when retrieval returned nothing relevant.
What actually helps is structural: give the model the source material rather than relying on recall, ask it to quote the passage it used so a claim can be checked, verify anything mechanically verifiable — run the code, resolve the URL, check the number — and give it an explicit licence to answer "not in the provided documents". Then design the interface so a wrong answer is cheap. Assume some percentage will be wrong and make that percentage visible and correctable, rather than trying to reach zero.
Frequently asked questions
Can hallucinations be fixed completely?
No, not with current architectures. Grounding in retrieved documents, tool use and automated verification cut the rate substantially, and a model can be trained to hedge more often, but the underlying mechanism — producing likely text without a truth check — remains. Treat output as a draft to be verified where correctness matters.
Why does the model invent functions that do not exist when writing code?
Because it produces the method name that would be most plausible given the library’s naming conventions, which is not the same as the name that exists. Version changes after the training cutoff make it worse. Code is the easiest case to catch, though: run it, and the error appears immediately.
Build it yourself
NorthernGo turns a plain-text description into a working web app with a database, login and a live URL. Local AI generation runs on your own GPU, is unlimited, and is free on every plan.