Why AI Makes Things Up — and How to Catch It Every Time

Ask an AI for the population of Manila and you’ll get a reasonable number. Ask it for the exact page of a quote in a specific book, and you may get a beautifully formatted, confidently stated, completely invented answer. Both replies look identical. Learning to tell them apart — and to check when it matters — is the single most important safety skill for everyday AI use.

What a hallucination is

A hallucination is when an AI states something false as if it were fact: a made-up statistic, a court case that doesn’t exist, a book the author never wrote, a URL that goes nowhere, a plausible-sounding medical claim.

The unsettling part isn’t that AI makes mistakes — everything makes mistakes. It’s that hallucinations arrive fluent, specific, and confident, indistinguishable in tone from correct answers. There’s no stammer, no “I think…”. The formatting is perfect. Only the facts are wrong.

Why it happens (30-second version)

If you read What Is AI, Actually?, you already know the cause: an LLM doesn’t look facts up in a database — it predicts plausible next words based on patterns it learned in training.

Most of the time, the most plausible continuation is the truth (“the capital of France is Paris”). But when the model is asked about something obscure, recent, or precise, the most statistically plausible text and the true text part ways — and the model follows plausibility off the cliff. It fills the gap with something shaped exactly like a right answer.

A useful mental model: the AI is an eloquent friend who has read everything but is incapable of saying “I don’t remember” unless you make that an acceptable answer.

The five highest-risk situations

Hallucination risk isn’t evenly spread. Be on alert whenever you ask for:

1. Citations and sources — paper titles, authors, court cases, book quotes. The classic failure. Lawyers have been sanctioned for filing AI-invented case law.
2. Precise numbers — statistics, prices, dates, dosages. The shape will be right; the digits may not be.
3. Niche and local facts — the model has seen little training data about your barangay, your industry’s small suppliers, or a 2019 municipal ordinance. Thin data → confident guessing.
4. Anything after the knowledge cutoff — news, prices, versions, laws. Unless the tool actually searched the web (look for cited links!), it literally cannot know.
5. Links and titles — URLs are predicted character-by-character like all other text. Many will 404.

Conversely, risk is low when the AI is transforming material you gave it — summarizing your document, rewriting your email, explaining a concept it has seen explained ten thousand times. That’s why “paste your source material” is both a quality tip and a safety tip.

How to reduce hallucinations before they happen

Four habits, in order of impact:

1. Give it the source material

Don’t ask what a document says — paste the document. Grounded questions get grounded answers.

Using ONLY the report pasted below, answer: what were the three main
findings? If the report doesn't address something, say so — do not
fill gaps from general knowledge.

[paste report]

2. Make “I don’t know” an acceptable answer

This one line measurably changes behavior:

If you are not confident about any fact, say "I'm not sure" and tell me
what you'd need to verify it. Do not guess.

3. Use web search for anything current or checkable

Most chatbots can search the web now (sometimes behind a button or a “search” toggle). A searched answer comes with links you can click — that’s the difference between testimony and evidence. For news, prices, laws, and product facts: always search mode.

4. Ask for confidence labels

Answer, then label each factual claim as [SOLID] (widely documented),
[LIKELY], or [SHAKY] (niche/recent — verify me).

The labels aren’t perfect, but they reliably flag the claims the model itself generated on thin ice — which is exactly where you should aim your checking.

The 60-second verification workflow

When an answer matters — money, health, law, reputation, a decision — run this before acting on it:

1. Extract the load-bearing claims. Usually 1–3 facts the whole answer stands on. Ignore the connective prose.
2. Check each against one independent source. A real search engine, the official website, the primary document. Not the same AI — and not another AI without search, which shares the same blind spots.
3. Click every citation the AI gave you. Does the page exist? Does it actually say what the AI claims? (The second failure is sneakier than the first.)
4. For high-stakes topics, ask a qualified human. AI is a phenomenal preparation tool for the doctor/lawyer/accountant conversation — not a replacement for it.

A prompt that helps with step 1:

List every factual claim in your last answer that, if wrong, would change
the conclusion. For each: where would a careful person verify it?

Calibrate, don’t panic

Two mistakes will cost you here. The first is blind trust — acting on an invented statistic because it arrived in confident prose. The second is blanket distrust — writing off a tool that’s genuinely transformative because it has a known, manageable failure mode.

The skill in between is calibration: match your level of checking to the stakes and the risk profile.

Situation	Checking needed
Brainstorm, drafts, rewrites of your own text	Little to none — you’re the source
Explanations of well-known concepts	Skim for sense; spot-check anything surprising
Facts, numbers, citations, current events	Verify before using — 60-second workflow
Health, legal, financial decisions	Verify + qualified human. Non-negotiable

Try it yourself

The fastest way to build the instinct is to watch a hallucination happen in a safe setting. Ask your AI (without web search on):

Give me the exact title, authors, journal, and year of three published
studies about [an extremely specific topic you know well].

Then actually search for them. If one or more turn out not to exist — congratulations, you’ve now seen the failure mode with your own eyes, and you’ll never fully trust an unverified citation again. That instinct is the whole lesson.

Where to go next

• Learn the prompting habits that prevent bad answers in the first place: Prompting Basics
• Decide which tool (with which search features) fits you: Choosing Your AI Tool
• The other half of AI safety — what you share with it: AI Privacy & Safety Basics