Interactive Learning
Test your knowledge across 3 games. You have 3 โค๏ธ to complete the challenge.
When It Makes Things Up
This is the last one, and it's the dangerous one.
The first three problems all announced themselves. Rambling looked too long. Complicated looked too hard. Generic looked too empty. You could see them.
This one you can't see. Because when the AI makes something up, it doesn't look wrong at all. It looks perfect. Confident, specific, well-written, exactly the shape a correct answer would take. It hands you a date, a statistic, a quote, a source, and every instinct you have says "yep, that's real."
And sometimes it isn't.
One of these has something completely made up in it.
Can you tell which?
The Apollo 11 mission landed on the moon on July 20, 1969. Neil Armstrong was the first to step onto the lunar surface, followed closely by Buzz Aldrin, while Michael Collins piloted the command module in lunar orbit.
The Apollo 11 mission landed on the moon on July 20, 1969. Neil Armstrong was the first to step onto the lunar surface, spending exactly 4 hours and 12 minutes outside the lander, while Michael Collins piloted the command module.
This is the one that can actually hurt you. A rambling answer wastes your time. A made-up fact in your essay, or in your head the night before an exam, does real damage. So this lesson matters more than the others. Let's get it right.
It's a plausibility engine, not a truth engine
To handle this, you have to understand one thing about what the AI actually does. It's not what most people think.
The AI is not looking facts up. It's not checking a database and reporting back. What it's actually doing, every single time, is generating the words most likely to come next. That's the whole machine. It produces text that sounds right.
Most of the time, the thing that sounds right is also true, because true things are common in what it learned. But here's the trap. When it doesn't actually know something, the machine doesn't stop and tell you. It can't. It just keeps doing the only thing it does: it generates the most plausible-sounding continuation. And a plausible-sounding fake date is just as easy for it to produce as the real one.
It's a plausibility engine, not a truth engine. It's built to sound right, not to be right. Usually those land in the same place. The danger is the times they don't.
Confidence is not a signal
Here's the consequence, and it's the single most important thing in this lesson.
Because the AI is just generating plausible text, it sounds exactly as confident when it's guessing as when it knows. There's no wobble in its voice when it's inventing. No "um." No hedge. It states a wild guess in the same crisp, certain tone it uses for the most basic fact.
So you cannot use confidence to judge truth. The thing your brain naturally reads as "they sound sure, they must be right" is worthless here. Sure means nothing. It was always going to sound sure.
Back in the rules, you learned to tell it "if you're not sure, say so." That's worth doing, and it helps. But it is not a guarantee, because the same engine that would invent a fact will cheerfully invent its own confidence about it. Asking "are you sure?" can get you a very convincing "yes, absolutely" about something completely false.
So if the voice can't be trusted, what can? Only one thing. Checking.
Know where it invents, and check those parts
You don't have to verify every word. That would be exhausting, and most of any answer is safe. You just have to know which parts to check.
Fabrication lives in the specific and the checkable. The more precise a claim is, the more you should look at it sideways:
- Names, dates, and numbers
- Statistics and percentages
- Direct quotes, and who said them
- Specific events, especially obscure ones
- Anything that sounds suspiciously exact
The primary cause of the economic shift was the sudden drop in manufacturing output, which began in 1982Date and resulted in a 14.3% decreaseStatistic in exports. This created a cascading effect across local economies. As noted by the labor board, "the recovery was entirely unprecedented."Direct Quote
The flip side is good news. The AI is much safer when it's explaining a concept, walking through reasoning, or working with material you gave it. It's most dangerous when you're asking it to pull a precise fact out of its own memory. The further it drifts from "explain this" toward "recall this exact detail," the harder you check.
The most convincing fakes are citations
One danger zone deserves its own warning, because students get burned by it constantly.
Ask the AI for a source and it will often hand you one that is flawless. A real-sounding author. A real-sounding book or journal. A year, a page number, the works. It looks like the most legitimate citation you've ever seen.
And it's completely invented.
The author, the title, the page, all generated because you asked for a citation and a citation is what plausible text looks like there.
Then go find it. If you can't locate it in about a minute of searching, it doesn't exist. Never, ever put a citation in your work that you got from the AI and didn't confirm with your own eyes.
According to Smith, J. & Miller, A. (2019). "Cognitive Load in Digital Environments." Journal of Applied Psychology, Vol 42(3), pp. 112-125.
Check the world, not the machine
So how do you actually verify? The rule is short. Don't ask the machine, ask the world.
If a fact has to be right, confirm it somewhere outside the AI. A quick search. The actual textbook. The real page. The thing about checking the world is that it doesn't care how confident the AI sounded. It just tells you whether the fact is real.
Still 100% false
The truth wins
And there's a way to dodge the whole problem when you can: give it the real material instead of asking it to remember. If you paste in the chapter and ask about that, it can't invent facts, because it's reading off your text, not its memory. Working from the real thing is the closest you get to fabrication-proof.
None of this is a reason to distrust the AI for everything. It's a reason to know your job. The AI is a brilliant thinking partner and a shaky reference book. Let it explain, draft, brainstorm, and work through your material all day long. Just be the one who checks the hard facts before they leave your desk. That single habit puts you miles ahead of everyone who takes a confident answer at its word.
๐ฏ Takeaway
The AI makes things up because it's built to sound right, not to be right, and it sounds exactly as sure when it's guessing as when it knows. So confidence tells you nothing. Verify the specific, checkable partsโnames, dates, numbers, quotes, and especially citationsโand verify them against the world, not by asking the AI again. Better yet, hand it the real material so it can't invent in the first place. Trust it to think with you. Don't trust it as your only source of truth.
And that closes out the troubleshooting.
You now know the four ways an answer goes bad, and the fix for each.
- ๐ When it rambles, it's too long, so put it on a budget.
- ๐งฉ When it's too complicated, it can't tell it lost you, so point at where.
- ๐คท When it's generic, it's feeding you the average, so put yourself into the question.
- ๐ And when it makes things up, it sounds sure either way, so check anything that has to be right.
Four problems, four fixes. The skill underneath all of them is the one this whole course has been quietly building: you don't accept a bad answer. You figure out what kind of bad it is, and you fix it.
Want more content like this?
Get instant access to everything we offer. Upgrade to a Monthly or Annual plan to unlock our entire library of advanced masterclasses, automation workflows, and custom AI blueprints.