Not Falling for It · Mission 4 of 7
AI can get it wrong – and make things up
Why a chatbot sounds sure even when it doesn't know, how to spot a made-up number or source – and how to use AI for school wisely.
about 8 minutesStart with the → key or the space bar.
💬A report due on Monday
Linh (13) has a report on Charles Bridge in Prague due on Monday. She asks a chatbot, and within seconds she has a finished text: dates, measurements, even the title of a book it supposedly comes from. It sounds clever and confident.
On Monday her teacher asks: “Where did you find that book?” Linh looks for it in the library and online. The book doesn't exist. And according to the report, the bridge is 2 kilometres long – in reality it's about 516 metres.
The chatbot didn't lie on purpose. It just made it up. And it didn't say so.
Why AI gets things wrong
A chatbot usually doesn't look up the answer in an encyclopedia. It puts together text from words that probably belong together – based on the huge amount of text it learned from. Most of the time that works well. But when it doesn't know something, it may not realise. Instead of “I don't know”, it can write something that only sounds right: a name, a number, a quote, a book title or a link.
This is called a hallucination. And the chatbot writes it just as confidently as the truth.
The companies that make chatbots admit this themselves. Their terms say that answers may not be accurate and need to be checked. One of them warns that an answer can be wrong even if it looks accurate because it's full of details. A chatbot also may not know the latest information – such as today's opening hours or yesterday's match result.
Game: Where is the AI making things up? · Message 1 of 3
Read the chatbot's answer. Tap the parts that are made up or can't be checked. One answer is fine, though.
Question: Write me a few sentences about Charles Bridge for my report.
Charles Bridge in Prague was founded by Emperor Charles IV in 1357. The bridge is exactly 2 kilometres long and stands on 100 arches. All historians agree on this. Source: the book “The Stone Secrets of Prague” (2016).
Marked: 0 · After the discussion, click the places the class agrees on. Then show the analysis.
Game: Where is the AI making things up? · Message 2 of 3
Read the chatbot's answer. Tap the parts that are made up or can't be checked. One answer is fine, though.
Question: When is our town library open on Saturday?
Your town library is open on Saturday from 9 am to 5 pm. I've just checked this on its official website. Happy reading! 📚
Marked: 0 · After the discussion, click the places the class agrees on. Then show the analysis.
Game: Where is the AI making things up? · Message 3 of 3
Read the chatbot's answer. Tap the parts that are made up or can't be checked. One answer is fine, though.
Question: What is photosynthesis? I need it for biology.
Photosynthesis is the process in which plants use sunlight to make sugar from water and carbon dioxide. While doing this, they release oxygen. Check the details, like the individual stages, against your textbook – I might get those wrong. Would you like me to explain it with an example?
Marked: 0 · After the discussion, click the places the class agrees on. Then show the analysis.
How to use AI wisely
- What it's good for: explaining a topic in other words, coming up with practice questions, helping with an outline or checking your own writing.
- What to always check: names, dates, numbers, quotes and sources. That's where mistakes matter most and are hardest to spot.
- You have to find the source yourself. If a chatbot mentions a book or an article, try to track it down in the library or online. If you can't find it, it doesn't belong in your work.
- Don't type personal things into the chat: your address, phone number, passwords or photos of friends. One big provider warns that some conversations are read by people who work on improving the chatbot, and advises not to type confidential information into the chat.
- Ask your teacher whether and how you can use AI for an assignment. And if you use it, say so.
Find out more about the “Fake proof” trick in the Manipulation atlas.
Quiz: Smart with AI · Question 1 of 3
A chatbot gave you a source for your report: a book with a title and a year. What do you do?
Click the answer the class decides on. Then show the right one.
Quiz: Smart with AI · Question 2 of 3
Why does a chatbot sound sure even when it doesn't know something?
Click the answer the class decides on. Then show the right one.
Quiz: Smart with AI · Question 3 of 3
What is AI most useful for at school?
Click the answer the class decides on. Then show the right one.
What to remember
- ◆AI sounds sure even when it's making things up.
- ◆Always check names, numbers, dates, quotes and sources somewhere else.
- ◆A source you can't track down doesn't belong in your report.
- ◆Don't type personal details or passwords into the chat.
💬Let's talk
- 1Do you use a chatbot for school? What does it help you with – and where has it got things wrong?
- 2Shall we ask it together about something we know for sure – like our town or a castle we visited on a trip? Does it get it right?
- 3What do you never type into a chatbot, and why?
Tip for the teacher
Chatbots' terms differ: some allow them from 13 with parental permission, others only from 18. Agree with your child (and the school) on what they may use one for. And try out together how easily a chatbot makes something up about a place you know well.
✓ Mission complete!
At home, children can play it again – with their parents, for example. Free and without an account.
thinkcritical.eu/