Not Falling for It · Mission 7 of 7

“9 out of 10”: numbers that mislead

Numbers look precise and convincing. You'll learn three questions that reveal when they mislead: Out of how many? Who measured it? Compared with what?

about 8 minutesStart with the → key or the space bar.

An ad between two rounds of a game

Jonah is playing his favourite game on his phone. Between two rounds an ad pops up: “9 out of 10 gamers recommend Echolo X headphones!” It sounds convincing. Nine out of ten – that's almost everyone.

Then Jonah notices the small print at the bottom: “Survey of 10 gamers who received the headphones free from the manufacturer.”

So, ten people. And all of them got the headphones for free. How convincing does that “9 out of 10” look now?

Three questions for every number

1. Out of how many? “9 out of 10” can mean nine people out of ten, but also 900 out of a thousand. That's a big difference. It's the same with percentages: “a 200% increase” can also mean that something happened once last year and three times this year.

2. Who measured it – and who paid for it? Did they ask randomly chosen people, or only happy customers? Was the survey done by the company that sells the product?

3. Compared with what? “The best result ever” – and what was it like at the same time last year? And elsewhere?

Charts that exaggerate

A chart looks like precise proof. But if the vertical axis doesn't start at zero, a small difference looks huge. Say one bar shows 92 and another 95. If the axis starts at 90, the second bar is two and a half times taller in the picture – even though the difference is only 3 points.

The UK's Office for National Statistics spells it out in its rules for charts: a bar chart's axis should always start at zero. When you see a chart, look at the numbers on the axis first.

Together isn't because

“Pupils who eat breakfast get better grades. So breakfast improves grades!” Maybe. But maybe the pupils who eat breakfast also sleep more or have more peace and quiet at home for studying. Two things going together doesn't mean that one causes the other.

A classic example: when it's hot, more ice cream is sold and more people get sunburnt. Ice cream doesn't cause sunburn – summer is behind both.

Find out more in the Manipulation atlas: Percentages without a base, Misleading chart and Correlation ≠ causation.

Game: Where are the numbers misleading? · Message 1 of 3

Tap the numbers and claims that are meant to fool you. One of the messages is honest.

Ad for a learning app

Brainomat: 87% of users get better grades!* Anyone who studies with it every day goes up a whole grade. Download it for free. *Survey of 15 users who had paid for a subscription.

Marked: 0 · After the discussion, click the places the class agrees on. Then show the analysis.

Game: Where are the numbers misleading? · Message 2 of 3

Tap the numbers and claims that are meant to fool you. One of the messages is honest.

Post on a local social media page, with a bar chart below it whose vertical axis doesn't start at zero

Bike thefts at our school are up 200%! Look at the chart – this year's bar is huge. Ever since we got the new bike racks, there's been more stealing. It's the racks' fault!

Marked: 0 · After the discussion, click the places the class agrees on. Then show the analysis.

Game: Where are the numbers misleading? · Message 3 of 3

Tap the numbers and claims that are meant to fool you. One of the messages is honest.

School magazine, October issue

How do we get to school? In September we asked 286 pupils from our older classes (out of 312). 58 of them cycle, so roughly one in five. Last September it was 41 out of the 295 pupils we asked. You'll find the full table on the noticeboard by the canteen.

Marked: 0 · After the discussion, click the places the class agrees on. Then show the analysis.

Game: Out of how many? Together or because? · 1 of 6

Read each claim. Is it fine, is it missing “out of how many”, or does it mix up things going together with one causing the other?

“This year we have 100% more top grades in maths!”