A headline lands in your feed: "New study proves vitamin D cures depression." You click, skim, and forward it to three people before lunch. Two days later a friend points out that the study involved 40 adults, ran for eight weeks, and ended by saying more research was needed. The headline wasn't lying, exactly. It simply left out everything that mattered.
Most science coverage isn't written by liars. It's written by people working fast, paraphrasing a press release, which is itself paraphrasing a paper. Distortion creeps in at every step, and the good news is that the same tricks turn up again and again. Once you know them, you can usually work out in under a minute whether a claim deserves your attention.
Follow the trail back to the source
Start by looking for the study itself. A decent article links to it, names the journal, or at least names the research team. If there's no route back to an original paper, you're reading a story about a story, and the details have had more chances to warp.
When you find the paper, check two things. First, is it peer reviewed, or is it a preprint that hasn't yet been examined by other researchers? Preprints aren't worthless — they're how a lot of science gets moving — but they're drafts, and some change substantially or never get published at all. Second, look at the date. A story about a decade-old study can circulate perfectly happily as though it were published this morning.
Ask who was studied, and how many
Read the methods, or at least the first line of the abstract, and find the subjects. There's a large difference between these sentences:
- Cells in a dish responded differently to the compound.
- Mice fed the compound ran further on a treadmill.
- Twenty-four university students reported feeling better after four weeks.
- Several thousand adults were tracked for fifteen years.
"Scientists say" usually means one team, one study, one narrow group. A result in mice is a reason to run a human trial, not an instruction to change your diet. A result in 24 healthy young volunteers tells you little about how a treatment behaves in a 70-year-old taking three other medications.
Watch the verbs
The verbs are where the exaggeration lives, and they're easy to check once you're looking. "Causes" and "cures" describe experiments with control groups and clear outcomes. "Linked to" and "associated with" describe observations, where two things turned up at the same time and nobody has shown which one, if either, moved the other.
Then there are the mood words. "Breakthrough", "miracle", "the end of", "scientists baffled" — none of these appear in the paper. They're added later by someone with a word count to fill. A genuinely significant finding usually arrives in unusually dull language with a lot of caveats attached.
Check the size of the effect, not just its direction
"Doubles your risk" sounds alarming until you know what the original risk was. Going from one case in a hundred thousand people to two is a doubling. It's also, for any individual, close to nothing. That's the gap between relative risk and absolute risk, and headlines almost always quote the first because it produces the bigger number.
Statistical significance is another phrase that gets inflated. It means a result is unlikely to be down to chance in this particular sample. It says nothing about whether the effect is large enough to matter to you. If the study reports a wide range of possible values, the honest summary is that the researchers don't know yet.
Beware the missing comparison
Ask what the new thing was compared with. A diet that improves sleep in a group of volunteers is unimpressive if a control group sleeping the same number of hours shows the same improvement. Without a comparison, you're reading about change, not cause.
Also ask who paid. Industry-funded research isn't automatically wrong, and publicly funded research isn't automatically right, but knowing who benefits if you believe the result is useful context. So is knowing whether the finding has been repeated. One study is a signal. A pattern across several independent studies is evidence.
The five-minute headline check
- Can I find the original study, and has it been peer reviewed?
- Were the subjects people, animals or cells — and how many?
- Does the headline claim cause and effect where the study shows only a link?
- Is the effect given as a relative risk, with the absolute numbers missing?
- Who funded the work, and who gains if I believe it?
- What does the paper's own conclusion say, and does it match the headline?
Six questions, two minutes at most. Most hyped stories fall down at question two or three.
Keeping your scepticism proportionate
Being careful isn't the same as being dismissive. Science moves in small increments, and most individual papers — even good ones — are tentative. Scepticism means asking what the evidence actually supports, not assuming everything is wrong. The reasonable position on most new findings is "interesting, let's see if it holds up", which is a perfectly good thing to say out loud when someone sends you a link.
It also helps to notice what kind of claim you're being sold. A new fossil or a physics result sits fairly self-contained. Advice about your health, your money or your children is different, because acting on it has consequences. For anything medical in particular, treat a headline as a prompt to ask a professional — a GP or pharmacist — rather than a reason to change what you're already doing.
Finally, watch your own instincts. Stories that confirm what you already believe get shared far more readily than stories that complicate it, and the flattering headline is the one least likely to be examined. If a claim makes you feel smug, surprised or alarmed, that's exactly the moment to run the six questions.
Photo: moritz320 / Pixabay
