After some terrifying claims about baby food last month, we’re diving into the science behind studies.
Our VERIFY team has been breaking down how to look past a terrifying headline and judge the research itself.
Did you know chocolate can help you lose weight? I read it in a study online, so it has to be true, right?
Wrong.
“Chocolate with high cocoa content as a weight-loss accelerator.”
This is a real study, published in a real scientific journal. It appeared on talk shows, magazines and newspapers.
But look past the headline and look at the credibility of the study, like the lead researcher, Johannes Bohannon. Google him and you realize:
- His name is actually John
- He’s not a nutrition researcher. He’s a journalist who has a PhD in Biology
- The “Institute of Diet and Health” funded the study, but that’s just a website that the researchers created themselves
Three obvious red flags with two easy Google searches.
And that was actually the point. John and his team weren’t faking information or trying to deceive you. They were testing to see just how far they could push “bad research” because often. readers don’t dig deep enough before sharing something that catches their eye.
In 2015, Bohannon himself wrote an article on this experiment, pointing out the flawed methodology. They only tested 17 people. The test only lasted three weeks. The diets of the participants weren’t controlled. and it was done right after Christmas and New Years.
That means they got, like, 20 people who just feasted and filled up on holiday food, then measured their weight loss for just 3 weeks.
That leaves a lot of room for error.
So how does bad research get into the mainstream and go viral online?
John’s team published a great press release. On paper, it was a peer-reviewed study from a reputable sounding group. And it’s headline was something everyone wanted to talk about.
So, it took off, but no one questioned whether the information was reliable.
This “bad” study shows some serious flaws with how researchers, journalists and even you look at studies.
Before you share the latest, greatest scientific study. do a little digging. Check:
- Who led the research?
- Who paid for the study?
- Did the test enough people?
- And was it long enough?
And remember, results from studies are only conclusions drawn from testing one specific theory, and those results can always change.
