In 2015, a study conducted by the Institute of Diet and Health found that eating chocolate accelerates weight loss. It was great news for chocolate lovers — until it turned out to be an elaborate hoax.
The news went viral, making headlines in more than 20 countries before the mastermind behind it all, science journalist John Bohannon, revealed that the study was intentionally flawed and the “institute” behind it was made up. He had made his point: bad science can easily make headlines.
As people around the world start considering their New Year’s resolutions, Seth Noar, a health communication expert and professor at the Hussman School of Journalism and Media, has something to say about this kind of health misinformation.
Here he shares a few tips for determining what’s real and what’s not this resolution season.
Look for the original source
Scroll through your Facebook newsfeed on any given day, and you’re likely to find news stories, advertisements and other messages about the latest trends in diet and exercise.
But a lot of what you find on social media could be bad advice, Noar said. The source is critical.
“Unfortunately, people should approach all information these days with some skepticism,” he said. “One of the first things to look for is, what’s the source? Where is it coming from? Is it a legitimate news organization that you know? If so, then you can have a little more faith in it. You have to remember that Facebook itself is not a source.”
See what else is out there
If 100 prior studies all show that eating chocolate makes you gain weight and one new study suggests that it helps you lose weight, that’s a red flag, Noar said.
“The news media sometimes treats a new study like it’s the only one out there on the topic, but that’s rarely the case. There’s also a little bit of a common-sense test you can give yourself in situations like that — when something seems too good to be true,” he said. “It helps to ask, ‘How much does this mesh with prior knowledge on this subject, and common sense?’”
Look past catchy headlines
Even experienced reporters can get it wrong sometimes, Noar said, especially when it comes to translating complex science into a seven-word headline.
“The news, by definition in many ways, is about what’s new and what’s going to be catchy,” he said. “The media will often take scientific evidence that is correlational, and they’ll frame it in a way that suggests causation.”
For example, Noar said, a headline that reads “Having a Dog Makes You Live Longer” might not really mean what it says. People who own dogs might have longer lifespans because they are more active or spend more time outside, not simply because of their canine friends. It could be reverse causation — more active people are more likely to adopt a canine friend.
“All of these health studies about ‘lifestyle,’ they’re almost always correlational studies in which we can’t rule out other causal factors,” he said. “I take any study about nutrition or lifestyle science with a grain of salt.”
Develop science literacy
When looking at a study, there are a few questions you should ask yourself, Noar said.
- Was the study done in people or only in mice?
- Was it a randomized trial or simply a correlational study (also called an ‘observational study?’)
- How big was the sample size? Very small studies are more likely to be error-prone.
- Who funded the research? Watch out for agendas among groups who have a profit motive based on research findings.
When it comes to nutrition, Noar also encourages people to brush up on commonly used advertising terms. For example, he said, the word “natural” has no regulatory meaning, and a term like “low fat” might also mean that the food product is packed with sugar or sodium.
In the end, it’s best to do your homework before taking any medical advice from the internet. And when in doubt, Noar said the simplest thing to do is to talk to your doctor or a trained nutritionist to sort out fact from fiction.