Headlines Sell: How to Spot Bullsh*t Science

Note: this article is in no way intended to start a debate about the C*VID vaccine, despite the fact that a meme about it started the thought process for this article.

In the past year, we’ve been inundated with science news coverage. Of course, the COVID-19 pandemic has dominated the spotlight but as most of us have spent more time at home, we’ve also been consuming a barrage of reports on global warming, mental health, nutrition, fitness—the list goes on.

We live in a world where misinformation is everywhere, whether that be in the form of fake news or a misleading scientific study. Some articles are blatantly incorrect, but most are more subtle. So how do you sort reliable from wrong?  

Here’s what got me going on this.

I recently ran across a meme on Facebook about statistics on the vaccine and miscarriage rates. It was a number that would have grabbed anyone’s attention, especially the attention of pregnant women or women considering pregnancy. And, the meme appeared in a Facebook group of women who already believe a lot of crappy science.

Of course, immediate outrage was my reaction. So I decided to look into what miscarriage rates are in a non-COVID world. And what I found in the actual article and elsewhere in the literature is that rates of first-trimester miscarriage are strikingly similar in vaccinated and unvaccinated women. No news here; this meme was sensationalism at its finest. Here’s some other misinformation that circulated.

Most news articles offer only a snapshot of a particularly exciting finding in the scientific community. Remember the Women’s Health Initiative and the sensational headlines about estrogen causing cancer? We now know that isn’t true, but 20 years later, many medical professionals and women still believe this myth. And the data that shows this has gotten NO press! 

So how to tell trash from truth? Here are some things readers can look for to evaluate the trustworthiness of the information. 

Sensational Headlines

Headlines exist to grab attention. They are what makes you decide to read an article or not. And the author’s job is to craft something catchy. But headlines that are too sensational can be a red flag. Think back to all of the headlines we saw about the extraordinary rates of COVID cure with hydroxychloroquine. Science ended up winning out over the noise about this. Follow-up studies found that the drug was ineffective for the treatment, and the FDA withdrew its emergency authorization in June 2020.

Who’s Footing the Bill?

Clinical trials are costly to conduct, and thus, most receive sponsorship from drug companies that have a financial interest in a positive outcome. This is not to say that all clinical trials are rigged in the drug company’s favor, but it does suggest that a Savvy Sister might want to dig into who was included and excluded in the trial and the methods used to analyze the data. Any study that is industry-sponsored should say so at the end of the study.

Evaluating Sources

Note the sources being used to back up the article. Well-researched science news articles should cite specific studies from reputable journals (although junk science CAN get published in journals with lofty reputations). In most cases, it should be a peer-reviewed article. Be aware that many predatory journals have absolutely no peer-review. 

Numbers Can Be Made to Say Just About Anything

Statistical methods used to interpret data to introduce a whole new level of confusion. Why?  Because numbers can often be manipulated to reflect a certain viewpoint by virtue of the method chosen to analyze them. Numbers can be hard to interpret or even left out altogether…leaving the reader to go look for the study. That’s what I did with the COVID meme. At least the meme was kind enough to include the actual citation! But as we all know, many people never read beyond the headline or the first couple of lines in an article.

The choice of statistical method depends on the way a study is designed. Be aware that any study that includes self-reported data by subjects is likely to be less reliable.

Uncovering Better Info

If an article raises any red flags, take a few steps to check out what you’re reading. If several news outlets are reporting the same findings, this may help…although if a piece is particularly sensational, they may all be repeating the same thing. Fortunately, many journals now track when a publication is cited in the media, making it easier to find additional coverage.

If there isn’t any other coverage of the topic and you still want to go deeper, a little more effort will be required. I pull the original article to read it (provided that I don’t have to pay for it). I read the introduction, the methods, and the summary/discussion….then the actual statistics if something still seems fishy. 

Becoming news literate and critical of the news is a skill that takes some practice. But always read your skeptical spectacles, just like a good scientist would.

Don’t want to do the digging yourself? Join the Savvy Sisterhood and I’ll do the work for you. As a pharmacist, I’m trained to interpret scientific studies…and I’ll happily share new data that comes out on midlife women’s health. Join the waitlist here: www.drannagarrett.com/waitlist or by clicking the button below!