Statistical Sins: Overstating Implications

Here we are - the first ever Statistical Sin post! In this series, I'll draw attention to sins against statistics, and more generally research and science. The perpetrators of these sins could be anyone: politicians, journalists, even the researchers themselves.

Today's Statistical Sinner is NBCNews - I know, not a high bar to begin with, as these types of news sources tend to gravitate toward click-bait when it comes to science reporting. You're much better off going to a science-specific news source, or a large newspaper that tends to be more thoughtful in its discussion of research (the New York Times and Washington Post stick out as good examples, though I'm sure they've had some really bad science reporting too).

The evidence of their sin? This super click-baity article: Science Says Looking at Cute Photos Can Rekindle Your Love Life. Oh man, where to begin... Here's their summary of the study:

How are long-term couples supposed to keep the love alive, so to speak? A team of psychological scientists from Florida State University (FSU) just discovered an unexpected answer — and it may lie in those images of cute puppies and bunnies that clog up your Facebook feed.

The team at FSU wanted to figure out if changing affective associations with a relationship partner through what they call “evaluative conditioning,” or learning to like or dislike something because of its perceived link with a consequence, could then lead to more positive feelings about the relationship. Married couples were asked to look at a short stream of pictures, including pictures of their partner, which were paired with — you guessed it — either cute pix of animals or less appealing subject matter, once every 3 days for 6 weeks. Wouldn’t you know it, the couples who looked at pictures of their partners paired with positive stimuli were feeling more positive about their partners than those who were forced to see their partners intermingled with negative stimuli. In conclusion, the scientists believe this result proves how our attitudes toward others can change through “passive exposure to information, and suggest novel avenues for relationship interventions.”

The original study was published by Psychological Science, which, thanks to my membership with the Association for Psychological Science, I have free access to. The original study did look at evaluative conditioning. But it was by pairing pictures of spouses with positive or neutral (not negative) images. And the sample was a bit more specific than the NBC article lets on: participants were members of heterosexual married couples who had been married less than 5 years and were all under the age of 40.

These were the eligibility requirements, so any couple married longer than 5 years and/or containing at least one member over 40 was excluded. Sure, you could argue that these couples are still very much in love (They could have just gotten married the day before and qualified to be in the study!), so demonstrating that you can make their attitudes even more positive could be quite impressive. But it's doubtful many members of the couple needed substantial "rekindling," as the NBC headline suggests. By many definitions of the word, these couples would probably still be called newlyweds.

So these findings really don't apply to older couples, couples married longer than 5 years, and same-sex marriages.

The NBC article goes on to talk about how reminders of why couples fell in love might rekindle affection, something completely outside of the scope of the cited study.

How can this article be improved? First off, get rid of words like proof; they really don't belong in scientific articles. Sure, laypeople don't think about science in the same way scientists do, but we need to stop feeding into those misconceptions, such as the misconception that we can prove anything in our research. And obviously, fix the mistake about negative versus neutral stimuli. I would guess the researchers didn't use negative stimuli because of ethical considerations; it really wouldn't be ethical to purposefully try to damage someone's relationship. Those are the low-hanging fruit that can be easily remedied.

The bigger issue has to do with the misleading headline and the lack of any detail about the sample. The headline is meant to capture attention, and a more descriptive headline (Study Finds Positive Images Can Improve Attitudes Toward Spouse Among Heterosexual Married Couples Married Less than 5 Years) is definitely not as eye-catching. But there has to be a compromise between click-bait-worthy and accurate, and even if you're unwilling to change the headline, make sure the article clearly states the specifics of the study sample.

At the very least: journalists, please stop using "science says" in headlines. It's the equivalent of answering a question "because science." I would much prefer the still generic but less hand-wavy, "because science says so" approach of "study says."

1 sin down, too many to count to go! I'll post another Statistical Sin next week.