Thursday, September 22, 2016

What Has Happened Down Here is a Miscommunication

Has this ever happened to you? You go to see a movie with a friend. You love every minute of the movie, laughing at the jokes, crying when something sad happens, and cheering when the hero saves the day. The credits roll and you turn to your friend and say, "What did you think?" Your friend proceeds to trash-talk the whole movie and you find yourself thinking, "Did we watch the same film?"

That's kind of what I thought when I read a blog post a grad school classmate shared. The post was a response to a forthcoming article for the APS Observer, the magazine of the Association for Psychological Science. The article, by social psychologist Susan Fiske, deals with the new(ish) trend of criticizing psychological research in social media settings (the text of her article is provided in the blog post linked above). While her insistence that criticism of psychological research should be done either in private (i.e., peer review) or in moderated settings (e.g., letters to the editor/invited responses or discussions during conference presentations) is a bit short-sighted in my opinion, she does make a point that because it has gotten easier for people to a) get their message out there and b) get contact information for researchers, some criticisms have been little more than attacks. Attacks that are not necessarily because of issues with the validity of the research or the soundness of the methods, but because of vehement disagreement with the conclusions of the research. Although her article is short and she doesn't call anyone out by name or topic area ("because ad hominem smear tactics are already damaging our field"), she's probably talking about this:
Some [researchers] have weathered frightening vitriol and threats to their reputations. Back in 1975, US Sen. William Proxmire bestowed the first of his infamous “Golden Fleece” awards on a small federal grant given to APS William James Fellows Elaine C. Hatfield of the University of Hawaii and Ellen S. Berscheid of the University of Minnesota. Proxmire denounced their study on social justice and equity in romantic relationships as a waste of taxpayers dollars. The publicity generated threatening letters and phone calls to both scientists, and their federal funding dried up because of the stigma.

In the 1990s, renowned memory researcher and APS Past President Elizabeth F. Loftus, at the University of California, Irvine, drew considerably hostile reactions when her studies challenged people’s claims that they had uncovered — often with the help of therapists — repressed memories of abuse, molestation, and even alien abduction. Loftus even had to have armed guards accompany her to lectures after she received death threats.
Fiske talks (once again, in the general sense) about attacks that share some common elements of these extreme cases:
The destructo-critics are ignoring ethical rules of conduct because they circumvent constructive peer review: They attack the person, not just the work; they attack publicly, without quality controls; they have sent their unsolicited, unvetted attacks to tenure-review committees and public-speaking sponsors; they have implicated targets' family members and advisors.

Which is why I was completely dumbfounded when, after sharing the article in its entirety, the author of the blog post, Andrew Gelman, summed it up as follows:
In short, Fiske doesn’t like when people use social media to publish negative comments on published research. She’s implicitly following what I’ve sometimes called the research incumbency rule: that, once an article is published in some approved venue, it should be taken as truth.
Did we really just read the same article?

But it gets even weirder. Gelman begins talking about the new movement in psychological science to encourage replication of past studies, a movement that has at least created some serious doubts about the validity of past studies. He aims a lot of criticism at Proceedings of the National Academy of Sciences, and a set of articles edited by Fiske. In fact, he's done this before. To be totally honest, I agree with many of his criticisms of these papers, his concerns about the validity of studies that current researchers have failed to replicate, and even the potential errors he highlights in one of Fiske's own papers. So yes, perhaps Fiske does deserve some criticism.

Except that's not what her article is about. She isn't saying there should be no criticism; she's saying that, just as there are ethical guidelines for the proper conduct of research, there are (or should be) ethical guidelines about how to offer criticism of research. But Gelman refers to Fiske as attacking "science reformers" - the people doing replication research - when I think she's referring to ad hominem attacks. I think she would have far less issue with Gelman going through Fiske's work and picking it apart, discussing methodological and analytical errors, than she would with someone writing a blog post about how much Fiske sucks and that she should lose her position at Princeton, and hey, here's the contact information of her department chair and dean of the school, why don't you, dear reader, call them up and tell them how much you hate Fiske.

So I agree with Gelman on his criticism of some of the key studies in psychological science, and his desire for more transparency and replication - something Fiske also references in her short article. And I agree with his final conclusion:
Let me conclude with a key disagreement I have with Fiske. She prefers moderated forums where criticism is done in private. I prefer open discussion. Personally I am not a fan of Twitter, where the space limitation seems to encourge snappy, often adversarial exchanges. I like blogs, and blog comments, because we have enough space to fully explain ourselves and to give full references to what we are discussing.
I frequently do the same thing on my blog. But I take issue with his insistence that Fiske's "destructo-critics" are Gelman's "science reformers." The winds may have changed in psychological science research, but Fiske and Gelman are sailing different seas.

No comments:

Post a Comment