Thursday 29 September 2016

“Methodological terrorism” and other myths



Most readers of my blog will be aware of Professor Susan Fiske’s leaked letter to the APS Observer last week. In her article, Fiske delivers a blistering invective on the dangers of social media, warning that scrutiny of psychological research by “online vigilantes” and “self-appointed data police” is destroying careers. In actions she describes as “methodological terrorism”, Fiske warns of “graduate students opting out of academia, assistant professors afraid to come up for tenure, mid-career people wondering how to protect their labs, and senior faculty retiring early”. Her solution? Scientific critique should be restricted to moderated groups and behind the walls of peer reviewed academic journals.

Well.

Fiske refuses to identify the “terrorists” because apparently that’s unprofessional (though see this argument from Neuroskeptic as to why it is vital if such allegations are true). This leaves her in an unenviable Trumpian territory of delivering a broadside against an entire community based on what “people have been saying”. Lots of people, no doubt. Good people.

The whole premise is so flimsy that it would probably have been ignored if not delivered by someone so high ranking.

At the time the article leaked I was at a conference in Germany that Fiske was also attending. I immediately emailed her, pointing out what I saw as some of the problems with her argument, and I invited her to join our session the next morning to publicly debate the issue. She declined. 

Two days later, PhD student Anne Scheel had the courage to stand up in front of a packed auditorium and challenge Fiske directly about her “terrorist” charge (if you’re not following Anne, go do that now). In response, Fiske claimed that everyone overreacted. Apparently, she just used that term to get people’s attention, but says she would still defend the “conceptual basis for that wording”.

Oh irony.

Now you may well ask, what evidence is there that Fiske's terrorists even exist in sufficient numbers to warrant such a panic? That answer is, not much. Since 2015, UCL neuroscientist Sam Schwarzkopf has operated a blog inviting those who believe they have been bullied by methodologists or the open science movement to speak up, anonymously if they prefer. How many victims have come forward?

Not one. 

So what’s really going on here? The truth is that we are in the midst of a power struggle, and it’s not between Fiske’s “destructo-critics” and their victims, but between reformers who are trying desperately to improve science and a vanguard of traditionalists who, every so often, look down from their thrones to throw a log in the road. As the body count of failed replications continues to climb, a new generation want a different kind of science and they want it now. They're demanding greater transparency in research practices. They want to be rewarded for doing high quality research regardless of the results. They want researchers to be accountable for the quality of the work they produce and for the choices they make as public professionals. It's all very sensible, constructive, and in the public interest. 

Placed in this context, Fiske’s argument – as inflammatory as it is – is actually quite boring and more than a little naive. Social media is democratizing science, and nobody, not even the most powerful among us, can stop its inexorable rise. Does anyone really think that Pandora’s box can be closed, returning to the halcyon days of the 1990s (or earlier) when critique of science happened behind closed doors among selected elites?

What’s happened instead is that technology has empowered the new generation to speak freely and publicly on scientific issues, to critique poor quality science and practices, to bust fraud, and to break the bounds of peer review. Twitter in particular has shaken the traditional academic hierarchy to its core. On Twitter a PhD student with thousands of followers suddenly has a greater voice than an Ivy League professor who might have no social media presence at all.

If the traditionalists want to preserve their legacies (and they are worth preserving) then they would do well to remember who they truly serve. Psychology does not exist to support the interests of researchers seeking to command petty empires and prestigious careers, less still to protect the reputations of those who have already “won” in the academic game of life. Psychology has but one master: the public. That’s why psychology featured in over 400 “impact case studies” in a recent audit of UK science, and it’s why millions of dollars, pounds and euros are being invested in new opportunities for all scientists, including psychologists, to do more transparent and robust research.

We remain forever accountable to that public for the quality of research we produce. Those who are prepared to critique science – and scientists too for their choices – are performing a public service that warrants our appreciation not condemnation. To Fiske’s “terrorists” I therefore say thank you. And to Fiske herself, who complains about the tone of scientific debate while calling it terrorism, I say: physician heal thyself. 

For other responses to Fiske, see these great posts by Andrew Gelman, Ana Todorović, Xenia Schmalz, Neuroskeptic, Andy Field, and Tal Yarkoni.

8 comments:

  1. Yes this pretty much nailed the issue I'd say.

    ReplyDelete
  2. Very good. I think "gatekeepers" is a good term for those who you describe as traditionalists looking down from their thrones, the authoritarians who are not used to being challenged.

    Just one fly in the ointment:
    "This leaves her in an unenviable Trumpian territory of delivering a broadside against an entire community".

    Which candidate was it who made a Fiskean broadside against an entire community, by describing millions of voters as a basket of racist, homophobic deplorables?

    ReplyDelete
    Replies
    1. As stupid as Clinton's statement was, it's safe to say that Trump is still way in the lead on the scoreboard of offensive broadsides.

      Delete
  3. Hi Chris, Interesting topic and I agree with the points made that it is not possible nor desirable to turn back the clock re. social media, and that scrutiny of psychological science data may encourage high standards, however, if you find yourself and your work being thoroughly investigated because the voluntary data police report some trivial error/plagiarism in one of your past papers you might feel that the issue warrants greater attention in terms of protection for the academic. I am aware that this has happened to a highly qualified academic due to such an organisation reporting that an ex. Ph.D. student copy/pasted part of the introductory section from a paper previously published, the senior academic was held to account in an investigation that lasted several months. As an academic myself, i found it quite concerning that one could be subjected to this type of treatment, and the resultant negative impact on a career that had a previously impeccable research record, all because a postgraduate student committed minor plagiarism that you didn't pick up on? Come on!!! To me the whole episode seemed like a witch-hunt, and i wondered who reported this to the voluntary organisation, and who are the people that sign up to these organisations (disgruntled students?). Anyway, with that scenario in mind, I feel that although Susan Fiske's speech was a bit OTT (but blistering invective???) she has perhaps done some service for academics in pointing out that it is a v. vulnerable position to find yourself in; these anonymous sources can set out to damage a career perhaps due to some personal grudge. For this reason i think i won't put my name to this -- it's an old saying now, just because you're paranoid doesnt' mean they're not out to get you! :)

    ReplyDelete
  4. Hi Chris, Interesting topic and I agree with the points made that it is not possible nor desirable to turn back the clock re. social media, and that scrutiny of psychological science data may encourage high standards, however, if you find yourself and your work being thoroughly investigated because the voluntary data police report some trivial error/plagiarism in one of your past papers you might feel that the issue warrants greater attention in terms of protection for the academic. I am aware that this has happened to a highly qualified academic due to such an organisation reporting that an ex. Ph.D. student copy/pasted part of the introductory section from a paper previously published, the senior academic was held to account in an investigation that lasted several months. As an academic myself, i found it quite concerning that one could be subjected to this type of treatment, and the resultant negative impact on a career that had a previously impeccable research record, all because a postgraduate student committed minor plagiarism that you didn't pick up on? Come on!!! To me the whole episode seemed like a witch-hunt, and i wondered who reported this to the voluntary organisation, and who are the people that sign up to these organisations (disgruntled students?). Anyway, with that scenario in mind, I feel that although Susan Fiske's speech was a bit OTT (but blistering invective???) she has perhaps done some service for academics in pointing out that it is a v. vulnerable position to find yourself in; these anonymous sources can set out to damage a career perhaps due to some personal grudge. For this reason i think i won't put my name to this -- it's an old saying now, just because you're paranoid doesnt' mean they're not out to get you! :)

    ReplyDelete
  5. Ah - if only life was so simple. Now one of my old professors courtesy of Sue Greenfield did the Royal Institution Christmas lectures a few years back. Now this guy hates philosophy, mainly because he can't do, wouldn't know Bertrand Russell from Russell Brand. Tried very hard to explain the Teleological concept to him. Not a chance, complete waste of time. However he knows and chooses to ignore his inability, like most of us, cognitive dissonance is a bitch. Anyway, does Sue Fiske have a point. No. Knowledge is not a closed field. She should welcome the distraction with open arms and a smile playing on the lips. First rule of learning - a wise man can learn from a fool, but it only works one way. CDH.

    ReplyDelete
  6. Hi Chris,

    I had seen your post before Titus referenced it in his post

    http://ivory.idyll.org/blog/2016-what-is-open-science.html

    under "researchers seeking to command petty empires and prestigious careers", but it only now occurred to me that you would be among the experts to ask: one can now see the slow formation of an alternative narrative to competition, the "commons". I can see this development both in scholarship (scholarly commons https://www.force11.org/group/scholarly-commons-working-group ) and in the wider political public, e.g.: https://europeancommonsassembly.eu/#section1

    On the competitive-collaborative spectrum, can individuals we reasonably classified as belonging more to one or the other end of the spectrum and if so, is there any data on which populations tend to be composed of which fraction of each class? Worded a little more neutrally: are there studies classifying different human populations according to a competitive/collaborative score and if so, what are the distributions found in these populations?

    ReplyDelete
  7. Big problem is that we train rather than educate our students. Serious thinking about difficult issues does not even glance off the edifice we have erected. Nor will it (any time soon), since those trained practitioners become next generation's mentors.

    ReplyDelete