RETRACT ALL THE THINGS!

Posted on September 26, 2012

2


PLoS has just put out an editiorial titled “The role of retractions in correcting the scientific literature” where they make a number of suggestions around retracting papers once the consensus of evidence and opinion suggests that the paper’s conclusions are false. The key section is this (emphasis my own):

… as editors and as a publisher we encourage the publication of studies that replicate or refute work we have previously published. We work with authors (through communication with the corresponding author) to publish corrections if we find parts of articles to be inaccurate. If a paper’s major conclusions are shown to be wrong we will retract the paper. By doing so, and by being open about our motives, we hope to clarify once and for all that there is no shame in correcting the literature. Despite the best of efforts, errors occur and their timely and effective remedy should be considered the mark of responsible authors, editors and publishers. We welcome further discussion of this important topic.

Retraction. This is an incredibly emotive word to many researchers because papers are typically retracted by the publisher when the work is deeply flawed or even fraudulent in nature. Often, a retracted paper is marked as if it was wearing a dunce’s cap, with large RETRACTED lettering running across its text (eg Wakefield’s infamous Lancet paper)

Putting aside that labelling an entire paper with a binary TRUE/REJECTED stamp (especially using a label that carries severe emotion baggage) is an absurd idea which misses the point and practice of literature review. To steal a phrase from Ben Goldacre, “it’s a little more complex than that.” It always is. I don’t think this move by PLoS will de-stigmatise the term, when all the other publishers will carry on regardless. They will be seen as the outlier, the one that is doing something wrong, regardless of motive.

There is a silver lining to consider however:

PLoS are willing to try unconventional things to improve the way we do science.

Yes, I agree, they got it wrong this time but please don’t just blast them out of the water with critism! Can we suggest things that might help us really use and make sense of the ever-growing mountain of published, peer-reviewed material? Consider these strawmen:

  • Annotating a paper’s abstract with links and contextual fragments from reviews and meta-analyses that discuss or use it?
  • Citation checking, likely human powered (‘crowdsourced’). A tally of whether a cited work a) exists and b) says what the citing paper suggests it says?
  • Finer grained citations – explicity  refute, reuse, review a paper, rather than just ‘cite’ a paper, so that this semantic difference is clear to us and to computer programs/text-mining apps/search engines/etc?
    • Including the ability to link to a section or even a sentence in a manner in which text-mining applications can understand.
  • Cross-domain suggestions: how can we help the biologist working with cellular fluid flow dynamics read the work of or even meet mathematicians who specialises in this area and vice-versa? There is so much literature to read out there, that it is easy and almost necessary to have a very narrow focus.

I saw a large number of responses to PLoS’s editorial on twitter last night (which have since evaporated into the ether. Twitter is not that citable anymore.) They mainly consisted of jokes: “My biology book is mostly wrong, should I burn it now?”, statements of offence: “How dare they delete papers!”, truly unhelpful knee-jerk critisms “That’s not how we do things” and so on, but some were links to better, much more thought out and reasoned arguments against it, such as this one: “The Dumbest Shit I’ve Ever Heard” which lays out why the broad-stroke application of a retraction is a very bad idea.

This may just be on my slice of twitter, but the majority of messages were not helpful. They were less than helpful, becoming A Typical Twitter Pitchfork Mob.

I’m not going to sidetrack and deal with why many reactions were inaccurate and simply say that PLoS should never have used the term ‘Retraction’

 "Retraction? You keep using that word, I do not think it means what you think it means"

A retraction is seen as a complete rejection of all the methods, data and conclusions in a paper, potentially due to malicious intent by the author. I do not think the consensus view is that a retraction is applicable in this case: “[a] paper’s major conclusions are shown to be wrong, we [PLoS] will retract the paper.” The aforementioned editorial even admits that the term is problematic: “There is much misunderstanding about retractions.” The English dictionary is a register of use as the English language never stops evolving. If  the majority of readers understand a term in a certain way (as a heinous or shameful thing in this case) then that is its meaning.

I don’t want to defend them further as it will get conflated with me defending the plan they put out (which I do not agree with.) Let me reiterate the important outcome here:

PLoS are willing to try unconventional things to improve the way we do science.

Criticising them inaccurately (“destroying the historical record” based arguments) or stating that it is wrong because it is “not how we do things are not helpful.

This is a perfect time to suggest ways in which the internet and online publishing might help or improve the way we do science. Let’s not discourage them but help them – tell them why this won’t work, but do so constructively. There is a large body of work already out there about ways to improve scientific discourse – let’s point it out to them.

About these ads
Posted in: Uncategorized