What a beautiful day! The sky is solid blue, the air is balmy, and I'm sitting outside doing one of my favorite things. No, I'm not drinking ‘tea’, (although that is one of my favorite things) – I'm talking to you! And isn't it a nice day? And guess what? I didn't get a scathing review today, so I'm doubly enjoying it!
If you are only just now joining us, we were talking about simple courtesy in this biomedical research thing we do. There were a couple of examples, and an odd non sequitur about the Hamilton sound track. (“I am not throwing away my shot, I am not throwing away my shot.” – now I've got that song stuck in my head. But I'll persevere.) And we left off, though, with something about this thing we do that is often blatantly, brazenly, even barbarously discourteous. That is, the way we anonymously destroy each other's work in the process we refer to as ‘peer review’.
It is often pointed out that the key feature here is the anonymous nature of the review process. Wouldn't it be nice, the thinking goes, if we all signed our names to the reviews of papers and grants we had critiqued? If we did that, we would have to be much more courteous. And we would take much more care that any criticism we conveyed would be carefully weighed against the harm it would do to our (the reviewer's) reputation. Life would instantly get better. For all of us, right?
Many journals have either instituted, or are at least considering, elevating the transparency of the review process, publishing the names of the reviewers (and academic editors, where they exist) with the published paper. This is generally viewed very favorably, not only because the reviewers actually get some credit for the (free) work they have done, but also because they are presumably more accountable for the process. What could possibly go wrong?
Some of you will complain that here I am, writing anonymously about being anonymous, simply assuming that ‘Mole’ is not my actual name. This will come as some surprise to my parents (who, coincidently, are also named ‘Mole’.) The fact is, the name I put to research papers is the one that is actually fictitious, but I digress. And the complaints actually make my point, if indeed, there is a point here. Which there is.
I suggest that there is an important reason that reviews of grants and papers is anonymous, and should remain so.
Once upon a time, long before PubMed, and even before the internet, there were scientists who did not get funding from granting agencies, foundations, or organizations. They were either independently wealthy, so much so that they could afford to not only work long hours without pay, but could even buy whatever they needed to do their experiments, or they had very wealthy patrons who supported their efforts. And when they made a discovery, they presented their findings by speaking at a society gathering and/or self-published and distributed their papers to the community. Review came in the form of public discussions or letters from fellow scientists. It wasn't always nice, and there were famous debates, disagreements, arguments, and outright fights among researchers and their detractors. And all of it was very transparent. Hated rivals publicly paired off to battle things out, and for the most part, often shook hands and agreed not to shoot each other at dawn (in these ancient times, scientists were invariably men, and mostly of high social status. I'm not saying that any of this was a good thing.)
With time, the value of science to society was sufficiently recognized by the public at large to the point that support for research became a goal embraced by governments and large organizations. It became possible for earnest, less financially endowed individuals to enter the arena of scientific exploration, and they could succeed with hard work, hard thinking, and some good luck to actually do science for a living wage. Suddenly, ‘scientist’ became a trade, an alternative to being a blacksmith, a cooper, a well digger, or (upon further enlightment) an upstairs maid. ‘Ordinary’ folks could be scientists, and could contribute just as much (and of course, often much more) to the effort as those wealthy self-funded researchers. Even a lowly insectivore such as yours truly could consider being one. (For the record, the Mole family did have two pennies to rub together, but very little else, and my training, let alone research, would have been out of the question for me. I suspect I would have tried my hand at the fool trade, as it's the only other thing I'm qualified for.)
But as a consequence, the modern process of doing science is frequently tied in with our livelihoods. I have been in the position that the publication of a paper, which in turn could lead to a grant, would make the difference between paying my rent or not (now I have a house, with a mortgage I have to pay). Publication and grants are our security, and if you threaten my security, I might bite you. (I have small, but very sharp teeth.) It is difficult, perhaps even impossible, to not take criticism of our work personally. Because, frankly, it is personal.
The trouble is, science isn't personal. Results are useful or they are not useful (as I've argued before, this is different from ‘true or not true’.) To make any sort of progress, we need a process where the quality of a body of work is effectively filtered to increase the likelihood that what we read can be used to make further discoveries. That is why work is reviewed, of course.
When we review, we have to be free to point out insufficiencies in and alternatives to the interpretation of the data. If an important control is missing, we note it. If a potentially flawed assumption was made, we describe the flaw. And if the data do not support the conclusions, we say so. Often, addressing our concerns translates, for the authors, into more time and reagents (that is, money). In short, as reviewers, we often place a drain on the resources of the authors' laboratory, resources that might have been directed toward projects that they do not regard as essentially ‘finished’, as they might consider the one you reviewed. This, in turn, often evinces frustration and anger.
And despite protests and assertions to the contrary, it is also the case that where a paper is published (once through this arduous and annoying filtration process) affects our grants, and in turn, our jobs. Anger can thereby turn to existential dread. And this is directed at the reviewers, of course. (Paper rejected=the reviewers killed us; paper accepted=we managed to get it in despite the reviewers).
By the way, even if you feel that you are in control of your emotions, never, ever, respond to negative reviews on the day you receive them. Please trust me on this (warning: digression ahead). For a number of years, I was editor-in-chief of a fairly well known journal, and would often receive ‘flaming’ emails from frustrated authors. One in particular described in some detail how half-witted, insipid, and obtuse the reviewers and the editor (me) were and how worthless the process had been, noting in the conclusion that of course there was no point in offering a rebuttal given the tone of the email. I responded that I made it a policy to reconsider decisions if the author felt they were incorrect and unjust. The author apologized for the harangue and thanked me for being open minded, to which I gently suggested the above maxim (on the cooling-off period). Minutes later I received another long ‘flame’ on my temerity in offering advice to this highly experienced author (at least in his own mind). I did not follow up. [Warning: the digression is ending, please exit carefully]. Anyway, yes, criticism can make us angry – sometimes very angry – and this can lead to hostility (and often does). It is not unusual for a colleague to approach me at a meeting to say that he or she was the reviewer on one of our recent papers. To my shame, I often immediately think (but of course do not say), “Oh which one? The one who demanded forty seven experiments that did not, and indeed, would not change the conclusions? Or the one who felt that the paper required an experiment that would take a minimum of two years to complete?” It is also not unusual that a colleague approaches me at the same meeting to tell me that they know I was the one who eviscerated their lovely paper (almost always they are wrong) and despite my protests, I have made an enemy (at least, for a little while until this, too, passes – I hope).
What can we do about this? Let's explore an alternative to the process. Instead of a review process, we dispense with all journals (wouldn't that be nice) and deposit manuscripts in an open access repository. Let the readers decide what is useful and what is not. People could leave ‘likes’ and perhaps comments, and could employ social media to attract attention to a particular study. We could envision metrics that would guide a reader to a manuscript of potential interest. Information would get out more quickly, and we wouldn't have to go through a tortuous review process. Fantastic!
Except that there is a little problem of scale. In 2018, there were (according to PubMed) 1,333,327 published papers. To only read the titles of these would require about 154 days (if you took no breaks for anything, such as sleep). One of these, for example, was “Short-term cortisol exposure alters cardiac hypertrophic and non-hypertrophic signalling in a time-dependent manner in rainbow trout,” which I'm sure is a perfectly good paper. Of course, most of these papers are not in my field. When I add up the papers that are in my own field (okay, one of my fields) the number reduces to a mere 31,109. It would still take me 10 days, 8 h per day, to only read the titles of the papers in my field for that year. I read a lot, but I can't read everything, not even in one of my own fields.
At least for now, I think we are stuck with journals. We decide which journals contain required reading, which contain reading of potential interest, and which we might not read at all (unless a paper pops up in a literature search and is particularly relevant). And this filtration of the literature is too emotional a process to move it from the realm of the personal. We publish to survive (or find some other work, which we scientists somehow relegate to ‘perish’, as though there is no other work). We cannot help but be angry with our detractors.
So, I think that the reviewing process must be anonymous. But that doesn't mean that it has to be so, well, nasty. Here is a modest proposal (one that, unlike that of Mr. Swift, does not involve cannibalism). When I review a paper, I make notes on the figures (yes, I actually do this on paper, sorry, but I'm pretty retrograde), pointing out not only concerns and criticisms, but also good things (“Interesting!” “Elegant!”). Couldn't we impart some positive enthusiasm in our list of negative critiques? And wouldn't it be wonderful to read a review that contains specific, positive comments about our hard work that went into the research, and as many of these as there are negative comments? Maybe, when we review, we might put ourselves into the authors' place (where we have all been) and point out the good along with the ‘bad’ (not to mention, ‘ugly’).
Wouldn't that be nice?
