How's everyone doing tonight? Yeh? Okay! You sir, where are you visiting from? Really? I spent a month there…one day! I mean, the town is so sleepy you need a prescription for a cup of coffee. Okay, so I got a new car. Really nice, very fuel efficient. Not much power, but the squirrels get a lot of exercise, and they work for nuts. Hmm, tough crowd. Hey, I'm putting in the time here, give me a break. I'm telling you – it isn't fair. Other comics? They get paid gigs, spots on Saturday Night Live, maybe even a stand-up special on Netflix. The system is broken. I'm telling you – it's broken. See how you'll feel if we all decide not to do this anymore. You think you don't need comedy? You'll see. Me and the squirrels are going home. Give me those peanuts, and I'm out of here.
Obviously, I'm not a comic (‘obviously’ – I heard that). But I do have a point. Comedy is incredibly hard. Even harder than this thing we do, this biomedical research thing. The only reason you do comedy is because you really want to. You try out your stuff on open mics, bomb, you crash and burn, get up, do it again, and maybe eventually – if you're good enough – you'll get paid a little. If you're really good, you'll make enough to do it for a living, but very few get that far. And those that do? They still have nights where they crash and burn.
Me, I don't have the talent, or courage, to do stand up comedy. But I know all about crashing and burning. My last lovely, beautiful grant wasn't even discussed. I'm currently awaiting the bad reviews on the next one, while I'm writing yet another one. I work very hard, and no, it doesn't feel fair.
Recently I upset a lot of people by writing about ‘quiet quitting’, and I'm sorry about that (not about you being upset, but because of what I said). I was out of line. I did not mean to imply that everyone should just work hard and stop complaining, and I was wrong to suggest that the system is only performance based. But let's talk about this ‘system’.
We talk a lot about the system being broken, so I thought it might be useful to talk about the system as I see it. Before you flame me, the following is just how I see it; your experience might be different, and I respect that. But it would be good to have a conversation and consider the alternatives. It might not fix anything (as I'm just an insectivore, and the powers responsible for the system do not listen to me) but maybe we can lay it out to see where the problems are. If nothing else, it might be cathartic.
Today, we're going to talk about grants. You know, GTFM (getting the freaking money). I have talked about this subject before [a few of these are: Peerless I and II, Money Back Guarantee I and II, FCTWAWKI] but it is one that is worth coming back to. Later we'll talk about institutions and how we disseminate what we discover. I should also point out that my perspective here is from my understanding of the funding mechanisms in my own country, the one that makes Indiana Jones movies (yes, I can't wait – I'm prepared to be disappointed, but still, it's Indy). I do have some experience with funding in other countries, but I could have the details wrong.
Here's how it works in my country. Each year, the government (hopefully) approves a budget that includes money for biomedical research. This is by no means guaranteed, and there are a great many people who work very hard behind the scenes to cajole, argue, plead, and make deals to try to obtain this money. The fact that most of the taxpayers in my country want biomedical research to happen is helpful, but it is never top of the things on which they want the government to focus. In any case, a very large amount of money is usually provided to this endeavor (note: by ‘very large’ I mean what a rich hedge fund manager is worth, but not enough to build, say, five nuclear-powered aircraft carriers).
If, indeed, the money is allotted for biomedical research, our government-supported institute apportions a part of this money to support grants from outside this body (that is, most of us doing research in my country). This distribution of funds is challenging, for several reasons. First, nobody agrees on what are the most important areas to support. At least once a week, representatives from this institute (actually, institutes, but you know what I mean) are called before the government body that provides the money to answer questions about where the money is going. And people within that body ask questions that concern their own constituents (or, very likely, donors to their campaigns). It is not all based on ‘science’.
Once the extramural (‘outside the walls’) money is apportioned, each research area sets up panels of scientists to evaluate grants that are submitted in the hope of getting a little bit of this money (a very good grant award in my country would pay for about 30% of a cruise missile, not that I'm trying to make a point with these comparisons. Maybe I am, though…). To try to give at least the illusion of fairness, these panels are meant to represent scientists from all regions of the country, with attention given to gender, under-represented minorities, and different levels of seniority. For some reason, these panels are called ‘study sections’ in my country, evoking oak-paneled rooms filled with aged scholars; in reality, we generally meet in low-budget hotels where the coffee is terrible (and no, they don't buy us coffee). This is challenging, because: (a) reviewing grants is a great deal of work; (b) there are minimal rewards for doing it (we are paid to reimburse most of the travel and lodging, and upwards of 50 cents an hour for our time); and (c) the dates are only set a few months before the panel meets, which can be tricky schedule-wise. So, while an effort is made to compose these panels with people who are at least familiar with the area of research covered by a particular group, one takes what one can get. And everyone gives a score on every grant unless they have a genuine or perceived conflict of interest.
Technically, these review panels are only advisory; it is the institution that ultimately decides who will get a grant funded. The attitude of the government-supported institute that holds the purse strings is that all reviews evaluate only the quality, novelty, and importance of the proposed research, and that it pretty much doesn't matter who might be doing the reviewing, since these factors are universally appreciated. Also, the people doing the reviewing are not at all concerned with whether or not the applicants are viewed as ‘already having enough money’, (whatever that is) or ‘living hand-to-mouth’ as it were. All of this is, of course, nonsense, but it is a useful fantasy.
Here are some of the ways in which I feel this system is ‘broken’. First, the money is spread so thin that it takes only one bad review (from out of three, sometimes four, reviewers) or one very bad score (from anyone on the panel) to kill the chances for a grant. It is daunting to sit in front of a room full of colleagues and offer an opinion; the safest thing to do is to land somewhere in the ‘it's good but not great’ range – it shows that one is open to the ideas in a grant but not willing to fight for it, and a few minor problems will justify this score. Examples of such problems can be that the work seems ‘ambitious’ – apparently this is a bad thing – or that the reviewer can envision other experiments that might be done. A score of ‘good but not great’ does not get funded. Reviewers can also, of course, make mistakes. A very good reviewer will own up to it if a mistake is called out during discussion, but there are very few very good reviewers in my opinion. I once reviewed a grant that I liked very much, but was chastened by another reviewer who noted that the levels of the molecule being studied were not physiological; when I subsequently pointed out that the applicant had shown that this was incorrect, he reverted to ‘lots of other problems’. If one of the reviewers doesn't like a grant, for whatever reason, it is essentially doomed. Oh, and here's something else: a statistical study of a large number of grants reviewed across the spectrum of scientific disciplines found that scores declined significantly with hours since the last meal the reviewers received (this is real); apparently blood glucose is a factor in the review process (and they don't buy us anything to eat). And don't even get me started on ‘virtual grants panels’.
It is what is referred to as Schopenhauer's Law of Entropy (one of Murphy's Laws): if you add a teaspoon of wine to a barrel of sewage, you have a barrel of sewage; if you add a teaspoon of sewage to a barrel of wine you have a barrel of sewage. (The panels my grants go to, of course, are barrels of fine wine that simply revile my grants).
I have only described how things work in the government-supported funding agencies in my country, and of course there are other foundations that support research. Many, but not all, of the issues I have mentioned apply to these as well (although some of them do give us coffee and lunch). The way science is supported in your country might be different in the details, but I suspect that we will agree on some of the issues.
There are a lot of proposals for how to fix grant reviews. One is to make everything blind with regard to the applicant, i.e. so that we don't know who is proposing the research. Aside from the fact that there are easy workarounds for this [“as we have shown (references 1–16)”], this only alleviates the perception that ‘people who have already been successful get all the money, only because they were previously successful’. That is indeed a perception, but one should consider that maybe they are successful because they propose very interesting, exciting, and important research. In any case, I honestly don't think that this approach of blind review will fix anything.
There have been other proposals. When a group of acclaimed scientists (including a number of those with a Nobel prize) were asked to fix things, they (apparently) seriously suggested that it would be easier and more cost-effective if all of the money were equally distributed among this advisory group, and they would, in turn, see to it that it went to the right places. Good idea (he says, perhaps, sarcastically).
I'm sure there are other suggestions, and I would love to hear them. But of course, I have my own (because, hey, I'm the Mole – I always have ideas, even if nobody listens to them). The main problem, as I see it, is that almost nobody really wants to review grants. But what if we made grant reviewing genuinely rewarding? So rewarding that lots of people would fight to be included in the process (letting our institutions pick from this greatly enlarged list). If I review a grant in my field, I have an inherent conflict of interest; supporting your project supports my own competition. I for one have no problem with this, but it might be a consideration for some. But what if we made it so that anyone who is chosen to review grants for a given period is: (a) barred from submitting grants during this time, and (b) given an extended payout on their grants from the funding body for the period for which they serve? That is, if we are reviewing for a year, our funding is extended for that year. I bet that anyone who is currently funded would see this as a very desirable deal; so desirable that if one were asked to be temporary (with no reward other than we might be picked for this deal if we prove valuable), we would happily do it. Quid pro quo. There would be a consequence for being an obstructive, lazy or retaliative reviewer (or even being perceived as one), since they might have the deal revoked. And we would do the very best job we could, in the hope that we would be asked back.
It's a start, maybe? Not that anyone is going to listen to me. I bet you have your ideas as well. Please share.
But there is another aspect to all of this that goes a bit deeper. Somehow, discoveries in biomedical science are meant to better the lives of the people from whom the money comes, that is, taxpayers and donors to foundations. The System is built on the idea that this will happen. But let's talk about that next time. Right now, though, I'm going to watch a comedy special. This comic cracks me up.
