In an article in yesterday’s Australian, David Uren alleged that ‘the distribution of NHMRC funds needs to be reformed before the pot is fattened.’
He continued, saying that ‘The peer-review process used by the National Health and Medical Research Council too easily descends into mates funding mates.’
Although our records show that Mr Uren has no personal experience in the NHMRC grant application or assessment process, it is important for us all to recognise that this belief lingers in the public sphere.
As CEO, I have made it a priority to demystify and to open up our peer review processes, to make them as fair as possible and able to identify the best research without fear or favour.
To this end, I have introduced a number of initiatives over the last 5 years.
I have separated the independent assessors who first consider grants from those who will consider and score the grants on the peer review panel in the next round (Grant Review Panel, GRP). This ensures separation of decision making and so that a panel member cannot determine who will be the assessors.
I have introduced a system of independent Chair. Each peer review panel is chaired by a respected senior researcher who does not participate in the peer review, but chairs the discussion, voting, and controls conflicts of interests, in line with NHMRC’s policies.
These polices include specific criteria for scoring, secret voting by all 13 or so panel members against the criteria, allowance for the opportunity that each applicant has to conduct research (e.g. consider their track record by taking into account clinical or teaching loads) and ensuring that our career disruption policies are adequately considered (allowing greater flexibility in assessments of achievement for these whose careers have been interrupted by parenting responsibilities or illness).
I have also introduced a system whereby independent community members attend each grant review panel to observe whether the panels are operating properly and without fear or favour. They report separately to senior NHMRC staff and their reports are provided to our Council for all our funding schemes. These observers are not researchers, but upstanding members of the community – lawyers, doctors, teachers and business owners. They serve to represent the Australian public to ensure that Australian dollars are spent in an impartial, just way. Perhaps we will invite Mr Uren in 2015!
There are other observers of the process too. Early career researchers are invited to attend – this gives them useful insight into how the peer review system works (and also means they are able to debunk myths they may have heard about grant assessments).
NHMRC also has robust and thorough rules around the declaration of interests and the management of conflicts of interests. If an assessor and an applicant are employed by the same institution or department, if they have collaborated on research in the past, or if they interact socially, amongst a number of other things, and it is deemed a high conflict of interest, they cannot participate in the panel’s considerations. In fact, under the new PGPA Act, all declarations of a conflict of interest are available to all panel members. This enables self-policing amongst the group to ensure that each declaration is ranked correctly.
Finally, if it were a case of mates funding mates, I must say a researcher would have to be incredibly popular. Before being awarded NHMRC funding, all grants are reviewed and scored by an average of 15-20 peers. This number means a huge range of views, ideas and opinions are represented. For anyone who has spent time with researchers, they will know that asking 20 to agree on dinner is difficult enough, let alone the value of a proposal.
At NHMRC, we monitor, evaluate and adjust our peer review policies and practices each year to ensure that decisions are as fair as possible, that vested interests are avoided, and that our panels are able to identify the highest quality research.
Importantly, this is informed by the thousands of health and medical researchers who give feedback to NHMRC. Our peer review panels are not simple or perfect, but they are transparent, which is crucial to ensure that public money is used responsibly.
For more information you can watch the NHMRC Grant Review Panels 2014 - Induction video, view the summary of the GRP process for 2014, or read my thoughts on peer review at the end of 2014 GRPs.