What is different?
Publication assessment in NHMRC’s track record based schemes will be founded on an applicant’s list of up to 10 of their top publications in the past 10 years (taking into account career disruptions). This change will support NHMRC’s policy of increasing emphasis on the quality and contribution to science of the publications, not their quantity, and in so doing:
- help drive sectoral change to value research quality rather than quantity of publications
- help make assessment of publications equitable for applicants across all career stages and research fields
- significantly reduce peer reviewer burden
- align with publication assessment practices of many international funding agencies.
For further details, please refer to the NHMRC communique.
Frequently Asked Questions (FAQs)
- When will this assessment change be implemented?
NHMRC has just concluded a targeted consultation with peak bodies in the health and medical research sector. Following this consultation process, this assessment change will be implemented first in the 2022 round of the Investigator Grant scheme. NHMRC will introduce this change in other funding schemes which include publication track record assessment wherever feasible and will highlight it in the Key Changes section of the scheme documentation.
- What can be included as ‘publications’? Can I include other research outputs in publications?
NHMRC accepts publications in ten categories: Accepted for Publication, Books/Chapters, Editorials, Journal Articles (Original Research), Journal Articles (Review), Letters to the Editor, Preprints, Research Report – commissioned by Government, Industry or Other, Technical Report and Text Book.
Other research outputs, including but not limited to patents, datasets, databases, educational aids, equipment, models, protocols, software etc., can be cited in the Research Impact section of Track Record to demonstrate or verify the impact of your research. Further details will be made available in the 2022 Investigator Grant Guidelines (and subsequent grant guidelines for other schemes as applicable).
- How do I include my publications beyond the 10-year time period?
Publications beyond those nominated in the (up to) top ten, or those older than 10 years, can be cited in the Research Impact section of Track Record to demonstrate the impact of your research. Further details will be made available in the 2022 Investigator Grant Guidelines (and subsequent grant guidelines for other schemes thereafter, as applicable).Should you wish to include your publications beyond the 10-year time period due to a career disruption, please refer to FAQ 5.
- How will publications be assessed?
Publications will be assessed based on the quality of the research and the author’s contribution to science. Applicants could approach this by explaining how the research met high standards of rigour and reproducibility of the central finding(s), the impact of the finding(s) on the advancement of science and the author’s specific role in the described work.
However, applicants should note that publications are just one component of track record assessment, the other two being Research Impact and Leadership. Although each component is scored separately, NHMRC peer reviewers make an informed judgement of the applicant’s track record based on all three components. Therefore, it is important to note that applicants may benefit from weaving a narrative of their contribution and impact through all three components that tells a ‘story’ of their track record.
- How can I ensure that my top publications over the last 10 years account for career disruptions?
If you have a career disruption(s), you do not need to include additional publications in the Career Disruption section. Instead, your nominated (up to) top 10 publications will include the last 10 years plus any additional time accounting for the career disruption(s) (e.g. if you have had a six-month career disruption within the last 10 years, you can nominate any publication within the last 10 years and six month period). This approach ensures that all nominated publications are part of a single list, rather than the extra publications appearing separately. Further details on how to nominate your (up to) top 10 publications will be made available in the 2022 Investigator Grant Guidelines (and subsequent grant guidelines for other schemes thereafter, as applicable).
- I am an early or mid-career researcher and do not have 10 publications over the last 10 years (accounting for career disruptions) that I can nominate in my application. What should I do?
Applicants are asked to nominate up to 10 publications over the last 10 years, accounting for career disruptions. Peer reviewers will assess an applicant’s publication list relative to the information provided in the ‘Career Context’ section of their application, as per NHMRC’s Relative to Opportunity Policy.
Further details on how to nominate your top publications will be made available in the 2022 Investigator Grant Guidelines (and subsequent grant guidelines for other schemes thereafter, as applicable).
- How do reviewers assess quality and contribution to science in publications?
NHMRC’s Research Quality Strategy aims to ensure quality and value of NHMRC-funded research by providing guidance and support for good research practices throughout the research cycle.
High-quality research that is rigorous, transparent and reproducible contributes to scientific progress. It is essential for the translation of outcomes into practical and clinical applications and evidence-based policy, delivering the highest possible value for research investment and promoting community trust in scientific findings.
NHMRC’s peer reviewers use their judgement and knowledge of their fields to assess quality of the research described in the publication, its contribution to science and the author’s specific role in the described work, relative to opportunity. Applicants should note that peer reviewers assess the quality of the publication and its contribution to science, rather than the quality of the journal in which it is published. There has been an increased movement in the academic community towards measuring quality, rather than quantity, of an individual’s research output [1] [2]. The Leiden Manifesto for research metrics, a best practice standard in research evaluation states that informed judgement on the local relevance of research, individual experience and expertise, and variation by field in publication and citation practice should all be taken into account by peer reviewers while making a holistic assessment of an applicant’s research and/or track record.
NHMRC is a signatory of the San Francisco Declaration on Research Assessment (DORA) [3] which notes that it is not appropriate to use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions. The scientific content of a paper is much more important than publication metrics or the identity of the journal in which it was published.
NHMRC encourages peer reviewers to use their own judgement and advises them to be cautious with use of tools such as citation metrics as a surrogate for research quality. Other indicators of quality may include the reproducibility of the research, pre-registration of the study (e.g. on the open science framework), the sharing of data and the plausibility or soundness of the research.