Measuring research impact – a necessary challenge

In Australian health and medical research, NHMRC receives many more great research ideas than we are able to fund.

And so, we assess and rank applications based on merit, and largely, the merit of an idea is judged according to its ability to have impact.

But measuring impact in any area of research is an inherently difficult task.

This challenge formed the foundation of discussions this week at the Research and Impact Assessment Symposium hosted by Philip Campbell, editor-in-chief of Nature. The symposium saw rigorous discussions of impact assessment from a broad range of disciplines including the humanities, science and social sciences.

From the perspective of health and medical research, the impacts of quality research are many.

We expect breakthrough science including that in which the eventual impact is as yet unknown - big advances in health have come frequently from research into the fundamentals of life.

We expect research to provide us with knowledge on how to better treat patients, to improve the effectiveness of health policies and practices, and to yield new devices, drugs and diagnostics.

But good research publications also have an impact on other researchers, who cite the publications in their own research as a source of ideas and techniques. This “research impact” has one advantage in that it can be quantified.

Of course, citations are a far from perfect measure of the impact of research or its quality, but they are an independent indicator of the level of recognition of Australian publications by other researchers worldwide.

At intervals throughout the last 20 years, NHMRC has undertaken a rigorous analysis of the citation performance of Australian health and medical research publications of an indication of the impact of Australian publications on others’.

Our latest analysis Measuring Up 2013 shows that Australian health and medical research publications are increasing relative to the rest of the world and that increasingly the publications are transnational (doubling over 20 years).These transnational publications are cited more than when all authors are Australians.

The results also show that the overall citation index - frequency of reference to Australian publications by others- for Australian papers has also increased progressively over two decades.

One especially pleasing statistic is that more than 2.5% of NHMRC funded publications are in the top 1% most cited papers world wide – that is in the 1% most significant papers as judged by other authors.

Though there is never room for complacency, we are pleased that NHMRC’s peer review decision-making processes continue to identify research that results in publications that have impact. An important impact, but just a milestone on the road to real impact in improve health and innovative industry development.

Professor Warwick Anderson AM
Chief Executive Officer
National Health and Medical Research Council