2025 Synergy Grants peer reviewer briefing

This video was recorded in May 2025.

Video transcript

Welcome (00:01)

My name is Justin Graf. I'm the Director of the Investigator Grants Section at NHMRC, and I have responsibility for delivering the Synergy Grants Scheme.

Thank you for joining, it's great to have you here.

I'd like to introduce Lindsay Hackett and Lauren Clarke, who you can see pictured here. Some of you will remember Nada Rankin, who used to look after Synergy Grants. Nada is now happily retired, and so we're in the very capable hands of Lindsay and Lauren.

They're looking forward to doing their first Synergy Grant round from start to finish.

Thanks, Lindsay and Lauren for being here today. So, this is the sixth round of Synergy grants. You might recall that we did skip a year because of COVID, but we're obviously well back on track. You may have also noticed we've got an extra $5 million of funding, a total of $55 million this year, providing 11 grants. That's also pretty positive news for this little scheme.

Today, you'll probably hear from me for around 25 to 30 minutes. It's designed to be a resource for you to use throughout the peer review process. So, it is quite text-heavy with a lot of information.

It points you to parts of the peer review guidelines to refer to where there'll be further detail. So, like most years we try and have a good portion of our peer reviewers return. I think this year we're up around 70-80% of peer reviewers that have done Synergy Grants before. So that's great. And we also love to have new people join the Synergy Grants family.

So, it's a bit of a refresher for those new peer reviewers, a good overview of the whole process. So, we'll get into it.

Objective and Intended Outcomes (1:33)

Just looking at the objective of Synergy Grant Scheme. It's to support outstanding multidisciplinary teams of investigators to work together to answer a major question that can't be answered by a single investigator or indeed, if individual projects are funded individually, it's bringing the right people together to create something that's actually extra special to answer that major question.

What we want to see is that multidisciplinary teams and multidisciplinary research is funded, that addresses that major problem across all areas of human health and medical research, from the discovery end to translation. And so, we're looking for highly collaborative teams of diverse researchers. Biodiversity, we'll talk about later, but that might include things like gender, career stage, and cultural background, all working together to solve this problem. And of course, there's the link to our Synergy Grants website for more information.

Responsibilities of peer reviewers (2:29)

The main responsibilities you have as a peer reviewer, the first bit of advice is to be familiar with the peer review guidelines. This really is your one-stop shop for all the information you need to know about how to do your job, as well as other documents like the grant guidelines, support packs, and any other documentation that NHMRC provides.

Those documents really provide all the information to you and you'll need to refer to those throughout the process. I think everyone has almost finished their conflicts of interest, so thank you very much for doing that. So, this year we employed a slightly different approach, something that will be familiar to people that have reviewed for Ideas Grants or Investigator Grants, where we've actually provided a bespoke list of applications to declare your conflicts and your suitability against that hopefully aligns with your expertise.

We use an in-house software tool to allocate those applications, and it's based on the information you have in your peer reviewer profile, the peer review information in your profile, and the information the applicant puts in the applications. We kind of put those 2 data sets together and try and allocate applications that you'll have expertise in. It's not completely perfect, so we know it doesn't always work for everyone.

We'd value any feedback on how those applications were allocated to you. We do encourage peer reviewers to undertake the online implicit association test for gender and science, and even if you've done that before for other schemes or for indeed Synergy, it's good to do that every year if you could and also review the Royal Society video on understanding unconscious bias.

It's also a useful tool, but really your main job is to conduct fair and impartial assessments against the Synergy Grant criteria using the score descriptors and we'll talk more about those for each application that is assigned to you and in a timely manner. We don't like to hold up things too much and we want to try and get outcomes to applicants as soon as we can.

Some of you may have a role to play in assessing applications relating to Aboriginal and Torres Strait Islander health, where you'll assess the application against the Indigenous Research Excellence Criteria.

We have 3 applications that meet the criteria this year and then scoring peer reviewers will need to take that into account if they're allocated those applications and then in the second stage of Synergy peer review, your role is to conduct fair and impartial assessments of each individual Chief Investigator's track record, taking into consideration their research achievements relative to opportunity, which we'll cover later on in the presentation, and including any career disruptions the applicants may have.

Disclosure of interests (5:16)

As you're all aware, you've just completed the disclosure of interest phase of the process and as you would realise, you will be conflicted. It's entirely normal to be conflicted with applications assigned to you and we do seek some clarification at times.

We're just in that process at the moment of going back to some of you to clarify your conflicts to ensure that we make the right rulings on those. It's important to remember that these conflicts can arise at any point.

Once you're getting into the applications, you might realise. Oh, actually I know this person, I'm conflicted and if that happens, just get in touch with us as soon as possible and we can reassign that application to someone else and just a reminder, it is a legal obligation under the Public Governance Performance and Accountability Rule.

Confidentiality undertaking (6:01)

Confidentiality, you’re reminded of the importance of this, and this is one of our risk mitigation strategies in that it's important to consider privacy across the whole peer review process.

So, it's a legislated responsibility of all peer reviewers not to disclose any confidential information to which they become privy to as a result of their role as a peer reviewer. So, documents contain personal information, potentially commercial and confidence information and you need to handle this accordingly according to the Privacy Act and ensuring that once the process is finished that you destroy any material, and you don't discuss this with anyone and this is a lifelong obligation.

So just when the peer review process finishes, you still need to maintain confidentiality.

Key changes (6:53)

Looking at some key changes to the scheme this year, the first one we'll talk about won't impact your role as a peer reviewer, but just to be aware that NHMRC has streamlined its cross-scheme eligibility framework.

Synergy Grants no longer count towards application or grant capping eligibility. Now that may have led to an increase in applications this year, we've gone from 56 applications last year to 85 this year. It may be because of other reasons, but it does no longer contribute to that eligibility rule. Applicants are no longer required to respond to the 3 research impact sub-elements in separate sections.

Previously they had a separate section for each of the 3 elements. Now we have 1 field in the application, and you'll be provided one block of text to assess the 3 research impact sub-elements separately. So, you still assess them independently. The applicant just has one field to address that.

They also have a second field now where they can provide all their evidence and research impact claims. So that's separate from that body of the 3 separate elements. We have included a section in all of our grant guidelines regarding our statement on sex, gender, variations of sex characteristics and sexual orientation in health and medical research. So just to be aware of that and probably something that's quite new for returning Synergy peer reviewers is that this year you'll have the opportunity to review the applicant feedback you provided in Stage One during a 2-week period where you can review the applicant feedback provided by other peer reviewers of the same application. And we'll talk a little bit more about your obligations in that process.

The path ahead (8:39)

Looking ahead, we've had 85 applications submitted. That's quite a lot. That's meant we've had to recruit more peer reviewers. There will be probably an increased number of new Synergy Grant peer reviewers. As you know, you've just been completing the conflict of interest and suitability.

We may need to go out to some of you for applications that we don't quite have the minimum number of 5 peer reviewers allocated or with suitability to see if we can get some more suitability for those applications that don't have suitable peer reviewers.

There may be 5 to 10 additional applications you might need to declare conflicts and suitability against, just to make sure we've got that coverage and we're able to assign the best peer reviewer with the relevant expertise.

We're also seeking assessments for those 3 applications relating to improving Aboriginal and Torres Strait Islander health to assess the Indigenous Research Excellence criteria. There were 3 of those applications.

Following this, so Stage One will kick off around 19 May. We're hopeful of that happening on the day, but it maybe a day or 2 [later] and that's the assessment of the first 2 criteria, which are Knowledge Gain and Synergy, and we'll talk about a bit more about the 2-stage process. you'll have 3 weeks to do those assessments.

We are introducing a new peer review mentor question and answer session during that Stage One period, probably around halfway in that 3-week period to answer any of the questions as you get stuck into your reviews. The applicant feedback sharing process will commence around 10 June, and we'll have 2-weeks for you to do that and then the Stage Two assessment of track record for those applications that progress to the shortlisting will commence around 14 July, and then scores will be due 3-weeks later. Another 3-week period for Stage Two.

Just note down those dates and fit that in around your busy schedules.

Planning your reviews (10:45)

Just some tips on planning your reviews. I guess the number one tip we say to any peer reviewer across our grant program is to start reviewing your applications early. It is a big load. It is a reasonably short period of time, 3-weeks and the key is to start early.

Looking at workload, we're hopeful of only being able to allocate maybe 5 to 10 applications for Stage One. For Stage Two, depending on the number of CIs on those applications, typically, if you're involved in Stage Two, you'd receive 25 to 30 individual track records to assess during that 3-week period.

Some sort of generic tips that we've got from previous peer reviewers and peer review mentors is to consider assessing one criterion at a time for all of the applications allocated to you and also consider completing your assessments in blocks and check back on previous scores.

This won't work for everyone, but there are some common themes that come through from peer reviewers, but really if you have any issues or concerns, please get in contact with the secretariats, which are Lindsay and Lauren early and as often as you like throughout the whole process. And we're more than happy to answer any questions or direct you through or liaise with the peer review mentors to seek guidance on other questions and I'll talk a bit more about that later.

Application stages (11:58)

Just a quick overview of the 2-stage approach. So obviously, applications are submitted and then the Stage One assessment is against the 2 criteria, Knowledge Gain and Synergy, and that's assessed by up to 5 peer reviewers.

Synergy has a great track record over the life of the scheme that every application receives assessment by 5 peer reviewers, so we intend to continue that this year. Then applications go through a short-listing process. I'll talk more about that later and then those applications that progress through to Stage Two, each chief Investigator's track record, so covering publications, research impact and leadership will be assessed by another 5 peer reviewers.

Following that, there's a ranked list that's based on those scores, and that's used to provide the funding recommendations to the minister to sign off on.

Stage One (12:51)

Looking more about Stage One. This covers 2 of the 3 assessment criteria, Knowledge Gain and Synergy which each contribute 30% to the overall score for the applicant. Peer reviewers won't have access to track record information. This is to ensure the independent assessment of Knowledge Gain and Synergy before the detailed track record is assessed.

Now, this is not to say that during the assessment of Synergy, perhaps or even Knowledge Gain, you won't get a little bit of an insight into someone's track record but you're not provided information about their publications, research impact, and leadership responses. And it really is to focus on only those applications that are competitive on Knowledge Gain and Synergy will progress through to Stage Two and for those new to Synergy Grants and maybe unfamiliar with other grant schemes such as Investigator and Ideas, Synergy Grants also employs what we call an application-centric peer review process. And so, this means that we don't hold in-person or Zoom grant review panels.

What we do is we match applications based on the suitability that you've declared through the process you've just undertaken with the subject matter of the application. So that by and large, the applications you receive will be in your field of research as close as possible and that way you understand the subject matter, you'll be familiar with their publications, and hopefully an overall easier process for you.

Assessing Knowledge Gain (14:21)

Just looking at Knowledge Gain, NHMRC defines Knowledge Gain as the quality of the proposed research and significance of the knowledge gained. It incorporates theoretical concepts, hypotheses, the research design, the robustness, and the extent to which the research findings will contribute to the research area and health outcomes.

When assessing Knowledge Gain, does the application demonstrate (14:42)

When assessing Knowledge Gain, it's important to look at how well the applicant has provided a clear and justified reason for the research hypothesis and their rationale.

You want to look at the strengths and weaknesses of their scientific framework and the study design, the methods and analyses they plan to employ, and whether the proposal tackles a major question addressing an issue of critical importance to advance the research or health area and so importantly here, it's not the prevalence or magnitude of the issue you know, so cardiovascular disease or prostate cancer, just because they're more prevalent than other conditions doesn't necessarily mean you should score higher.

So you need to look at whether or not they have access to technical resources, they’ve got the necessary infrastructure and equipment and facilities, and if required, access to additional expertise throughout the life of the project and so you'll be looking for the potential for this project to produce significant and transformative change in scientific knowledge, practice or policy underpinning the human health issue they're focusing on.

And whether or not the research has the potential for producing intellectual property, publications, policy advice, products, teaching aids, a whole variety of different outputs there depending on the type of research being proposed.

Assessing Synergy (16:08)

Looking at the Synergy Grant, the Synergy criteria. NHMRC defines Synergy as the quality of a diverse team's multidisciplinary and collaborative approach to solve a major health and medical research question, while building workforce capacity and I'd like to just re-emphasize that last bit about building workforce capacity. That's really something that's important to Synergy and quite unique for the scheme.

So this criteria will assess whether the specific research team has the appropriate mix of research skills and collaborative experience to answer the research question. So successful Synergy Grant proposals are outcomes focused and they demonstrate the skills essential to solve the research question and they provide evidence of discernible benefit over just homogenous research teams of similar skill sets.

I think that's really important to look out for one thing to note in relation to the score descriptors for the Synergy criterion is it's essential that all the descriptors relating to a particular score are met across the 3 elements of diversity, multidisciplinary, and collaborative gain, which I'll talk a bit more about now.

When assessing Synergy, does the application demonstrate (17:19)

Looking at the diversity component of Synergy, this is looking at the composition of the team and the diversity in gender, career stage, or researchers from different cultures and whether or not they provide the expertise. They're building capacity, which is aligned to the research question, and whether or not they're bringing vital skills and perspectives to the research team, without which you just couldn't answer this research question.

You're looking for a multi, a truly multidisciplinary approach that will ensure the research is integrated and cohesive and relevant.

Different disciplines are all integrated together. I think that the word integration is important there, and that you're bringing together researchers with complementary expertise, skills and perspectives across different disciplines which will produce transformative change, which wouldn't be possible by just funding those individual CIs to do their individual projects in isolation. It's really more than the sum of their parts when you bring them together.

Collaborative Gain (18:30)

Importantly, the collaborative gain aspect of Synergy is looking for whether the applicant has shown that they've got comprehensive and suitable plans for the team to work synergistically. They've got milestones, evaluation measures, strategies for the exchange of intellectual information or intellectual property.

They've got governance arrangements and grant sharing things in place and importantly, these calibrations need to be sustainable and should extend beyond the life of the project. You're looking at whether each investigator's previous experience and they have success in collaborative research.

So that's not necessarily with the same researchers on the application, but that have shown previous experience taking a collaborative approach in their research and also to look out for strategies which will integrate and provide mentoring, and development opportunities increase the capabilities of underrepresented groups. You know, that could be consumers, community groups, policy makers and I guess to focus on the 2 images down the bottom, there is no magic formula for Synergy. There are some perceptions in the sector that you must have an EMCR, you must have someone from a diverse ethnic background, you must have different genders.

That's not necessarily the case, but that's helpful, but there is no magic formula and it really does depend on the research question, the sorts of people who are going to bring those different perspectives to answer that research question, which is different for every application.

And I guess in this respect, the second image there is just to lookout for any tokenism by any applicants, that they're simply just putting these people on just to think that they can get a better score and synergy. That's not what we're looking for. We're looking for people that are genuinely there to provide different perspectives and bring in different skill sets that are related to the research problem. So a bit more about the shortlisting process.

Shortlisting Process (20:23)

Applications will be subject to a minimum threshold score of 4.801 for both the Knowledge Gain and Synergy criteria. So, applications that fall below that threshold score for either one of those won't be considered further.

And just given the limited amount of funding, not all applications that meet the threshold for both of those criteria will be automatically shortlisted. It'll only be the most competitive applications. And typically, we take through around 20 to 30 applications for Stage Two peer review.

Applicant feedback – Stage One only (20:57)

So, a bit more about the applicant feedback stage. This is something that some of you might not be familiar with, but if you've done Investigator Grants or Ideas Grants, you'll be familiar with providing constructive and qualitative applicant feedback to applicants and so this only happens after the first stage. So, you only provide applicant feedback relating to their Knowledge Gain and the Synergy criteria.

So, the feedback should focus on the strengths and weaknesses of the application, and you've got up to 1000 characters and importantly not 1000 words. You do have to be quite concise with your comments to fit it into that character limit. We do see some comments that are artificially truncated, and we do check for that, so just be mindful of that. The comments and the scores for Stage One will be released to applicants who don't proceed to Stage Two after that short listing process and then the Stage One comments and scores for those unsuccessful applicants will be provided after the Stage Two process.

What we're looking for in good feedback, we're looking at comments that highlight the key elements that influenced and reflected your scoring and so you really need to consider and highlight to the applicant the strengths and weaknesses of the application. You will need to be concise. Importantly, it's to be constructive, to be kind and professional. I think that's really the take-home message there, is to be kind and professional and really to think about if you were receiving this feedback, if it was your application that you'd spent weeks and months putting together and a lot of your time, what would you like to know to improve that?

And just to be mindful, these are people that will be disappointed, and this feedback needs to be constructive and kind. It's important to frame that feedback against the assessment criteria and the score descriptors and provide specific advice or references to you that you think might have been overlooked in their application.

Avoid Comments that (23:11)

I guess there's a whole host of things to avoid, and I won't go through all of these, but needless to say that discourteous, derogatory and unprofessional or emotive language is not recommended. We don't like to see the use of words like disappointingly or unfortunately they're not particularly helpful.

We don't like to see people, we don't like to see comparisons made between applications that you've reviewed as well, or accounting details of the application. You've only got quite a small character limit to get across what were the strengths and weaknesses and not to recount information that the applicant already knows about their application.

There's a whole host there. I'll leave that for you to go through after the briefing.

Applicant feedback sharing (24:20)

A bit more on the applicant feedback sharing, which is new this year. So this will be new for previous and experienced Synergy Grant applicants. So this is really an opportunity for you to review the applicant feedback provided by other peer reviewers of the same applications assigned to you.

Its not designed for you to fine tune your scoring, it's another check on made an errors or a transcription error and their comments don't relate to that application, that does happen occasionally, but also whether the comments are appropriate and professional and it also gives you an opportunity to sort of get into the minds of other peer reviewers, given we don't have that grant review panel process for Synergy Grants.

If you do have concerns during this process during that 2-week period about some of the comments made by your fellow peer reviewers, please contact Lauren and Lindsay as soon as possible, and we'll look at the comments and make a determination and potentially contact peer reviewers to seek some clarification on those comments.

And just remember that we don't change any of the feedback provided by you, that is provided to applicants verbatim. So even down to spelling mistakes and full stops, we don't change anything. So it goes directly to them and I guess just to finish on this point is that it's entirely normal and I think it's to be expected that if you get a group of 5 scientists together and to review their opinions, they're all going to have different views. They're going to focus on different parts of the application. So we actually expect and hope to see different opinions come out in those comments. And that's entirely normal. So you should expect that there will be some different opinions there.

Stage Two (25:51)

Looking at the Stage Two process for those 20 to 30 applications that progress through, this is the assessment of each Chief Investigator's track records relative to opportunity, covering their publication record, an example of research impact, and their leadership achievements.

Generally, we like to provide Stage Two peer reviewers with the same applications that they assessed in Stage One. However, depending on how many of those progress and the number of CIs on those applications, we may need to balance workload so that anyone of you aren't particularly overburdened. For example, if 4 of your 10 applications progress through and they each have 10 CIs, doing 40 track records is a lot of work.

So, we'll reallocate those to someone else, some of those to other peer reviewers to assess and so each member of the CI team for the application will have those 5 reviews.

Assessing Track Record – Publication (26:48)

Looking more about assessing track record and specifically publications, hopefully most of you are familiar that NHMRC now assesses publications by assessing the applicants chosen up to 10 publications from the past 10 years. So that does factor in any career disruptions they may have had.

For example, if they've had a year off for carers' leave, their 10-year time frame will extend to 11-years, and it's good to check whether or not their publications do fit in with their 10-year assessment time frame. So you really need to have a look at whether or not they've had career disruptions and if you do have any concerns, please contact your secretary and we can have a look in more detail about that.

Accompanying the 10 publications is an explanation of why the applicant has included these publications and so moving to this top 10 in 10, as we call it, moves away from a focus on quantity and focuses on the quality of an applicant's track record and their contribution to science. So, you'll continue to use the seven-point scoring system and the score descriptors for assessing publications and I guess one thing to note that some applicants use that explanation field to introduce additional track record information, which is out of scope and so things like conference participation, awards or patents, and if they do include that information, to disregard that from your assessment of their publications.

Use of impact factor and other metrics (28:17)

So just a reminder, many of you will know that NHMRC is a signatory to the San Francisco Declaration on Research Assessment, and they make a whole host of recommendations, one of which is that publication metrics such as journal impact factors shouldn't be part of the assessment of grant applications. So just to be mindful of that, we do see some comments by applicants and peer reviewers that they've considered the impact factor of the journal.

That's not appropriate and that's not part of the assessment but note that applicants may provide field-weighted metrics and other citation metrics will be helpful in determining the publication's importance in their field. So, I guess you need to look at the overall impact, quality, and contribution of that published journal article to that chosen field, and that will differ across different fields and their contribution to the publication itself.

Research Impact (29:15)

Looking at research impact, as I mentioned, the 3 elements of research impact include the reach and significance of their claimed research impact, the contribution of the research program to the research impact, and then the contribution of the applicant to the research program and they're equally weighted about 5% across those 3 elements.

So, like I said, those 3 components are now completed by the applicant in a single block, but you still need to assess those 3 elements individually against the seven-point score descriptors we have in Appendix D.

And just one thing to note here is that for the assessment of the reach and significance, the seven-point scoring system is further divided for applicants that are less than 10-years post-PhD and those that are greater than 10-years post-PhD. Just to recognise that the earlier and mid-career researchers have had less time to accumulate research impact.

Leadership (30:11)

Looking at the leadership criteria. So again, a leadership covers the past 10-years, taking into account any career disruptions across the 4 leadership elements of research mentoring, research policy and professional leadership, institutional leadership and research programs and team leadership. And again, their score descriptors in Appendix D, which you should follow closely to determine what score you'll provide there.

Relative to opportunity and career disruptions (30:31)

Applications are assessed relative to opportunity, again, taking into account career disruptions, and we'll talk a little bit more about that. So relative to opportunity should be taken into account for all applications to ensure that you're assessing it versus their opportunity they've had in their careers.

We acknowledge that it's quite difficult. Applicants have had varying career trajectories, and this is designed to create a little bit more of a level playing field and it does differ across different fields of research. So, to factor that in, it's different at different career stages and different career disruptions affect people's careers differently.

When you're thinking about relative to opportunity considerations, it's things like the amount of time they've spent as an active researcher, the resources available to them, whether or not they've had a clinical, administrative or teaching workload that's affected their research, whether they have a disability or illness, which has contributed to them conducting research.

They may have had to relocate their lab or their clinical practice setting, which was a major disruption for them. They might have restrictions on publications if they've come from industries or had a different focus in their research career.

Hopefully not, but unfortunately, some people have been subject to calamities, bushfires and cyclones in certain parts of Australia, and that could impact their research careers and of course, there's career disruptions, which are a special subset of relative to opportunity considerations, and they are only limited to a period of 90 days or more for pregnancy, major illness or care responsibilities.

Peer Review Mentors (PRMs) (32:13)

We're fortunate to have a couple of really experienced peer review mentors that will be able to help you through the process. So, I think Julie is here today.

Yep, fantastic. Welcome, Julie. It's great to have Julie back on board as a Synergy Grant peer review mentor and we've also got Professor Andrew Hill, who will be on hand this round to provide some valuable expertise.

The peer review mentors can be contacted through Lindsay and Lauren. If you have a question relating to the process, not about the specifics of all the details of the application itself, but more about how to approach the assessment. Julie and Andrew will be able to provide some guidance.

We'll take your query, provide it to Julie and Andrew, and they'll be able to give a really experienced and considered response and this year, like I mentioned at the start, we will have a session during Stage One where you can ask Julie and Andrew, any questions you have. We'll be on hand to answer any policy-related questions, but that's a good way to interact directly with the peer review mentors. So, thanks Julie for being here today, and thanks Andrew for coming on board as a new Synergy PRM.

Peer Reviewer feedback to NHMRC (33:26)

And just a reminder, we value feedback throughout the whole process, but you will have a formal opportunity to provide feedback through our peer reviewer survey after the completion of the whole process. So that's really important for us to make improvements, either policy or process improvements, make your lives better, provide better understanding for applicants and future peer reviewers.

So please engage with that process but at any time, feel free to provide Lauren, Lindsay, or myself feedback about your experience with the Synergy peer review process and I think I forgot to mention at the start whether or not we were joined by Brian, our community observer, not today.

We had Brian Thomas, who's our community observer. He will be on hand throughout the process and we provide him with regular updates on the processes that we've undertaken to make sure that they align with what we said we would do and he also provides a report on how we went at the end of the process, which factors into how we might improve the process and the scheme for next year.

Thankyou.

End of transcript.