Implementation: Good institutional practice guide

3. Implementing cultural change

As institutions vary in size, maturity, resources and organisational structure, so too does the research culture within and between institutions. Many institutions already have processes and initiatives in place to support the conduct of high-quality research and to continually improve their research culture. This Guide has been designed to allow leaders to take a flexible approach to implementing the suggested activities to promote an open, honest, supportive and respectful institutional research culture.

It will depend upon an institution’s circumstances as to whether leaders choose to implement some or all of the suggested activities.

There are many and varied approaches that leaders can and do take to bring about cultural change. This Guide focuses on one of the available options – the ‘Strategy for culture and behaviour change’.¹ The strategy describes five levels at which actions can be taken in a progressive manner, reflecting the fact that successful implementation of higher levels depends on successful implementation of lower levels (see Figure 1).

1. Provision of basic infrastructure, including tools and skills, makes the desired change ‘possible’.
2. Ensuring that this infrastructure is user-friendly makes it ‘easy’ for the research community to adopt the desired change.
3. Once the desired change is recommended and accepted by (a large part of) the research community, its adoption may be considered ‘normative’.
4. Incentives may be introduced to make adoption of the desired change ‘rewarding’.
5. Implementation of the desired changes is made ‘required’ through the development and adoption of relevant institutional policies.¹

By providing practical guidance in the form of suggested activities for each of the five levels of action, this Guide focuses on incremental, values-aligned changes that can be made within institutions. Embedding these changes is intended to foster a research culture in which researchers will feel supported to conduct high-quality research.

It is important to recognise that cultural change takes time and can be achieved with gradual improvements. It can involve a continuous process, implemented along different time frames for different elements of the research culture. Each element can be gradually and progressively improved through a continuous process of self-assessment, planning, implementing, monitoring and evaluating.

3.1 Examples

Samples of graded implementation activities, based on those suggested in this Guide, are outlined in Table 2 and Table 3. An example of how the values outlined in Section 1 can be given practical expression by implementation of the suggested activities in this Guide is provided in Table 4.

Table 2. Sample: Graded implementation activities – Leadership, supervision and mentorship (see Section 4.1)

Intervention type	Activities: Leadership, supervision and mentorship
Make it possible	Identify training needs in research leadership, supervision and mentorship, to determine where efforts should be focused.
Make it easy	Ensure training in leadership, supervision and mentorship is open to a wide and diverse range of staff and held at times convenient to those with out-of-work responsibilities.
Make it normative	Make the qualities and characteristics of good leaders, supervisors and mentors a regular topic for discussion at meetings (for example, research group meetings, faculty meetings) and in communications from leaders.
Make it rewarding	Include evidence of supervision and mentorship skills as part of promotion and institutional award processes.
Make it required	Include requirements for training in leadership, supervision and mentorship in institutional policies.

Table 3. Sample: Graded implementation activities – Open science

Intervention type	Activities: Open science
Make it possible	Provide infrastructure for supporting open science, such as: repository infrastructure to support open access to research outputs like publications data storage infrastructure to manage, curate and store data and code in accordance with Findable, Accessible, Interoperable and Reusable (FAIR) principles2 tools for transparent record keeping, for example, Electronic Laboratory Notebooks.
Make it easy	Appoint qualified librarians to advise and support researchers across a range of topics including the scholarly information lifecycle, research metrics, open science practices and data management.
Make it normative	Research leaders to provide an example to other researchers by using institutional resources that support open science practices; for example, repository infrastructure, data storage infrastructure and tools for transparent record keeping.
Make it rewarding	Develop incentives for staff to take advantage of centrally provided services; for example, statistical support, repositories for publications, data storage infrastructure, transparent record keeping, communities of practice.
Make it required	Require research data and code to be made FAIR2 and, when relevant, to align with the CARE principles for Indigenous data governance3 requiring an ‘open as possible, closed as necessary’ approach. Mandate the use of tools for transparent record keeping and require these records to remain openly available in accordance with open science practice.4

4. Implementation

This section builds on the values outlined in Section 1. that support an open, honest, supportive and respectful research culture conducive to the conduct of high-quality research. It provides practical guidance in the form of suggested activities for each of the elements outlined in Section 2, to assist leaders to translate and embed the values into cultural norms within an institution and achieve continuous improvement. Sample self-reflection questions can be used by leaders as prompts to assess their institution’s research culture and determine their stage of implementation of cultural change. Case studies based on real-world examples, and hypothetical scenarios, outline examples of how some institutions have achieved positive change in their research culture.

4.1 Role modelling and leadership

4.1.1 Introduction

Traditionally, health and medical researchers are perceived as strong and successful leaders if they oversee a large team of staff, have many PhD students, and have a continuous flow of publications in high impact journals.

This guide seeks to reinforce a different version of what makes a good leader in research: one who promotes a vision for the future that is positive and values-based, and promotes reflection, collegiality and responsible research practices, including open science.

When a leader exemplifies and reinforces the institution’s core values, they help to create a culture that reflects these values and inspire staff to behave and act accordingly. A good research leader, with institutional support as appropriate:

• actively seeks to develop and broaden their team’s talent and skills
• recognises the contributions and achievements of all members of their team, including those from diverse backgrounds and disciplines
• values the team’s diverse outputs, impacts, practices and activities that maximise the quality and rigour of research
• creates a supportive and encouraging environment where everyone can speak freely about the team’s research activities and data, including its strengths and weaknesses
• effectively addresses concerns such as unhealthy competition, publication pressure, detrimental power imbalances and conflicts
• is honest and open about their decisions and mistakes
• practices humility and is open to alternative views
• ensures the equitable and transparent distribution of resources for which they are responsible
• reinforces an environment where ethical and responsible research practices are the norm including through regular discussions about improving research practices; taking effective, swift and positive action when questionable research practices first occur; and supporting those who identify and seek advice about potential questionable research practices
• models self-care and demonstrates that high-quality research does not need to be associated with excessive or burdensome workloads
• facilitates succession planning by supporting the development of leadership skills in research students and EMCRs.

When such attributes and behaviours are reflected in a team’s leadership, the research environment is more likely to foster a culture in which everyone feels supported and appreciated and strives to conduct high-quality research.

Supervision of early career researchers, Higher Degree by Research (HDR) candidates, undergraduate students and other research trainees plays a critical role in the responsible conduct of research and comes with many and varied responsibilities. The supervisory role incorporates oversight of all relevant stages of the research process from conceptualisation and planning through to dissemination of findings and as appropriate, publication and follow-up activities. Leaders should actively promote supervisory best practice, acknowledging and, as appropriate, rewarding genuine excellence in supervision.¹¹ The Australian Code for the Responsible Conduct of Research outlines that institutions have a responsibility to ensure supervisors of research trainees have the appropriate skills, qualifications and resources.¹²

It is the institution’s responsibility to provide ongoing training and education that promotes and supports responsible research conduct for all researchers and those in other relevant roles. This includes assisting researchers to develop their supervisory practice and follow their institution’s policies and other relevant disciplinary-specific policies.

A good supervisor, with institutional support as appropriate:

• serves as a role model to less experienced researchers
• maintains a high degree of professionalism and current knowledge of their field or discipline
• reflects on their own competence to provide advice and to seek objective feedback and support where necessary
• reflects the behaviours of good leadership
• initiates regular discussions about responsible research practices
• facilitates and supports access to ongoing learning opportunities
• creates a respectful research environment where scientific critique is encouraged
• acknowledges the work performed by others and recognises their contributions in a rigorous and fair manner, especially with respect to authorship of publications and on funding applications
• develops their own knowledge and skills in communication with, and management of, staff.

72% of respondents agreed that mentoring programs that address research quality and career development are amongst the most significant institutional interventions to improve research quality.

NHMRC sector survey¹³

A mentorship program can help facilitate a positive research culture for mentees by providing:

• encouragement and a second opinion
• assistance with managing their work pressures
• connections to relevant institutional and external support services and resources
• help to expand their networks of useful contacts
• ideas about what skills and achievements should be valued
• input and feedback on responsible research practices at all stages of the research cycle
• support regarding career opportunities.

4.1.2 Implementation

Implementation of better support systems for researchers (in particular, research students and EMCRs, and researchers experiencing difficulties such as vicarious trauma as a result of working on sensitive topics), encouragement of interdisciplinary collaboration and teamwork, and greater equity and sustainability of research career paths, will require a collaborative effort with institutions helping their staff to become better research leaders, supervisors and mentors. Suggestions for how leaders can help their staff to do this are provided in Table 5.

Table 5. Graded implementation: Role modelling and Leadership

Phase	Suggested activities
Make it possible	Identify training needs in research leadership, supervision and mentorship, to determine where training efforts should be focussed. Support the career development of future generations of leaders by providing research students and EMCRs with opportunities to discuss and explore their own research ideas. Ensure that leaders understand the importance of including diverse perspectives during the creation of knowledge, in order to challenge inequity and address social needs. Consult with staff and students about implementing mentorship programs. Consult with Aboriginal and Torres Strait Islander researchers and students about their needs from their supervisors and mentors, both Indigenous and non-Indigenous. Consult with researchers from underrepresented groups, such as those who identify as culturally and linguistically diverse, neurodiverse or LGBTQIA+, about their needs from their supervisors and mentors. Provide funding, support and resources (for example, administrative support, material resources) for training in leadership, supervision and mentorship. Hire people with appropriate qualifications and expertise, or train existing staff, to provide relevant training in supervision and mentorship. Consult with staff and students to understand what support systems leaders can put in place.
Make it easy	Ensure training in leadership, supervision and mentorship is open to a wide and diverse range of staff and held at times convenient to those with out-of-work responsibilities. Encourage staff to participate in relevant training. Ensure training opportunities are widely publicised. Ensure all non-Indigenous supervisors and mentors receive training in cultural competence and cultural safety. Provide easy access to staff with expertise in responsible research practices (for example, librarians).
Make it normative	Make the qualities and characteristics of good leaders, supervisors and mentors a regular topic for discussion at meetings (for example, research group meetings, faculty meetings) and in communications from leaders. Establish formalised mentoring structures for Aboriginal and Torres Strait Islander students and researchers. Establish communities of practice within/between institutions and facilitate regular meetings/social gatherings for peer support (with a focus on mentoring programs). Establish an Aboriginal and Torres Strait Islander peoples-led researcher network to support early and mid-career researchers and to forge closer connections between students and Aboriginal and Torres Strait Islander academics. Encourage both new and experienced supervisors to reflect on their own supervision practices and seek objective feedback to foster a culture of good supervision. Socialise examples of best practice via communications both internally and externally (for example, newsletter articles). Ensure modelling by leaders through their active support for leadership/supervisor training and mentoring programs and that they undergo relevant training themselves. Provide opportunities for and encourage research students and EMCRs, including Aboriginal and Torres Strait Islander researchers, to take on leadership roles.
Make it rewarding	Establish institutional/faculty/school awards for staff who model exemplary leadership/mentor/supervisory values and behaviours. Recognise supervision and/or mentorship of students in staff workload. Include evidence of supervision and mentorship skills as part of promotion and institutional award processes. Establish a system for recognising staff who demonstrate excellence in the provision of training in supervision and mentorship. Include evidence of supervision and mentorship skills as part of placement/hiring, promotion and institutional award processes.
Make it required	Include requirements for training in leadership, supervision and mentorship in institutional policies. Provide clarity about institutional expectations for leadership, supervision and mentorship in institutional policies, procedures and communications within the institution. Where appropriate, make leadership, supervision and mentorship skills a key performance indicator (KPI) during performance appraisals and promotions. For senior staff, an additional KPI could be the leadership, supervision and mentorship skills of the staff they manage.

4.1.3 Self-reflection questions

The following sample self-reflection questions could be used as prompts for leaders to determine their stage of implementation as outlined in Table 5.

4.1.4 Case studies and scenarios

4.2 Institutional resources

4.2.1 Introduction

An institution's commitment to, and the value it places on, the conduct of high-quality research can be demonstrated by the provision of expert and technical advice, administrative support and material resources for conducting high-quality research, to all relevant staff and students. This, in turn, can reinforce a positive research culture. Activities that facilitate the conduct of high-quality research, such as mentoring, education and training, the provision of rewards and recognition, and interdisciplinary research collaborations, require resource allocation.

4.2.2 Implementation

Leaders should aim to provide sufficient resources to facilitate the conduct of high-quality research and researchers’ use of responsible research practices. Suggestions for the graded implementation of institutional resources to support the conduct of high-quality research are provided in Table 6.

Table 6. Graded implementation: Institutional resources to support the conduct of high-quality research

Phase	Suggested activities
Make it possible	Identify and gather information on responsible research practices. Examine faculty/school and institutional policies relevant to the conduct of responsible research practices and determine where extra resources are needed to put policies into practice. Simplify and harmonise policies to ensure clear and consistent messaging to researchers. Appoint, train and support people to provide advice to all institutional staff on matters relating to research quality. Hire staff with relevant expertise in responsible research practices and/or train existing staff to become experts in responsible research practices. Provide infrastructure for supporting responsible research practices, such as: appropriate library services to provide access to a curated collection of information resources and evidence-based information collections repository infrastructure to support open access to research outputs like publications data storage infrastructure to manage, curate and store data and code in accordance with the FAIR2 principles and the CARE principles for Indigenous data governance3 tools for transparent record keeping, for example, Electronic Laboratory Notebooks. Establish communities of practice (within and between institutions) to discuss responsible research practices and ways to support high-quality research. Establish an Aboriginal and Torres Strait Islander peoples-led researcher network to support early and mid-career researchers and to forge closer connections between students and Aboriginal and Torres Strait Islander academics. Establish Indigenous-led governance of Aboriginal and Torres Strait Islander peoples’ research.
Make it easy	Set aside funding for quality improvement practices. Gather data and evidence on what policies work at your institution. Support centres and standard frameworks for specific types of research (for example, clinical trials, Indigenous research). Support provision of expert advice to institutional committees such as Human Research Ethics Committees and Animal Ethics Committees. Provide easy access to: staff with expertise in responsible research practices (for example, statistical advice; librarians, veterinary advice for animal research, legal and financial teams) for provision of advice at all stages of the research cycle infrastructure for supporting responsible research practices independent peer review (internal or external) of research plans and publications information about responsible research practices (for example, website, intranet, internal communications, training). Support and facilitate opportunities for collaboration across the institution and between disciplines to support responsible research practices (for example, through a community of practice). Support Aboriginal and Torres Strait Islander students and researchers to form peer networks.
Make it normative	Leaders to provide an example to other researchers by: using institutional resources that support responsible research practices promoting the use of statistical support and research collaborations, including interdisciplinary collaboration supporting communities of practice (within and between institutions) for discussing responsible research practices. Include the institutional resources that are available to support the research being conducted as a regular item for discussion during research group and faculty/school meetings. Provide regular information about institutional resources for supporting responsible research practices in internal communications. Leaders to maintain awareness of the latest research on research quality. Support and encourage staff to attend, and present to, conferences on research quality and be open to hosting conferences and meetings on research quality. Continue to support peer networks of Aboriginal and Torres Strait Islander students and researchers who come together to produce peer generative power.
Make it rewarding	Include expertise in responsible research practices as a criterion for staff performance review. Recognise the time contributed by staff who provide support/training for, and advice about, the conduct of high-quality research and responsible research practices. Recognise staff who give up their time to serve on committees. Develop incentives for staff to take advantage of centrally provided services for example, statistical support, repositories for publications, data storage infrastructure, transparent record keeping, communities of practice. Appropriately acknowledge staff who provide their expertise and ensure that they are included as authors where appropriate. Formally recognise Aboriginal and Torres Strait Islander students and researchers who use peer generative power to produce outputs, processes and actions, which improve the quality of research practices.
Make it required	Ensure policies and procedures for resource allocation within the institution include the conduct of high-quality research. Require researchers with projects that involve statistical analysis to seek statistical advice at key points in the project.16 Require management and stewardship of research data and code to accord with the FAIR principles2 and, when relevant, the CARE principles for Indigenous data governance3 requiring an ‘open as possible closed as necessary’ approach. Mandate the use of tools for transparent record keeping and require records to remain openly available in accordance with open science practice.4

4.2.3 Self-reflection questions

The following sample self-reflection questions could be used as prompts for leaders to determine their stage of implementation as outlined in Table 6.

4.2.4 Case studies and scenarios

4.3 Education and training about responsible research practices

4.3.1 Introduction

Researchers engage in ongoing development of their knowledge and skills throughout their careers. On the job training has traditionally been the pathway by which research students and researchers gain proficiency in research skills. Although many Australian institutions do provide training for their researchers about responsible research practices, it is internationally recognised that there is a need for greater consistency in the content and delivery of education and training programs for researchers and assessment of knowledge and skills attained.

72% of respondents indicated that provision of education, training and supervision was a key feature of the research environment that encouraged the production of high-quality research.

NHMRC sector survey¹³

Effective and continuing education and training about responsible research practices equips researchers with the necessary knowledge and skills to conduct high-quality research. It also ensures that researchers have a common understanding about the requirements and expectations for the conduct of research throughout their research career.

4.3.2 Implementation

Key considerations when developing or reviewing education and training programs for researchers include (but are not limited to):

• regular assessment and evaluation of the education and training needs of members of the institutional research community
• recognition that participation in education and training does not necessarily equate to attainment of knowledge or skills
• mechanisms for assessment of knowledge and skills attained by a suitably qualified assessor
• that the programs need to be accessible, suitable, flexible, practical, engaging, relevant and implementable
• how to meet the needs of individual researchers; for example, an experienced researcher (and recognition of prior learning) versus less experienced researcher/student
• the variety of routes to education and training in addition to formal lectures/tutorials and the traditional ‘master-apprentice’ model used for PhD training, with the attainment of some skills better suited to education via supervision
• the variety in the timing of delivery so that necessary knowledge and skills are maintained, and new ones are attained as required, during a researcher’s career
• provision of adequate support, resources and promotion of relevant programs
• regular assessment of the outcomes of the education programs with appropriate modification when required.

Suggestions for the graded implementation of education and training of researchers about responsible research practices are provided in Table 7.

Table 7. Graded implementation: Education and training of researchers

Phase	Suggested activities
Make it possible	Assess education and training needs of students and researchers to identify where to focus efforts.22 Provide support and resources (for example, expert, technical, and administrative support, infrastructure, material resources, list of recommended training/tools) for all modes of delivery. Hire people with appropriate qualifications and expertise, or train existing staff, to provide relevant education and training. Establish a staff member whose responsibilities include the administration and coordination of the education and training of researchers. Ensure leaders actively support education and training programs and provide researchers with sufficient time to attend relevant education and training.
Make it easy	Provide clear information about processes for obtaining required knowledge and skills. Provide variety in modes and timing of delivery so that education and training is accessible, suitable, flexible, practical, engaging, relevant and implementable and meets the individual’s needs (for example, experienced researcher versus less experienced researcher/students, cohort-based learning for Aboriginal and Torres Strait Islander students).23,24 Modes include classroom/lecture-based learning, tutorial, practical, online, mentorship, simulated environment, theatrical, gamification and quizzes. Timing includes undergraduate, postgraduate, and early career researcher stages; upon recruitment; regular refresher; ‘advanced’ programs for senior researchers; ongoing during conduct of particular activities. Provide a process for recognition of prior learning (for example, for experienced researchers prior to being required to undertake formal education and training). Provide easy access to support and resources for training (expert, technical and administrative support, as well as material resources). Support implementation of new knowledge and skills in the workplace. Evaluate the effectiveness of the education and training programs and modify as required.
Make it normative	Make education and training related to responsible research practices a regular/standing topic for discussion at research group meetings and in communications from leaders. Remind staff and students of the institution’s expectations about the attainment of knowledge and skills, and relevant legislation and codes of conduct, that apply to their work (for example, at research group meetings, internal communications). Include education and training about responsible research practices in standard human resources and research higher degree processes for staff and PhD students. This includes any ‘welcome’ packs or onboarding for new staff and students. Establish communities of practice within/between institutions and facilitate regular meetings/social gatherings for peer support. Establish an Aboriginal and Torres Strait Islander peoples’-led researcher network to support early and mid-career researchers and to forge closer connections between students and Aboriginal and Torres Strait Islander academics. Ensure leaders attain relevant knowledge and skills themselves.
Make it rewarding	Include relevant education, training and attainment of relevant knowledge and skills in staff workload and promotion criteria. Establish a system for recognition within research groups of knowledge skills that have been attained by group members. Create incentives to reward research groups that have high engagement rates in education and training. Establish institutional awards for excellence in provision of education and training.
Make it required	Include the requirements for and attainment of relevant knowledge and skills in applications for ethics approval for human and animal research (for example, specific training in animal research procedures). Include requirements about education and training and attainment of relevant knowledge and skills in institutional policies. Provide clarity about institutional expectations in policies, procedures and communications. Require compulsory education and training and attainment of knowledge and skills about responsible research practices for all research career stages. Make relevant education and training and attainment of knowledge and skills a KPI during performance appraisals and promotions; for senior staff an additional KPI could be the education and training of the staff they manage. Develop systems for regular audit of knowledge and skills attained. Verify whether the education and training programs provided are helping staff attain core knowledge and skills for the conduct of high-quality research.

4.3.3 Self-reflection questions

The following sample self-reflection questions could be used as prompts for leaders to determine their stage of implementation as outlined in Table 7.

4.3.4 Case studies and scenarios

4.4 Rewards and recognition

4.4.1 Introduction

There is a clear and growing international consensus for the need to reform researcher assessment practices to further support the quality of research and the attractiveness of research environments.^{27,28,29,30,31} The current assessment of researchers usually relies on a narrow set of quantitative journal and publication-based metrics such as the journal impact factor, article influence score and h-index as proxies for quality and impact. These assessment processes focus strongly on past performance thereby missing the opportunity to consider a researcher’s potential, promote quantity and speed at the expense of quality and rigour and promote individualism over collaboration. Furthermore, there is mounting evidence to show that assessment processes that rely on publication and journal-based metrics are prone to multiple biases and discrimination.^26,28

44% of respondents felt that the features that had the most negative effect, and hence discouraged the production of high-quality research, were emphasis on publishing in top-tier journals and how researchers are assessed for promotion.

NHMRC sector survey¹³

A positive research culture is supported by assessment practices that recognise collaboration, openness, engagement with society, and the diversity of activities, outputs and outcomes that maximise the quality and impact of research, while providing opportunities for diverse talents.²⁷

By rewarding and recognising activities and behaviours that support such a culture, leaders can play an important role in encouraging and reinforcing those activities and behaviours, which ultimately contribute to the conduct of high-quality research.

Assessment practices that contribute to a positive research culture also include the recognition of researchers’ workloads that are beyond their formal roles or employment arrangements such as membership of committees, contributions to broader institutional and community activities and organising conferences. In addition, they include the recognition of diverse perspectives from underrepresented groups such as Aboriginal and Torres Strait Islander researchers, people with disability and LGBTQIA+ researchers in academia, all of whom suffer from lack of representation in senior leadership positions.³² Without role models at higher levels, researchers belonging to minority groups may feel uncertain about promotion criteria and are less inclined to pursue career advancement. Marginalised individuals face the added burden of having to advocate for their rights, assert their place in research, often struggle to fit into the traditional academic mould, and are often called upon to advocate for their community.³⁰ Aboriginal and Torres Strait Islander researchers are frequently asked to take on workloads beyond their formal roles or employment arrangements, for example, contribute to specific events institution-wide, connect Aboriginal and Torres Strait Islander peoples’ and communities with institutions, become committee members, run cultural awareness training for staff and students, be a representative at events (for example, NAIDOC event), revise and review Aboriginal employment plans and Reconciliation Action Plans. All of these activities are time consuming and are rarely appropriately remunerated or recognised in assessment metrics or as part of recruitment approaches.³²

EMCRs and diversity groups have indicated a need for enhanced transparency in how institutions incorporate research assessment into promotion criteria, recruitment processes and career progression.³⁰

Promotion committees with a diverse makeup are more likely to respond with empathy and understanding towards marginalised individuals. The creation of specific career pathways and clear guidelines can support people with diverse needs seeking to progress their careers.²⁷

4.4.2 Implementation

Rewarding and recognising positive contributions to the institutional research culture and the use of responsible research practices can be achieved through formal processes such as appointments, promotions and awards, and through informal processes such as peer recognition. Suggested activities to achieve gradual improvements in these areas are highlighted in Table 8.

Table 8. Graded implementation: Rewards and recognition

Phase	Suggested activities
Make it possible	In consultation with the institutional research community, assess processes and criteria for the following, and identify areas for improvement: rewards and recognition for responsible research practices hiring and promotion. Examine what behaviours are promoted in the institution, and whether these behaviours contribute to a positive research culture and high-quality research. Tools such as the S.P.A.C.E. rubric33 and the Hong Kong Principles29 may assist leaders to identify where they might focus activities for improvement. Seek regular input from staff and students about criteria for assessment of researchers for hiring and promotion, including criteria that make allowances for career disruption and recognise achievement relative to opportunity.
Make it easy	Provide easily accessible information on processes for assessment of researchers. Provide advice about activities that do not support good institutional research culture, such as journal impact factors, number of publications, and awards whose criteria are based on the quantity instead of the quality of research. Develop clear accessible guidance: about assessment of researchers for hiring and promotion about membership of appointment and promotion committees, including a diversity of membership for staff involved in recruitment and promotion decisions that explicitly cautions against the inappropriate use of publication metrics and encourages them to value a full and diverse range of research outputs and contributions28 to support people with diverse needs seeking to progress their career. Provide support to staff involved with hiring and promotion; for example, providing examples of questions that can be asked in interviews that focus on responsible research practices.
Make it normative	Progressively implement criteria relevant to research quality in hiring and promotion guidance and procedures. Inform researchers about such changes and ensure they understand that criteria relevant to research quality will be considered by appointment and promotion committees. Include information in staff inductions about a positive research culture and responsible research practices and how these are relevant to performance and promotion criteria. Regularly recognise responsible research practices at research group meetings, for example: positive role modelling and good leadership behaviours by leaders responsible research practices and behaviours by researchers.
Make it rewarding	Provide public acknowledgement (for example, at faculty/school meetings) of a research group or an administrative or support unit who are championing a positive research culture and high-quality research. Provide development opportunities (for example, opportunity to work on a project that provides ‘stretch’ goals, attendance at education and training courses, shadowing of a senior staff or group member). Provide research group, faculty/school and/or institutional excellence awards for: mentoring, research training and supervision with award criteria based on the quality of mentoring/training/supervision rather than metrics such as the number of students supervised researchers who demonstrate behaviours that foster high-quality research such as promoting the use of responsible research practices and collaborative and interdisciplinary research34,35,36 administrative and support units (for example, library, information technology, statistical support, Research Office) that promote and/or support responsible research practices. When hiring, reviewing and promoting researchers, recognise those who demonstrate and implement responsible research practices. Recognise and reward individuals for their contributions to the research environment beyond their formal roles. Recognise and reward Aboriginal and Torres Strait Islander researchers for their contributions to the broader institutional culture.
Make it required	Commit to (and formally sign where appropriate) internationally recognised principles/declarations that promote responsible research assessment (for example, DORA37, CoARA38, the Leiden Manifesto39, the Hong Kong Principles29). Provide information about the key steps being taken by leaders to implement the principles. Include assessment criteria relevant to research quality in formal institutional policies for hiring and promotion. Where appropriate, include responsible research practices and contributions to a positive research culture as KPIs during performance appraisals and promotions. Make public statements on rewards and recognition (for example, those produced by the United Kingdom Reproducibility Network40,41). Prohibit the use of language in job advertisements that refers directly or indirectly to specific journals as a proxy for quality. Ensure rewards do not have criteria based on simple metrics such as impact factors.

4.4.3 Self-reflection questions

The following sample self-reflection questions could be used as prompts for leaders to determine their stage of implementation as outlined in Table 8.

4.4.4 Case studies and scenarios

4.5 Communication

4.5.1 Introduction

Transparent and regular communication with all staff and students about institutional policies and procedures is integral to conveying the institution’s expectations and shaping research culture. This can be achieved within the institution through formal channels with staff and students, as well as informally within research groups, faculties/schools and through peer networks. Making the institution’s policies and procedures publicly available, wherever possible, contributes to sharing of information within the sector. While awareness raising will not bring about cultural change on its own, when staff are provided with the guidance and support to implement policies and procedures, it makes an important contribution.

4.5.2 Implementation

Suggestions for how leaders can enhance communication about relevant institutional policies and procedures both internally and externally are provided in Table 9. It does not include suggestions for how researchers could better communicate with each other or communicate about their research activities.

Table 9. Communication about institutional policies and procedures relevant to research culture

Phase	Suggested activities
Make it possible	Review relevant institutional policies and procedures to ensure that they align with a research culture that is open, honest, supportive and respectful.
Make it easy	Encourage communication between research groups/disciplines/institutions to facilitate exchange of ideas and information about research culture and high-quality research including for interdisciplinary and collaborative research. Provide clear, easily accessible detailed information about responsible research practices. Provide clear, easily accessible information about relevant institutional activities and programs, for example: staff opportunities related to education and training about responsible research practices (see Section 4.3), mentoring (see Section 4.1) institutional resources to support the conduct of high-quality research (see Section 4.2) criteria for assessment, appointment and promotion of researchers (see Section 4.4) awards for excellence in research quality (development, design, methodology, conduct and analysis of research) (see Section 4.4) role descriptions and contact details for people who have responsibility for aspects of research culture and research quality.
Make it normative	Include research culture and responsible research practices as a regular item for discussion during research group and faculty/school meetings. Establish communities of practice, within and between institutions, to facilitate discussion of research culture and responsible research practices. Provide easily accessible information about the outcomes of institutional monitoring and evaluation of activities to foster a positive research culture (see Section 4.6). Ensure information is publicly available about the institution’s policies and procedures about research culture and responsible research practices.
Make it rewarding	Recognise staff or students who give seminars about following and improving research practices, including for interdisciplinary and collaborative research. Recognise staff who develop successful initiatives for communicating about research culture and responsible research practices within the institution. Establish institutional awards for excellence in communication about research culture and responsible research practices.
Make it required	Require information provided for recruitment, promotion and induction of staff to include clear communication about the institution’s policies and procedures that support a positive research culture and responsible research practices. Commit to (and formally sign where appropriate) internationally recognised principles/declarations that promote responsible research practices, (for example, DORA37, Leiden Manifesto39, Hong Kong Principles29) and provide information about the key steps being taken by leaders to embed the principles.

4.5.3 Self-reflection questions

The following sample self-reflection questions could be used as prompts for leaders to determine their stage of implementation as outlined in Table 9.

4.5.4 Case studies and scenarios

4.6 Monitoring, evaluation and reporting

4.6.1 Introduction

Monitoring, evaluation and reporting by institutions about their progress in implementing the suggested activities in this Guide will allow them to identify strengths and weaknesses, areas for improvement and potential issues; to track progress; and to measure positive changes.

Cultural change may be slow. Consequently, monitoring, evaluation and reporting efforts need to be planned for and supported in the long term. This requires an enduring institutional commitment to both cultural change and evaluation.

Many institutions already have processes and initiatives in place to support the conduct of high-quality research and continually improve research culture. As approaches, policies and processes may vary between institutions, this Guide allows for flexibility in its application. It is therefore expected that there will be variation in the way that each institution chooses to monitor, evaluate and report on their progress.

For those institutions that lack a unit focused on evaluation, the ‘Resources’ section (Section 6.2.9) includes information about useful resources and toolkits provided by the Commonwealth and state governments and the Global Evaluation Initiative.

4.6.2 Implementation

The values and elements outlined in this Guide provide a structure for a monitoring framework. That is, the monitoring framework could capture data relating to the key values (Table 10) or data relating to each of the five elements identified as contributing to an institution’s research culture (Table 11).

Suggestions for how institutions can achieve gradual improvements in the monitoring, evaluation and reporting about their progress in implementing the suggested activities in this Guide are highlighted in Table 12.

Table 12. Monitoring, evaluation and reporting

Phase	Suggested activities
Make it possible	Choose the evaluator/evaluation team. Identify a person/group to be responsible for making recommendations about the allocation of resources for monitoring and evaluation, for receiving reports of the outcomes of evaluation, directing the implementation of the monitoring and evaluation framework, and making the required changes based on the outcomes of evaluation activities. Before developing a monitoring framework (see Table 10 and Table 11 for examples), consider the following factors: What types of measurement are feasible within the institution? What types of data are already being collected (directly, or available via a proxy measurement)? What would progress look like with respect to each measurement (for example is progress demonstrated by ‘more is better’ or are there minimum threshold values that must be exceeded)? How frequently will data collection and reporting within the framework take place? Does improving performance against some measurements have priority for the institution? What institutional resources are available to be used to improve performance and how will reporting against the framework be linked to the allocation of these resources? Develop a formal Terms of Reference document that outlines the requirements for the evaluation.
Make it easy	Develop the monitoring and evaluation framework. Develop an evaluation plan covering: purpose of the evaluation key questions the evaluation is seeking to answer sources of data that will help answer these questions tasks, resourcing and timeframes.49 Allocate resources for monitoring and evaluation. Identify and develop a process for receiving reports of the outcomes of evaluation.
Make it normative	Develop a work plan that sets out the specific activities to implement evaluation, including identification of the reporting requirements and a dissemination plan. Implement the process for receiving reports of the outcomes of evaluation. Make baseline measurements of key aspects of culture according to the chosen framework. Clarify and document the institution’s long-term commitments to conducting monitoring, evaluation and reporting. Develop a communication plan to help ensure findings are shared in the most appropriate way with staff and students.
Make it rewarding	Recognise and reward the person/team involved in making the required changes based on the outcomes of evaluation activities.
Make it required	Implement the selected monitoring and evaluation framework. Implement the required changes based on the outcomes of evaluation activities. Commit to the long-term conduct of monitoring, evaluation and reporting on key aspects of research culture. Ensure transparency of evaluation findings and their release within the institution.

4.6.3 Self-reflection questions

The following sample self-reflection questions could be used as prompts for leaders to determine their stage of implementation as outlined in Table 12.

4.6.4 Case studies and scenarios

¹ Nosek B. Strategy for culture change. June 2019. Center for Open Science. Accessed on 14 November 2024 from: https://www.cos.io/blog/strategy-for-culture-change

² FAIR Principles. GO FAIR [Internet]. Accessed 22 Nov 2023 from: https://www.go-fair.org/fair-principles/

³ CARE Principles for Indigenous Data Governance. GIDA: Global Indigenous Data Alliance. [Internet] Accessed 21 Nov 2023 from: https://www.gida-global.org/care

⁴ UNESCO Recommendation on Open Science [updated 21 Sep 2023] [Internet] Accessed 21 Nov 2023 from https://en.unesco.org/science-sustainable-future/open-science/recommendation

⁵ NHMRC Workshop report: Strengthening and growing capacity and capability of Aboriginal and Torres Strait Islander health researchers, Melbourne University Business School, 16-17th May 2018 [Internet] Available from: https://www.nhmrc.gov.au/health-advice/aboriginal-and-torres-strait-islander-health/capacity-and-capability-building-and-strengthening

⁶ National Health and Medical Research Council. NHMRC Innovate: Reconciliation Action Plan II April 2022-April 2024. [Internet] Commonwealth of Australia, Canberra. Accessed 13 Dec 2024 from https://www.nhmrc.gov.au/about-us/publications/reconciliation-action-plan-2022-24

⁷ Reconciliation Australia. Reconciliation Action Plans. [Internet] Accessed 5 Mar 2025 from: https://www.reconciliation.org.au/

⁸ University of Glasgow. Research and Innovation Services: research culture. [Internet]. Accessed 21 Nov 2023 from: https://www.gla.ac.uk/myglasgow/ris/researchculture/

⁹ University of Glasgow. Institutional strategic priorities for research culture 2020-2025. [Internet]. Accessed 21 Nov 2023 from: https://www.gla.ac.uk/media/Media_705595_smxx.pdf

¹⁰ Stanford Medicine: Stanford Program on Research Rigor & Reproducibility. [Internet]. Accessed 21 Nov 2023 from: https://med.stanford.edu/sporr/about/missionstrategyvalue.html

¹¹ Supervision: A guide supporting the Australian Code for the Responsible Conduct of Research. National Health and Medical Research Council, Australian Research Council and Universities Australia. Commonwealth of Australia, Canberra. Published 2019, Publisher NHMRC. [Internet] Accessed 21 Nov 2023 from: https://www.nhmrc.gov.au/about-us/publications/australian-code-responsible-conduct-research-2018

¹² Australian Code for the Responsible Conduct of Research, 2018 Guide to Managing and Investigating Potential Breaches of the Australian Code for the Responsible Conduct of Research, 2018, and NHMRC policy on misconduct related to NHMRC funding, 2016. [Internet] Accessed 21 Nov 2023 from: https://www.nhmrc.gov.au/research-policy/research-integrity

¹³ NHMRC: Survey of research culture in Australian NHMRC-funded institutions. 2020. Accessed 21 Nov 2023 from https://www.nhmrc.gov.au/research-policy/research-quality

¹⁴ Ryan T, Ewen S, Platania-Phung C. It wasn’t just the academic stuff, it was life stuff: the significance of peers in strengthening the Indigenous health researcher workforce. The Australian Journal of Indigenous Education. 2020; 49(2):135-144 [Internet] https://doi.org/10.1017/jie.2020.14

¹⁵ University of Cambridge. Data Champions. [Internet] Accessed 21 Nov 2023 from https://www.data.cam.ac.uk/intro-data-champions

¹⁶ American Statistical Association. Overview of statistics as a scientific discipline and practical implications for the evaluation of faculty excellence. A position paper of the American Statistical Association. 5 Jan. 2018 Accessed on 16 Dec 2024 from: https://www.amstat.org/asa/files/pdfs/POL-Statistics-as-a-Scientific-Discipline.pdf

¹⁷ Bruckner T. Tips and tricks for improving clinical trial reporting from Karolinska university. 2023 TranspariMED. [Internet] Accessed 21 Nov 2023 from: https://www.transparimed.org/single-post/karolinska-university

¹⁸ Possmark S, Martinsson Bjorkdahl C. Responsible reporting of clinical trials: Reporting at Karolinska Institutet is well underway. Lakartidningen [Internet] 2023;120: 23007. Accessed 21 Nov 2023 from: https://lakartidningen.se/klinik-och-vetenskap-1/artiklar-1/temaartikel/2023/05/ansvarsfull-rapportering-av-kliniska-lakemedelsprovningar/

¹⁹ UK Reproducibility Network, Terms of reference, [Internet] 2023 Apr. Accessed 5 October 2023 from: https://www.ukrn.org/terms-of-reference/

²⁰ Committee UKRNS. From grassroots to global: A blueprint for building a reproducibility network. PLOS Biology [Internet] 2021;19(11): e3001461. https://doi.org/10.1371/journal.pbio.3001461

²¹ Australian Reproducibility Network. [Internet] Accessed 21 Nov 2023 from: https://www.aus-rn.org/

²² Héroux ME, Butler AA, Cashin AG, McCaughey EJ, Affleck A, et al. Quality Output Checklist and Content Assessment (QuOCCA): a new tool for assessing research quality and reproducibility. BMJ Open [Internet] 022;12:e060976. https://doi.org/10.1136/bmjopen-2022-060976

²³ Kuei-Chiu Chen, Hester L A dramatized method for teaching undergraduate students responsible research conduct, Accountability in Research [Internet] 2023;30:3, 176-198, https://doi.org/10.1080/08989621.2021.1981871

²⁴ Erasmus University Rotterdam. Dilemma game. Professionalism and Integrity in Research. [Internet]. Accessed 21 Nov 2023 from: https://www.eur.nl/en/about-eur/policy-and-regulations/integrity/research-integrity/dilemma-game

²⁵ Weeks KL, Henstridge DC, Salim A, Shaw JE, Marwick TH, McMullen JR. CORP: Practical tools for improving experimental design and reporting of laboratory studies of cardiovascular physiology and metabolism. American Journal of Physiology-Heart and Circulatory Physiology [Internet] 2019;317:3, H627-H639. https://doi.org/10.1152/ajpheart.00327.2019

²⁶ ARRIVE guidelines. [Internet] Accessed on 19 Dec 2024 from: https://arriveguidelines.org/

²⁷ Australian Council of Learned Academies (2023), Research Assessment in Australia: Evidence for Modernisation. A report to the Office of the Chief Scientist, Australian Government, Canberra. Accessed 13 Nov 2024 from: https://acola.org/research-assessment/

²⁸ Science Europe. Position Statement and Recommendations on Research Assessment Processes. 2020. [Internet]. Accessed 21 Nov 2023 from: https://www.scienceeurope.org/our-resources/position-statement-research-assessment-processes/

²⁹ Moher D, Bouter L, Kleinert S, Glasziou P, Sham MH, et al. The Hong Kong Principles for assessing researchers: Fostering research integrity. PLOS Biology [Internet]. 2020;18(7): e3000737. https://doi.org/10.1371/journal.pbio.3000737

³⁰ Wellcome Trust. Guidance for research organisations on how to implement responsible and fair approaches for research assessment [Internet]. Accessed 22 Nov 2023 from https://wellcome.org/grant-funding/guidance/open-access-guidance/research-organisations-how-implement-responsible-and-fair-approaches-research

³¹ Edwards MA, Siddhartha R. Academic Research in the 21st Century: Maintaining scientific integrity in a climate of perverse incentives and hypercompetition. Environmental Engineering Science [Internet]. 2017 Jan;51-61. Accessed 22 Nov 2023 from: http://doi.org/10.1089/ees.2016.0223

³² Diversity Council Australia. Aboriginal and Torres Strait Islander Peoples- Leading Practice: Understand the 10 truths to centre Indigenous Australians’ voices to create workplace inclusion. [Internet] Accessed 7 Mar 2025 from: https://www.dca.org.au/resources/aboriginal-and-torres-strait-islander-peoples/aboriginal-and-torres-strait-islander-peoples-leading-practice

³³ Schmidt R, Curry S, Hatch A. Research Culture: Creating SPACE to evolve academic assessment eLife [Internet] 2021 Sep 23. 10:e70929 https://doi.org/10.7554/eLife.70929

³⁴ The Einstein Foundation Berlin. The Einstein Foundation Award for Promoting Quality in Research. [Internet]. Accessed 22 Nov 2023 from: https://www.einsteinfoundation.de/en/award/

³⁵ Berlin Institute of Health. The QUEST 1,000 Euro NULL Results Award.[Internet]. Accessed 22 Nov 2023 from: https://www.bihealth.org/en/translation/innovation-enabler/quest-center/calls-and-awards/quest-calls-and-awards/null-and-replication

³⁶ Berlin Institute of Health. The QUEST 1,000 Euro Preregistration Award.[Internet]. Accessed 22 Nov 2023 from: https://www.bihealth.org/en/translation/innovation-enabler/quest-center/calls-and-awards/quest-calls-and-awards/preregistration

³⁷ DORA: The Declaration on Research Assessment [Internet]. Accessed 22 Nov 2023 from: https://sfdora.org/

³⁸ CoARA: Coalition for Advancing Research Assessment [Internet]. Accessed 22 Nov 2023 from: https://coara.eu/

³⁹ Hicks D, Wouters P, Waltman L, et al. Bibliometrics: The Leiden Manifesto for research metrics. Nature [Internet] 2015; 520:429–431. Accessed 22 Nov 2023 from: https://doi.org/10.1038/520429a

⁴⁰ United Kingdom Reproducibility Network Supervisory Board, 2021. UKRN Statement on Responsible Research Evaluation. OSF Preprints. https://doi.org/10.31219/osf.io/4pqwv

⁴¹ United Kingdom Reproducibility Network, 2021. UKRN Statement on Rewards and Incentives for Open Research. OSF Preprints. https://doi.org/10.31219/osf.io/v5jrm

⁴² Strech D, Weissgerber T, Dirnagl U, on behalf of QUEST Group (2020) Improving the trustworthiness, usefulness, and ethics of biomedical research through an innovative and comprehensive institutional initiative. PLOS Biology [Internet]. 2020 Feb 11;18(2): e3000576. https://doi.org/10.1371/journal.pbio.3000576

⁴³ Benedictus R, Miedema F, Ferguson M. Fewer numbers, better science. Nature [Internet] 2016; 538, 453–455. https://doi.org/10.1038/538453a

⁴⁴ ANZCCART. Openness Agreement on Animal Research and Teaching in Australia. [Internet]. Accessed 22 Nov 2023 from: https://anzccart.adelaide.edu.au/openness-agreement

⁴⁵ Lear AJ, Hobson H. Concordat on Openness on Animal Research in the UK. Annual Report 2022 [Internet]. Accessed 22 Nov 2023 from: https://concordatopenness.org.uk/resources

⁴⁶ SOPs4RI consortium. How to create and implement a research integrity promotion plan (RIPP). A guideline (ver. 2.0) [Internet]. Accessed 22 Nov 2023 from: https://sops4ri.eu/wp-content/uploads/Implementation-Guideline_FINAL.pdf

⁴⁷ SOPs4RI consortium. Toolbox for research integrity. V5.0 (CC BY 4.0) [Internet]. Accessed 22 Nov 2023 from https://sops4ri.eu/toolbox/

⁴⁸ Nosek BA. Improving my lab, my science with the open science framework. Association for Psychological Science [Internet]. 2014 Feb 28. Accessed 22 Nov 2023 from: https://www.psychologicalscience.org/observer/improving-my-lab-my-science-with-the-open-science-framework

⁴⁹ NSW Government website: Evaluation resource hub. Evaluation design and planning. Accessed on 17 Dec 2024 from: https://education.nsw.gov.au/teaching-and-learning/professional-learning/pl-resources/evaluation-resource-hub/evaluation-design-and-planning

⁵⁰ McGee R. Evaluation in RCR training- are you achieving what you hope for? Journal of Microbiology & Biology Education [Internet]. 2014 Dec 15;15(2). Accessed 22 Nov 2023 from: https://journals.asm.org/doi/10.1128/jmbe.v15i2.853

⁵¹ Stanford School of Medicine. Stanford Program on Research Rigor and Reproducibility. Focus group and Stanford-wide survey on data sharing and management practices. [Internet] Accessed on 20 Nov 2024 from: https://med.stanford.edu/sporr/monitoring.html?tab=proxy#focus-groups-and-stanford-wide-survey-on-data-sharing-and-management-practices