Research Assessment as a lever to improve research quality and research culture
A conversation with Karen Stroobants and Noemie Aubert-Bonn about the need for change in the academic research assessment system.
by Karen Stroobants, Noemie Aubert-Bonn, and Jo Havemann
Published onMar 13, 2023
Research Assessment as a lever to improve research quality and research culture
·
In this podcast episode, Karen, Noémie and Jo discuss the need for change in the academic research assessment system. They highlight the importance of considering values and purpose in addition to metrics and publications, and suggest changing the narrative around success in research. They also discuss the need for diversity in hiring and the importance of preparing early career researchers for a range of career paths both within and outside academia as well as the need for a cultural shift and collaboration between sectors to create a more holistic and inclusive research environment.
Karen Stroobants emphasized the importance of changing the narrative around success in academia. She suggested that researchers should ask different questions, such as "what the research is about?" and "who will it be important to?", rather than focusing solely on metrics and publication venues.
Noemie Auburn added that individuals should reflect on their own work habits and question whether they align with their personal values and goals in science. She also emphasized the need to rewrite the unwritten rules of academia and build a different culture that aligns with research integrity and best practices.
Overall, both Karen Stroobants and Noemie Auburn stressed the importance of addressing the systemic challenges in academia and changing the incentive structure to prioritize quality over quantity. They also highlighted the need for individuals to reflect on their own practices and contribute to changing the culture of academia.
Karen, Noémie and Jo talk about Research Assessment, Research Culture, and
Karen Stroobants is a policy adviser and consultant on research policy and strategy. She combines roles as lead policy advisor on research landscape & economy for the Royal Society of Chemistry and as freelance consultant, focusing most recently on co-creating the Agreement on Reforming Research Assessment. She is a Governing Board Member for EuroScience and advises the policy activities of the Marie Curie Alumni Association. Previously, Karen held roles at RAND Europe, and the Royal Society where she led the development of the narrative Resumé for Researchers that since has been introduced by funders. Karen moved to the UK initially as a Marie Sklodowska-Curie post-doctoral Fellow at the University of Cambridge. Prior to this, she obtained her PhD in chemistry at KU Leuven in Belgium.
Bio: Noémie Aubert Bonn is a post-doctoral researcher at Hasselt University, (BE) and Senior Policy Advisor at Research England - UKRI (UK). She is interested in understanding how research assessment and research careers influence research practices and cultures. Her PhD focused on such questions. After her PhD, Noémie took part in the SOPs4RI project together with a research team from Amsterdam University Medical Centres (NL). The project built a toolbox and guidelines to help research institutions and funders foster research integrity and supportive research environments. Noémie’s current work at Hasselt University and Research England - UKRI now touches on diverse areas of research assessment, research integrity, and funding distribution, all aiming to improve research environments, research cultures, and research quality by better supporting researchers and research-enabling staff.
Which researcher – dead or alive – do you find inspiring? KS: Marie Sklodowska-Curie NAB: To me this is a really challenging question… There are way too many inspiring researchers out there for me to pick one. I’ve exchanged with so many incredible experienced researchers who taught me how to listen, read, think, and stay open for change; fascinating early career researchers who pushed boundaries of what I believed was possible; engaging research students who taught me a new way of seeing science and the world; but also extraordinary researchers who left academia and expanded their horizons made genuine impact in society… I can’t name just a few.
Name your (current) favorite song and interpret/group. KS: C'est que du bonheur (Stromae) NAB: Right now I’m really into the early songs of Leonard Cohen, not so surprising as a part Montréalaise!
What is your favorite dish/meal? KS: Impossible to choose... NAB: Indeed very difficult to choose and season-dependent, but Phở is probably the dish I could never refuse…
Action items from the conversation
Change the narrative around success in research
Prepare early career researchers for a range of career paths both within and outside academia
Create a more holistic and inclusive research environment through a cultural shift and collaboration between sectors
Develop more qualitative indicators for research quality
Consider a variety of indicators beyond metrics and publications, such as contributions to knowledge generation, development of people around you, contributions to the scientific community, and contributions to wider society
Redesign scholarly series to incorporate research output in different forms beyond journal articles, such as preprints, conference proceedings, data sets, and non-results shared in a repository
Promote open science and open access
Encourage researchers to question their own work practices and ensure they align with their values and beliefs
Reflect on the impact of research and consider the unintended consequences of evaluation systems
Advocate for diversity, equity, and inclusion in research teams and institutions
Raise awareness and engage in conversations with colleagues about the need for change in research assessment and evaluation systems
Support initiatives and organizations that are working towards more fair and inclusive research practices, such as DORA and Black Minute
Encourage collaboration and teamwork in research, rather than focusing solely on individual performance
Celebrate a wider range of achievements beyond traditional metrics, such as successful collaborations or contributions to interdisciplinary research
Reflect on the importance of work-life balance and the need for researchers to take care of their physical and mental wellbeing
These action items represent a starting point for researchers to engage in the ongoing conversation around changing research assessment and evaluation systems. By collectively working towards a more inclusive and equitable research environment, we can create a culture that values diverse contributions and prioritizes impact on society and the world.
Quotes
"We are not getting the best from scientists at the moment because of the way the system is set up and incentivized because of systematic barriers. Are we measuring productivity or are we measuring quality? We need to be clear that in those instances that is what we're looking at and not conflating the two which I think we are often doing at the moment."
Karen Stroobants
"We realize that quality is not necessarily quantifiable. So we're moving towards more qualitative indicators, towards more indicators that provide more variety."
Noemie Auburn-Bonn
References
Coalition for Advancing Research Assessment, https://coara.eu/ The Coalition brings together organisations who have signed the Agreement and are willing to be members of CoARA to enable working together and learning from each other.
Creative Commons Attribution 4.0 International License (CC-BY 4.0)
