News | editors
April 24, 2023 | Diversity in evaluation committees enhances the quality of research funding. Better assessment criteria emerge in panels with scientists and non-scientists. This is the conclusion of research carried out by the University of Leiden into operations funded by ZonMw and the Heart Foundation. The importance of broad participation coincides with a workshop on diversity at the recent Appreciation Festival.
Diversity among committee members
In science it is Peer review (peer rated) gold standard. This method is not only applied to posts, but also to search applications. There are two stages here. In the first stage, the committee members read the application booklets, after which they discuss their evaluation as a committee in the second stage. However, this process is not standardized, which means that the individual inputs of the panelists make a significant difference to the outcome.
Researchers from Leiden University We have now analyzed the arguments used by the different committee members. This research was carried out within the framework of requests for funding from medical funder ZonMw and the Dutch Heart Foundation. There were four criteria: scholarly quality, social adequacy, the role of the applicant and the role of the committee. A total of ten sessions were studied in eight research programmes, which handled dozens of applications simultaneously.
Non-scientists improve research evaluation
In total, the researchers recorded more than a thousand discussions, an average of about 126 per meeting or 15 per request. Remarkably, most of the committee members were scientists and that scientific arguments were therefore the most popular. Research feasibility was particularly important in this regard. Scientists also give arguments more often than non-scientists. The social significance is discussed a little lower, while the roles of the applicant and the committee are discussed only with difficulty.
The newsletter is available exclusively to employees of our partners.
In contrast, non-scientists such as healthcare professionals, patients, and policy makers have often discussed the social significance of research. Interest in the apps effect came strongly from patients, who provided about 10 percent of the arguments. Policymakers, who made only about 5 percent of the arguments, were knowledgeable in both areas. In general, the arguments of the non-academic committee members were more varied than those of the scholars.
The differences between the public organization ZonMw and the private Hartstichting were also evident. For example, ZonMw selects non-academics for commissions as a standard, while Hartstichting does so on an ad hoc basis. In the case of the latter, the composition of the committee determines further arguments that are put forward. Therefore, the researchers concluded that the diverse composition of evaluation committees contributes to addressing other areas of research funding.
Recommendations for investigation committees
According to the study, research proposals can refer to different users, from specialists and patients to society at large. So definitions of scholarly quality, societal significance, and excellent research can vary from person to person. Based on mutual respect, a team of scientists and non-scientists can come to conclusions that are broader, more realistic, and more applicable than the research from the ivory tower.
The researchers also make other recommendations Peer review of search requests. The pamphlets on which the paintings are based should provide better definitions to guide the arguments. Now, aside from feasibility, no argument comes up consistently in the majority of conversations. In those conversations it would be better to evaluate proposals rather than rank them. This also prevents it from being more about the applicants’ reputation than it is about the application. Furthermore, patients must be trained to be prepared to participate in the committee.
Diversity in scientific culture
Finally, the research is aligned with developments in scientific culture initiated by the San Francisco Declaration on Research Assessment (DORA). This statement also mentions including users or patients in the research process. The researchers also confirm that A Progressive An approach to science that anchors the basis of academic conclusions in society. For this reason, research funders should Peer reviewPaintings should be diversified and open to non-scientists.
In the month of April, ScienceGuide research for experiences and opinions about it Recognition and appreciation program. All scholars from universities and institutes of knowledge are invited to complete the survey. click here to reconnaissance (And here for the English version).
The idea that all backgrounds are in the Academy is in line with the workshop organized by Working Group 0.7 during the recent Appreciation Festival. Academics from different backgrounds can make non-typical arguments that help science advance. During the workshop, the question was whether such diversity should be mandatory for each committee, or whether it was more important to spread and awareness of diversity across all committees. For now, ZonMw is keen on adopting recognition and recognition software. In medical research, diversity is by no means value added than social significance.
“Twitter junkie. Lifelong communicator. Award-winning analyst. Subtly charming internetaholic.”