Initially a bit of a goofy idea, the raffle is starting to be taken seriously by many research funding agencies. Indeed, it is a promising solution to increase equity, diversity and inclusion (EDI) in academia.
Several members of the research community consider that the peer review does not enable a fair comparison of files deemed to be similar, making the selection between these files more sensitive to the unconscious bias and other valuation biases. It is important to remember that the evaluation by peers, although it aims to be as objective as possible, always includes an element of subjectivity.
My research focuses on issues of equity, diversity and inclusion in research and post-secondary education. Thus, I am constantly looking for ideas and practices that promote participation in research that is fair, equitable and representative of the entire population. Knowing that lotteries are little known at the Canadian level, I present here some cases from abroad and recently presented in a report produced for the Quebec Research Funds.
Anonymization of records to increase diversity
To encourage support for innovative research projects, the Health Research Council of New Zealand (HRC) implemented in 2013 a process for anonymizing and randomly drawing applications from the Explore grant. The Explorer Grant supports research proposals with the potential to make game-changing change in the New Zealand healthcare sector.
In this program, the files to be evaluated are first anonymized. The committee evaluates the applications to determine whether the proposed project meets the two selection criteria: the potential for scientific progress and the viability of the project. Files that meet both criteria are all equally eligible for funding. A random number generator then orders the folders.
In 2020, a study probe the acceptability of the draw in the allocation of the Explorer grant. Of the 126 people surveyed, 63% said that the lottery is a good way to select projects to be funded under this program, with this support being higher among people who have received an Explorer grant. More informally, in an interview, the HRC team shared with us that the anonymization process is linked to an increase in the diversity of candidates’ backgrounds.
In the same vein, in 2019 the Austrian Research Fund (FWF) launched a pilot project for its “1000 ideas” program. This program aims to fund bold projects in all disciplines, with the potential for revolutionary discovery, and the selection process also includes an anonymized project and a draw.
Reduce bias and decide between equivalent files
In 2018, Matthew Egger – President of the Research Council of the Swiss National Science Foundation (SNSF) – introduces the idea of the draw to the Swiss research community. Shortly thereafter, the Swiss National Science Foundation set up a lottery procedure for a postdoctoral fellowship program. In addition to reducing the impact of unconscious bias, this measure also makes the application evaluation process less time-consuming and resource-intensive.
Following this positive experience which was the subject of a scientific publicationthe draw was included in the organizational regulations in the following way:
Funding decisions can rely heavily on the assessment by external experts or, in the event of insufficient incentive funds, applications can be selected by lottery that are objectively indistinguishable .
The SNSF informs the applicants whose funding has been accepted or not by drawing lots. In March 2021, the draw was used on only 9 out of 278 proposals, i.e. 3.2%.
More recently, in September 2022, the British Academythe National Academy of Humanities and Social Sciences of the United Kingdom has announced the setting up of the draw to allocate funding to two equivalent applications. Indeed, the members of the evaluation committee affirm that there are twice as many proposals that reach the threshold of excellence, but that it is impossible to accept them all. The drawing of lots therefore makes it possible to “choose” the files financed without bias.
(Shutterstock)
Why adopt the draw?
The arguments put forward above by the HRC and the FNS in adopting the draw are reflected in a study on the factors facilitating or limiting the implementation of random draw methods in granting agencies.
Findings from interviews with influencers in granting agencies show two motivations for adopting lottery:
-
equity goals, including differentiating between equivalent proposals when funding is limited; And
-
encourage a wider range of topics and types of research, including potentially groundbreaking research.
The study also notes that applicants show less hostility in receiving a denial of funding than expected by granting agencies.
For more
This discussion, which seemed impossible five or ten years ago, is taking more and more of magnitude. In Canada, granting agencies are questioning the practices to adopt in order to increase the representativeness of the people and projects funded.
In my opinion, the drawing of lots is a simple and advantageous solution which notably makes it possible to lighten the considerable task of evaluation. But above all, in the face of the draw, each individual or project is completely equal!
The examples of raffle set up for the New Zealand Health Research Council and the Swiss National Fund are taken from the report Innovative EDI Practices by Granting Agencies published in November 2022 by the Quebec Interuniversity Network for Equity, Diversity and Inclusion for the Quebec Research Funds.
