ƬƵ

EDI in Research: AI Biases as a Case Study

Did you notice the new faces in the equity team? As part of its commitment to equity, diversity, and inclusion (EDI), the Office of the Provost hired two new members to focus their work on EDI in research and scholarship. I sat down with Dr. Uzma Jamil, Senior Equity Research Advisor, and Andrea Clegg, Research Equity Advisor to learn more about their roles within the equity team.

Woman writing on computer sitting in front of a bookshelfFirst of all, what does EDI in research mean?

EDI in research is about integrating the principles of equity, diversity, and inclusion in all aspects of the research enterprise – for instance in research design, team composition, outcomes, and dissemination. It therefore goes beyond selecting specific research projects or topics with an EDI component, and it can lend itself to any field of research.

EDI in research is also used as a lens to critique the ways in which we do research. Research has been historically done without attention to specific issues surrounding EDI. As such, it may not capture the full way to see the world, it may be incomplete, or even biased. By meaningfully integrating the experiences and expertise of those historically marginalized, we promote a better and more effective way to do research.

For example, Indigenous community-based research, participatory action research, social action research, ethnography, and other forms of community-based research, all exemplify methodologies where members from groups being researched are part of the research team and where the research outcomes have a positive benefit to the communities most impacted by the research. These methodologies can be non-traditional, in that they do not follow a Western, positivist tradition for research.

Four people in a meetingWhy do we need to think about integrating EDI in research within universities?

Beyond the inherent benefits of using EDI in research, the academic research structure in Canadian universities integrates components of EDI. This can be seen through the evaluation criteria for various grant agencies, and more significantly through the Canada Research Chair (CRC) program.

First, as a response to inequities in research conducted in universities, major research grant proposals and programs from federal (CRC, CERC, CFREF, NFRF) and provincial grant agencies have equity criteria or targets or include EDI plans as requirements. There is a clear commitment to improving research conducted in Canadian universities by integrating EDI into the evaluation method. This is done in an effort to inform academics and researchers on the importance of integrating EDI into their work.

Second, and most importantly, . In 2003, eight complainants, all scholars within Canadian universities, filed a complaint with the Canadian Human Rights Commission regarding the program, for discrimination on how the Chairs were awarded. This challenge led to a settlement in 2006, where the equity targets for the Canada Research Chairs program were established. The settlement was also , notably to reflect the principle of intersectionality and revised equity targets.

Artificial Intelligence as a Case Study for Why EDI is Important in Research

Through a case study of research regarding artificial intelligence (AI), we can explore the ways in which research can be done with a focus on EDI principles and practices.

Reading glasses in front of a computer screen

AI has been emerging in a variety of fields, . Just last year, ƬƵ, in collaboration with UdeM, inaugurated a . However, is its tendency to exert, and at times amplify, biases in decision making. Dr. Safiya Umoja Noble, a scholar in the fields of information science, machine learning, and human-computer interaction, coined the term “” to portray this phenomenon. These biases are , including preferences found in the training data from human input, or from historical and contemporary inequities found in society. In other words, AI is flawed, because humans are flawed. Using EDI research lens may help to mitigate this.

There are various ways to integrate EDI into research in this context. First, by focusing on the outcome of the research. This means asking the right questions and thinking about the impact research has on various minority equity-seeking groups. As explained above, we know that algorithms drawing on the sociological and historical information may lead to biased decision making. The questions then become; how do I conduct research in this area to reduce biases found in the world instead of amplifying them? And, how to make a more accurate determination?

Second, through the research team composition. By integrating the voices of relevant communities directly into the research team, we may gain insight on relevant methodologies and there may be more buy-in. This can ultimately translate into impactful research for the community. In the example of AI, as the applications of the technology are so vast, this part would depend on the context. One context could be in the criminal justice system in the US. AI technology has been used in the US to . This method of evaluation was contested in court because it had replicated racial biases already present in the justice system. Having people with lived experiences with the justice system within the research team may help alleviate the possible negative outcomes of the research itself.

Third and finally, beyond team composition, through research methodologies sensitive to the community the research is based on. In other words, using methodologies that integrate the experiences of minority equity seeking group members into the research by design is reflective of an EDI research methodology. In the case of AI, this could mean using data training vetted by a diverse body of researchers and participants.

Ultimately, biases found in AI decision making are reflective of what we consider to be “neutral” or “objective” information and ways of knowing. The technology therefore replicates the unevenness and inequity of the world into its outcomes and impact. By challenging this assumption of objectivity through EDI in research, we ultimately present a more accurate representation of the world, and our research becomes much more impactful.

To learn more about EDI in research and for internal and external resources, visit the EDI Research section of the Equity website.


ƬƵ University is on land which has long served as a site of meeting and exchange amongst Indigenous peoples, including the Haudenosaunee and Anishinabeg nations. We acknowledge and thank the diverse Indigenous peoples whose presence marks this territory on which peoples of the world now gather.

For more information about traditional territory and tips on how to make a land acknowledgement, visit our Land Acknowledgement webpage.


Back to top