Development and validation of the Diversity, Equity, and Inclusion Index (DEII) tool for assessing DEI in medical education lectures: a pilot study

Abstract

Background: Evaluating diversity, equity, and inclusion (DEI) in health education is crucial to ensure an inclusive and effective learning environment. In this pilot study, we aimed to create and validate the first tool, the Diversity, Equity, and Inclusion Index (DEII), to assess DEI within health education lectures. We hypothesized that the DEII would demonstrate strong face and content validity as well as acceptable to excellent interrater reliability. Methods: This pilot cross-sectional study was conducted from January 2022 to December 2023. We developed the Diversity, Equity, and Inclusion Index (DEII) and assessed its face and content validity. The study followed the COSMIN taxonomy guidelines. Interrater reliability was calculated using intraclass correlation coefficients from DEII scores of 50 YouTube lectures. Setting: The study was conducted at several academic medical centers in the United States of America, encompassing diverse health education settings. Participants: 10 expert-DEI reviewers as well as 10 non-expert DEI reviewers who represented the major sections of health professions including physicians, nurses, researchers, social workers, respiratory therapists, and physical therapists, as well as multiple institutions assessed face and content
validity. Expert reviewers were defined based on specific criteria, including board certification, leadership positions in DEI, authorship of peer-reviewed DEI publications, national presentations on DEI topics, and implementation of DEI initiatives. Non-expert faculty were selected based on their involvement in health education but without specific DEI expertise. Participants were recruited through a convenience sample approach. All approached participants agreed to take part in the study. To assess interrater reliability, three researchers scored 50 health education lectures on YouTube whose audiences ranged from respiratory therapists, nurses, physicians, physician associates and those training to be each. Results: Interrater reliability analysis demonstrated that 16 out of 17 items met the threshold for acceptable reliability (ICC ≥ 0.50).  Specifically, 7 items exhibited excellent reliability (ICC ≥ 0.90), 5 demonstrated good reliability (0.70 ≤ ICC < 0.90), and 4 showed moderate reliability (0.50 ≤ ICC < 0.70). At the domain level, three of the four domains achieved acceptable reliability (ICC ≥ 0.70), with one domain reaching the threshold for excellent reliability (ICC ≥ 0.90). Discussion: In this pilot study, the DEII is a reliable and valid tool for assessing diversity, equity, and inclusion in health education lectures. Its implementation may enhance the inclusivity of health education, ultimately leading to better-prepared
healthcare professionals and improved patient outcomes. Further research is needed to refine the tool and explore its impact on educational practices and outcomes.