A cross sectional 15-year analysis of the effect of student and faculty demographics on student assessment

Abstract

Background:

Medical school assessments impact residency placement and professional progression. Recently, standardized tests found to be highly impacted by demographics have been de-prioritized in an effort to focus on faculty-assessed competency. However, small studies suggest that faculty assessment may also be affected by demographic factors. The current study examines 1) how student race/ethnicity are associated with standardized medical knowledge assessments and 2) whether demographics are associated with faculty evaluations.

Method

Using a 15-year data set (2007- 2022) from a large academic medical center in the United States, generalized mixed regression models were conducted to determine whether student race/ethnicity was a predictor of: 1) failing National Board of Medical Examiners (NBME) score (<60%) 2) exceptionally good NBME score (>90%); and whether student and faculty race/ethnicity predicted 3) low performance evaluations (LPE, failing evaluations). Covariables included age, gender, and rotation characteristics such as academic month, academic year, and subject.

Results

Compared to White students, other racial groups had higher odds of failing the NBME (Asian OR 2.24 [1.69, 2.93], p < 0.001; Hispanic students OR=1.59 [1.14, 2.23]; p = 0.003; Black OR=4.64 [3.45, 6.24]; p < 0.001). Compared to White students, all other races/ethnicities had lower odds to score 90 or above. Asian students had higher odds to receive an LPE from an Asian faculty member (OR 1.75 [1.03, 2.97]; p = 0.04). Black students had higher odds to receive an LPE from a White faculty member (OR 2.43 [1.46, 4.04]; p < 0.001). Black, Asian, and Hispanic students, compared to White students, had higher odds of failing the NBME than they were to receive an LPE.

Conclusions

Demographic differences exist in both faculty evaluations and exam scores, and were not mitigated by racial concordance between learners and evaluators. More work is needed to develop strategies that promote an equitable learning environment.