Use of artificial intelligence algorithms and mixed effects models to analyze educators’ and students’ perceptions and attitudes towards objective structured clinical examinations

Abstract

Background: During the COVID-19 pandemic, Tunisia implemented Objective Structured Clinical Examinations (OSCEs) for healthcare students at the School of Health Sciences and Technologies in Sousse. This study analyzed satisfaction levels using advanced analytical techniques. Methods: A cross-sectional study was conducted involving 128 final-year students across all healthcare programs and 31 educators. Two anonymous satisfaction surveys using five-point Likert scales were administered. The analysis employed supervised machine learning (SML), unsupervised machine learning (UML), and linear mixed-effects models (LMEM). The SML utilized Random Forest classification, whilst UML implemented K-means clustering and principal component analysis. Results: The SML demonstrated robust predictive accuracy for satisfaction categories (students: 78%; educators: 100%, though the latter may reflect overfitting due to the small sample size) with strong reliability (kappa values 0.65 and one). UML identified three distinct satisfaction clusters among students and seven among educators, with the highest satisfaction cluster showing a mean score of 3.82±0.34. LMEM revealed that student satisfaction increased with age (β=0.041), whilst educator
satisfaction correlated negatively with age (β=-0.017) but positively with teaching experience (β=0.016). Those from the Emergency Medical Care program consistently demonstrated higher student satisfaction levels. Conclusions: Integrating advanced analytical methods provided a more profound understanding of OSCE satisfaction patterns than traditional statistical approaches. The findings suggest multiple demographic factors influence OSCE satisfaction, necessitating context-specific approaches to the educational specialty rather than universal solutions. These results have important implications for healthcare education practice, suggesting the need for differentiated approaches to OSCE implementation based on student specialty and educators’ experience levels.