Use of AI machine learning algorithms to assess medical student engagement and predict performance

Abstract

Background: To understand how students engage with course activities, it is important for educators to predict students´ degree of participation. Artificial intelligence (AI) has become a valuable tool in higher education, particularly in predicting students’ academic engagement. This study compares nine AI machine learning algorithms to determine students’ engagement in a basic medical science course and examine its correlation with their assessment scores. Altair RapidMiner studio software was employed for data visualization, calculation of correlation coefficient, and predictive analysis. Methods: We employed machine learning (ML) classification algorithms to analyze students'
engagement in a Medical Parasitology course taught to first year medical students. The independent variables used included their performance scores, and their level of interaction with course materials on the learning management system, such as frequency of viewing content and completing activity. The dependent variable was students´ engagement levels across various activities. To predict students' engagement, we applied nine ML algorithms to the dataset: namely Naïve Bayes, Generalized Linear Model, Logistic Regression, Fast Large Margin, Deep Learning, Decision Tree, Random Forest, Gradient Boosted Tree, and Support Vector Machine. Their performance was evaluated using several metrics. Results: The Logistic Regression exhibited the highest performance among the models tested, achieving an accuracy of 95%, classification error of 5%, precision of 100%, recall of 88.4%, F-measure of 93.8%, sensitivity of 88.4%, specificity of 100%. Discussion and Conclusions: The number of student logins to course materials was strongly related to students´ engagement. Highly engaged students achieved better results on assessments compared to those with lower engagement. Additionally, students with minimal engagement participated less frequently in various course activities. These findings highlight the potential use of RapidMiner as an effective AI tool for educational institutions to accurately classify students as engaged or non-engaged.