Original Research Paper

Effects of Active Learning Techniques on Learners’ Perceptions of Engagement and Effectiveness in Pre-Clinical Courses

Chosang Tendhar PhD ¹, Clara Chen MD ², Christopher Duffy MLIS AHIP ³, Keith Metzger PhD ⁴, Elizabeth Koltz EdM ⁵

¹Director of Institutional Effectiveness and Assessment, Hackensack Meridian School of Medicine, Nutley, New Jersey, United States

²Internal Medicine Resident PGY-1, Stony Brook University Hospital, Stony Brook, New York, United States

³Associate Dean, Interprofessional Health Sciences Library, Hackensack Meridian School of Medicine and Seton Hall University, Nutley, New Jersey, United States

⁴Assistant Dean for Medical Education, Professor, Department of Medical Sciences, Hackensack Meridian School of Medicine, Nutley, New Jersey, United States

⁵Director of Instructional and Curricular Design, Assistant Professor, Department of Medical Sciences, Hackensack Meridian School of Medicine, Nutley, New Jersey, United States

ABSTRACT

Background: Due to COVID-19, second-year medical students (M2s) at the Hackensack Meridian School of Medicine (HMSOM) completed a substantial portion of their pre-clerkship training in person, while first-year students (M1s) completed their entire pre-clerkship training online. The purposes of this study are threefold. To examine (1) the impact of various active learning techniques employed at HMSOM on learners’ perceptions of engagement and effectiveness; (2) differences in perceptions of the impact of active learning techniques on engagement and effectiveness between cohorts; and (3) the relationships between pre-work and engagement/effectiveness. Methods: The data for this study were collected from M2s and M1s using a 22-item questionnaire that the authors of this study developed after pilot testing it. SPSS v. 27 was used to compute descriptive statistics, independent t-tests, and correlations. Results: The “Zoom polling tool” and “Case-based session” were ranked highest, while “Professors randomly calling on students individually” and “Making it mandatory to keep video cameras on” were ranked lowest. Significant differences were found between M2s and M1s in their perceptions of engagement and effectiveness related to some active learning techniques. Discussion: The findings of this study have numerous theoretical and practical implications. The Zoom polling tool and Case-based session were perceived to have the greatest impact on students’ engagement and effectiveness. Significant differences were observed between M2s and M1s in their perceptions of the impact of three active learning techniques. M2s, with in-person class experiences, scored higher on “Professors randomly calling on students individually”. On the other hand, M1s, who only had online experiences, scored higher on pre-work video modules. Some of the limitations of this study are discussed for potential future research.

Keywords: in-person vs. online learning, active learning techniques, student engagement, learning effectiveness, prework

Background

The COVID-19 pandemic impacted various aspects of everyday life. One of its most notable consequences was that the entire global education system was forced to proceed online. Our “new normal” has begun to settle in, with COVID-19 being an inevitable virus that may never go away. While most schools have returned to in-person classes, the decision to continue with some online delivery of educational content is being considered across the board. Medical education is no exception.

Online learning can take one of two paths: synchronous and asynchronous. Synchronous means that students and faculty are in a learning environment simultaneously using videoconferencing and chat rooms. In contrast, similar technologies are used asynchronously, such as email and discussion boards, but students and faculty are not together simultaneously.¹ At Hackensack Meridian School of Medicine (HMSOM), the courses were delivered through the synchronous method.

The popularity of online learning has risen recently. There has been steady growth in 100% online universities, for example, Western Governors University, Capella University, and Walden University. Similarly, the number of online courses offered by massive open online courses (MOOCs), such as Coursera and Udemy, and participants in those courses, have significantly increased. Specifically, online learners increased from 300,000 in 2011 to 220 million in 2021.²

Regardless of the learning environment, whether online or in the classroom, the utility and application of instructional techniques that actively engage learners are equally relevant in both settings. Active learning is an instructional technique that employs student-student and student-facilitator interaction in various ways to transform the learning environment from passive to active.³ Some active learning techniques used at HMSOM include: Case-based sessions, Small group breakout sessions, a Zoom polling tool, and Problem-based learning. Active learning techniques have positive learning experiences for learners, including retaining students in STEM fields.^4–7

A robust literature exists on the efficacy of online learning. including meta-analyses.^8–11 However, studies still need to be conducted to examine the efficacy of various active learning techniques in the online learning environment, especially within the medical education literature. In addition to the rising popularity of online learning, many advocate for blended learning in the post-pandemic era.^12–13 Furthermore, we agree with Althwanay et al ¹² that pandemics tend to recur over time. Online learning will then likely become a prominent fixture in various forms in the educational landscape.

This exploratory study will further enrich the medical education literature about active learning techniques in the context of online learning. This study focuses on the effect of active learning techniques used in the online learning environment, and on learners’ perceptions of their engagement and effectiveness. Specifically, we examined the impact of each dynamic learning technique used at HMSOM on enhancing learners’ engagement level in the class, and effectiveness in learning the medical school course content.

The constructs of engagement and effectiveness are of deep interest to the authors of this study because they are related to better academic outcomes. A study found a moderately strong and positive correlation between student engagement and academic achievement.¹⁴ These findings are echoed in other studies.^15–16

Student engagement is “time and energy students devote to educationally sound activities inside and outside the classroom.” ¹⁷ Similarly, academic engagement refers to the amount of time and energy spent on activities that lead to positive academic outcomes.¹⁸ More engaged learners are likely to attain a higher level of academic success,^19–20 and they are more likely to graduate, receive better grades, and feel more satisfied with their collegiate experience.^21–22 The authors of this study argue that learners’ positive perceptions of the effectiveness of learning course content will yield similar results as that of more engaged learners, for example, better achievement results and greater satisfaction with the learning experience.

We believe that active learning techniques will influence both learners’ level of engagement in the class and effectiveness in learning course materials. Some of the commonly employed active learning techniques at HMSOM are: (1) Pre-work video modules; (2) Application exercise component of Team-Based Learning (TBL) sessions; (3) Zoom chat tool for student comments/questions; (4) Zoom polling tool; (5) Making it mandatory to keep video cameras on; (6) Case-based sessions; (7)

Professors randomly calling on pre-assigned student groups; (8) Small group breakout sessions; and (9) Professors randomly calling on students, individually. The authors note that while pre-work may or may not have active learning elements, pre-work is a significant component of active learning. We wanted to get input from the students, so it was essential to include it in our active learning study.

The first research question of this study was to investigate the impact of each active learning technique on learners’ perceptions of engagement and effectiveness during the preclinical program, and to rank the active learning techniques in order. The rank order will help determine the strength of each active learning technique. A study that proposed the concept of rank ordering can be gainfully used in this study.²³ According to this author, the rank-ordering of competencies helps individual learners and programs identify areas of strength and improvement.²³ In this study, rank ordering of active learning techniques will help us determine the methods that may be more often utilized in the online learning environment, based on their impact.

It is worth noting that second-year medical students (M2s) at HMSOM had some in-person experiences before the COVID-19 pandemic, while first-year medical students (M1s) experienced only virtual learning. Therefore, the second research question of this study is to examine differences in perceptions of the impact of active learning techniques on engagement and effectiveness between M2s and M1s.

It is safe to assume that learners who come prepared to the class by completing all the pre-work are likely to have a higher level of engagement in the class, and find it more effective in learning the course content. Therefore, the third research question examines the correlation between pre-work and engagement, and pre-work and effectiveness.

Methods

Participants

We invited 208 medical students, consisting of M1s and M2s at HMSOM, to participate in this study. One hundred twenty-three (123) students responded to the survey, resulting in a response rate of 59%. Of those 123 respondents, 46 (37%) represented M2s, while 77 (63%) represented M1s. We did not collect any demographic data, such as race and gender.

Instrument

We collaboratively developed a 22-item questionnaire, rigorously reviewed it, and as a result, made multiple revisions. We pilot-tested the questionnaire with a small group of students. We assessed two important constructs: engagement and effectiveness. Study participants were asked to respond to their perceived engagement level and effectiveness when those nine active learning techniques were used (e.g., Application exercise component of TBL sessions and Case-based sessions). Engagement was measured on a five-point Likert-type scale ranging from 1 (highly disengaged) to 5 (highly engaged), while effectiveness was also measured on a five-point Likert-type scale ranging from 1 (highly ineffective) to 5 (highly effective). We included operational definitions of engagement (The amount of energy students expend whereby they stay attentive, involved, and motivated to learn) and effectiveness (Session’s ability to help students better understand learning topics covered in the session) in the questionnaire so that there was a common frame of reference for every respondent.

Data Analysis

We used the Statistical Package for Social Sciences (SPSS) version 27 to compute descriptive statistics, Independent Sample t-tests, and correlations.

Ethical Considerations

The Institutional Review Board (IRB) at Hackensack Meridian Health (HMH) approved this study (Study # Pro2021-0520).

Results

We examined the impact of nine active learning techniques employed at HMSOM on learners’ perceived engagement and effectiveness. Tables 1 and 2 present the descriptive statistics and rank ordering of those active learning techniques.

Table 1 Descriptive Statistics of Active Learning Techniques and their Impact on Learners’ Perceived Engagement

Table 2 Descriptive Statistics of Active Learning Techniques and their Impact on Learners’ Perceived Effectiveness

The third-, fourth-, and fifth-highest ranked active learning techniques differ for engagement and effectiveness. The rankings of the remaining techniques are similar for both constructs.

Our second research question pertains to investigating mean differences between M2 and M1. Tables 3 and 4 present the mean differences between the two groups and the results of their t-tests.

Table 3 Group Mean Differences Between M2 and M1 on Engagement and Significant Values of Independent Samples T-Tests

Table 4 Group Mean Differences Between M2s and M1s on Effectiveness and Significant Values of Independent Samples T-Test

As Table 3 shows, the mean scores of M2 were higher on “Professors randomly calling on students, individually” and “Professors randomly calling on pre-assigned student groups”, while the mean score of M1 was higher on “Pre-work video modules” and the mean differences were statistically significant. No other significant differences were observed.

Of the nine active learning techniques, learners’ perceptions of their impact on effectiveness varied for four of them (Table 4): Making it mandatory to keep video cameras on, Professors randomly calling on students, Professors randomly calling on pre-assigned student groups, and Pre-work video modules. In the first three instances, the mean scores of M2s were significantly higher, while the M1s had a higher mean score on the pre-work video modules.

The third research question addresses relationships between pre-work completed and the two constructs of interest: Engagement and Effectiveness. For a detailed exploration of these correlations and their statistical significance, refer to Table 5.

Table 5 Correlations between pre-work and Engagement, and pre-work and Effectiveness

Discussion

This exploratory study addressed three questions. The findings of this study have numerous theoretical implications and practical implications for pedagogy.

The first research question pertains to the effect of nine active learning techniques employed at HMSOM on learners’ perceptions of engagement and effectiveness. Three active learning techniques: Application exercise component of TBL sessions; Pre-work video modules; and Zoom chat tool for student comments/questions ranked third through fifth, but the rank order of these three active learning techniques slightly varied between engagement and effectiveness. For example, the application exercise component of TBL sessions ranked third in engagement, while pre-work modules ranked third in effectiveness. Tables 1 and 2 show the detailed ranking information.

There is consistency in the impact of active learning techniques on both engagement and effectiveness, particularly evident at the top and bottom of the list. Specifically, the Zoom polling tool and Case-based sessions ranked first and second, respectively, for both engagement and effectiveness. Those two active learning techniques not only enhance learners’ attentiveness, involvement, and motivation to learn, but also contribute to a deeper understanding of the learning topics covered in the session when compared to other techniques. In essence, these two techniques were notably more impactful in enhancing both engagement and effectiveness than other active learning approaches. Additionally, the ranking of instructional techniques, specifically ranging six through nine, shows a similar pattern for both engagement and effectiveness. Using a five-point Likert type scale, mean scores for “Professors randomly calling on students, individually” and “Making it mandatory to keep video cameras on “ were below three, ranking eighth and ninth, respectively. In essence, those two active learning techniques were perceived as the least impactful for both engagement and effectiveness.

The second research question investigated mean differences between M2s and M1s regarding their perceived benefits of various active learning techniques. Before COVID-19, M2s completed the majority of their pre-clerkship courses in person while M1s completed their entire pre-clerkship courses online. Significant mean differences were detected for three variables: Professors randomly calling on students, individually; Professors randomly calling on pre-assigned student groups; and Pre-work video modules. M2s exhibited higher mean scores on the first two variables, whereas M1s had a higher mean score on the last variable. The findings of this study suggest that individuals with in-person experiences feel more comfortable being called on by professors compared to those with online-only experiences. Moreover, learners with exclusively online experiences tend to complete their “pre-work video modules” at a higher rate than those with substantial in-person experiences.

Instructors have control over the active learning techniques they choose to adopt. In terms of practical implications, the ranking information presented in Tables 1 and 2 from this study serves as a robust guide for selecting appropriate active learning techniques. It is advisable to more frequently employ those active learning techniques that are ranked higher, and less frequently those techniques at the bottom of the list.

The third research question examined the correlation between pre-work and engagement as well as the correlation between pre-work and effectiveness. Of the nine active learning techniques, Table 5 illustrates that completing pre-work significantly correlated with both engagement and effectiveness in four active learning techniques: Application exercise component of TBL sessions, Professors randomly calling on students individually, Case-based session, and Pre-work video module. However, pre-work also exhibited a significant correlation with Professors randomly calling on pre-assigned student groups and Small group breakout sessions—but only with effectiveness.

While it’s important to note that correlation does not imply causation, a theoretical and logical argument can be made that individuals who consistently complete Pre-work may exhibit higher engagement levels, especially when participating in activities such as Case-based sessions and Professors randomly calling on students individually. The completion of assigned pre-work enables learners to actively contribute to the class. Similarly, the effectiveness of these two active learning techniques in facilitating the understanding of course content may be attributed to the prior completion of Pre-work.

Learners’ academic engagement has been widely studied for reasons mentioned earlier and so too are predictors of engagement. For example, one study investigated impacts of academic belonging, social integration, and resilience on academic engagement.²⁴ Similarly, there are other studies where predictors of student engagement such as personal motivation, family, peers, and teachers have been examined.²⁵ However, the relationships between various active learning techniques and engagement have not been previously examined. Therefore, this study contributes significantly to theory by identifying active learning techniques as potential antecedents of learners’ engagement. Importantly, it sheds light on their diverse impacts on learners’ engagement in different learning systems—specifically, in-person versus online. In other words, the study reveals that the effectiveness of various active learning techniques varies between individuals with substantial in-person experiences and those whose pre-clerkship experiences were entirely online. Notably, Professors randomly calling on students or groups were perceived as more effective and engaging for those with prior in-person experiences, while Pre-work video modules were considered more effective and engaging for those without such experiences.

Limitations

We identified some limitations in this study that can be addressed in future studies. Firstly, the study was conducted at a single institution, suggesting the need for cross-validation through samples from multiple schools to enhance the generalizability of the findings. Secondly, determining the effects of self-reporting bias is challenging. Thirdly, future studies should explore the relationship between the completion of pre-work and more objective performance measures, such as learners’ grades. Fourth, given that various active learning techniques are not employed at HMSOM, further research could investigate the impact of these techniques on learners’ perceptions of engagement and effectiveness. Finally, understanding the relationships between learners’ perceptions of effectiveness and more objective measures, such as course grades and performances on board exams, is important.

Conclusion

This study has made significant contributions to the research on active learning techniques, exploring their relationships with both engagement and effectiveness, in online and in-person sessions. This is the first study to examine the impact of active learning techniques in online classes. Employing the rank-ordering technique, we identified active learning techniques at HMSOM that exhibit varying levels of impact. Specifically, the Zoom polling tool and Case-based sessions emerged as the most impactful, influencing students’ perceptions of engagement and effectiveness. We observed statistically significant mean differences between M2s and M1s regarding their perceptions of three active learning techniques. For instance, M2 students were more comfortable when professors randomly called on them, compared to their M1 counterparts, while M1 students scored higher on pre-work video modules. These differences may be in part attributed to the different learning experiences—with M2 students experiencing in-person learning, and M1 students exclusively engaging in online experiences.

The authors of this manuscript have no declarations of interest to report.

1. Hrastinski S. Asynchronous and synchronous e-Learning. Educause Quarterly. 2008;31(4):51–55.

2. Shah D. A Decade of MOOCs: A review of stats and trends for large-scale online courses in 2021. EdSurge Newsletter. EdSurge. 2021. https://www.edsurge.com/news/2021-12-28-a-decade-of-moocs-a-review-of-stats-and-trends-for-large-scale-online-courses-in-2021. Accessed February 27, 2023.

3. Al-Bahi A. Development of a design phase checklist for outcome-based active/cooperative learning courses. 2006. https://peer.asee.org/development-of-a-design-phase-checklist-for-outcome-based-active-cooperative-learning-courses. Accessed February 27, 2023.

4. Seymour E, Hewitt NM. Talking about leaving: why undergraduates leave the sciences. Boulder, CO: Westview Press; 1997.

5. Matusovich H, Paretti M, Jones B, Brown PR. How problem-based learning and traditional engineering design pedagogies influence the motivation of first year engineering students. Proceedings of the American Society for Engineering Education Annual Conference and Exposition. June 10–13, San Antonio, TX;2012

6. Watkins J, Mazur E. Retaining students in Science, Technology, Engineering, and Mathematics (STEM) majors. Journal of College Science Teaching. 2013;42(5):36–41.

7. Tendhar C, Singh K, Jones BD. Effects of an active learning approach on students’ motivation in an engineering course. Journal of Education and Training Studies. 2019;7(3):58–64. http://doi.org/10.11114/jets.v7i3.3916

8. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. Journal of the American Medical Association. 2008;300(10):1181–1196. https://doi.org/10.1001/jama.300.10.1181

9. Soffer T, Nachmias R. Effectiveness of learning in online academic courses compared with face-to-face courses in higher education. Journal of Computer-Assisted Learning. 2018;34(5):534–543. https://doi.org/10.1111/jcal.12258

10. Dost S, Hossain A, Shehab M, Abdelwahed A, Al-Nusair L. Perceptions of medical students towards online teaching during the COVID-19 pandemic: a national cross-sectional survey of 2721 UK medical students. BMJ Open. 2020;10(11):e042378. https://doi.org/10.1136/bmjopen-2020-042378

11. Korde S, Kale G, Dabade T. Comparative study of online learning and classroom lLearning. ANWESH: International Journal of Management Information Technology. 2021;6(1):23–32. https://doi.org/10.3389/fcomp.2019.00007

12. Althwanay A, Ahsan F, Oliveri F, Goud HK, Mehkari Z, Mohammed L, Javed M, Rutkofsky IH. Medical education, pre- and post-pandemic era: a review article. Cureus. 2020;12(10):e10775. https://doi.org/10.7759/cureus.10775

13. Bashir A, Rana K, Lambert P, Vernallis A, Bashir S. Post-COVID-19 adaptations; the shifts towards online learning, hybrid course delivery and the implications for biosciences courses in the higher education setting. Frontiers in Education. 2021; 6. https://doi.org/10.3389/feduc.2021.711619

14. Lei H, Cui Y, Zhou W. Relationships between student engagement and academic achievement: a meta-analysis. Social Behavior and Personality: An International Journal. 2018;46(3):517–528. http://doi.org/10.2224/sbp.7054

15. Wu Z. Academic motivation, engagement, and achievement among college students. College Student Journal. 2019;53(1):99–112.

16. Grant Ll, Opperman MJ, Schiller B, Chastain J, Plawecki MH, Richardson JD, Eckel C. Medical student engagement in a virtual learning environment positively correlates with course performance and satisfaction in psychiatry. Medical Science Educator. 2021;31(3):1133–1140. https://doi.org/10.1007/s40670-021-01287-x

17. Kuh GD. The National Survey of Student Engagement: conceptual framework and overview of psychometric properties. https://scholarworks.iu.edu/dspace/handle/2022/24268. Accessed February 27, 2023.

18. An BP. The role of academic motivation and engagement on the relationship between dual enrollment and academic performance. Journal of Higher Education. 2015;86(1):98–126. http://doi.org/10.1080/00221546.2015.11777358

19. Tinto V. Classrooms as communities: exploring the educational character of student persistence. Journal of Higher Education. 1997;68(6):599–623. https://doi.org/10.2307/2959965

20. Pascarella ET, Terenzini PT. How college affects students: A third decade of research. Volume 2. San Francisco, CA: Jossey-Bass; 2005.

21. Astin AW, Sax LJ. How undergraduates are affected by service participation. Journal of College Student Development. 1998;39:251–263.

22. Kuh GD, Kinzie J, Schuh J, Whitt E. Student success in college: Creating conditions that matter. San Francisco, CA: Jossey-Bass; 2010.

23. Tendhar C. Rating scale data and its utility: additional evidence to be explored in the medical education continuum. Journal of Assessment in Higher Education. 2021;2(1):54–62. https://doi.org/10.32473/jahe.v2i1.121588

© Education for Health.

Education for Health | Volume 37, No. 1, January-March 2024

(Return to Top)