Attendance at live vs virtual didactic sessions in a U.S. emergency medicine residency Brief Communication

¹MD, MS, Assistant Professor of Emergency Medicine, University of Colorado School of Medicine, Denver Health Medical Center, Department of Emergency Medicine and Department of Orthopedics, Denver, United States

²MD, Chief Resident, Denver Health Medical Center, Department of Emergency Medicine, Denver, United States

³PhD, MS, MSc, University of Colorado, Department of Biostatistics and Informatics, Aurora, United States

⁴MS, University of Colorado, Department of Biostatistics and Informatics, Aurora, United States

Date submitted: 31-July-2024

Email: Spencer Tomberg (spencer.tomberg@denverem.org)

This is an open access journal, and articles are distributed under the terms of the Creative Commons Attribution-Non Commercial-Share Alike 4.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as appropriate credit is given and the new creations are licensed under the identical terms.

Citation: Tomberg S, Dewispelaere W, Wogu A, Wang N. Attendance at live vs virtual didactic sessions in a U.S. emergency medicine residency. Educ Health 2024;37:383-388

Online access: www.educationforhealthjournal.org
DOI: 10.62694/efh.2024.150

Published by The Network: Towards Unity for Health

Introduction

The COVID-19 pandemic forced medical educators to adapt curricula to the virtual learning space with over 99% of emergency medicine residents partaking in virtual didactics.¹ While virtual learning brings flexibility and efficiency, it may also impact learner attendance.

There is limited research on the differences between live and virtual conference attendance in the graduate medical education (GME) literature. Outside of GME, a study of undergraduate Norwegian business students demonstrated the virtual setting increased learner attendance,² while other studies in primary/secondary education and undergraduate/graduate education have linked virtual learning to decreased learner engagement, chronic absenteeism, lower test performance and decreased attendance compared to in-person lectures.^3,4,5,6,7 Reasons for this decline include lack of learning community, distractions (other work/social media/entertainment), and the mental and physical exhaustion associated with video conferencing known as “Zoom fatigue.” ^1,8,9,10,11

This paper compares attending and resident physician attendance between live and virtual settings in an emergency medicine residency didactic program over a three-year period that spans the pre-pandemic, intra-pandemic and endemic COVID-19 phases; to our knowledge there are no other studies that make a similar comparison. First, we study the difference in participation in our morbidity and mortality (M&M) conference between live and virtual settings. M&M is robustly attended by residents and attendings and warrants a separate analysis. We also explore the difference in attendance in the didactic education sessions that follow M&M in live vs. virtual settings. These sessions are predominantly attended by residents.

Methods

This study examines attendance in an emergency medicine didactic program over a three-year period. The dataset begins in January 2020 before the COVID-19 lockdowns when conferences were exclusively live. From March 2020 through the summer of 2021 all learning was in the virtual environment. Starting in the summer of 2021, the ongoing emergency medicine conference was a hybrid of virtual and live sessions. There were no significant changes to the curriculum or attendance requirements over the course of the study.

Our emergency medicine conference is held every week (except for holidays) and starts with a 90-minute M&M session, followed by three-hours of didactic lectures which cover core emergency medicine topics. There is no integration of asynchronous learning into the didactic sessions. Attendings and residents affiliated with our residency program log their attendance for each M&M and didactic lecture into an online Airtable database (San Francisco, CA). Attending physicians are obligated to attend 50% of M&M conferences, and residents are obligated to attend 70% of total conference time (M&M and didactics) through each academic year. There were 68 residents and up to 149 faculty/fellows employed during each academic year.

Attendance data were extracted at five points during the conference day: 7:30am represents attendance during the 90-minute M&M conference that opens each session, and the remaining data points are didactic lectures occurring at 9:15am, 10:00am, 11:00am and 12:00pm. Each conference day was recorded as being live or virtual.

As a formal statistical analysis, we used generalized estimating equations (GEE) to fit marginal models with a Poisson family and long link function because of the lack of independence among repeated measures of outcome over the consecutive conference times.^12,13,14 The GEE approach lets us fit a model for an average response for observations sharing similar covariate information as a function of the covariates and, at the same time, accounting for within conference date correlation among repeated measures.^15,16,17 The independent working correlation was picked as the best working correlation structure. For each of the fitted marginal models, appropriate odds ratios (OR) and 95% confidence intervals (95% CI) were generated. All analyses were performed using R Statistical Software (version 4.2.2; R Foundation for Statistical Computing, Vienna, Austria).

The study was approved by the Colorado Multiple Institution Review Board as part of an established exemption protocol for educational research in the University of Colorado Department of Emergency Medicine.

Results

Data was available for 156 conference days. In the study period there were 10 days where the conference was canceled and there is no data for those weeks. Attendance at M&M was 20 percent higher in the virtual setting compared to in-person (OR 1.20; 95% CI: 1.15–1.26, p<0.001). There was a significant decrease in attendance between our M&M session (7:30am) and the beginning of didactics (9:15am) in both the live and virtual settings. After the transition to our didactic lectures at 9:15am, there was no significant difference in participation between live and virtual settings.

Table 1 Attendance at five time points throughout live and virtual conference days.

Discussion

When evaluating engagement in educational initiates, Appleton et. al described three dimensions to measure learner engagement: behavioral, cognitive and affective. The behavioral component includes the baseline question of if learners attend a lecture.¹⁹ This study addresses if there is a difference in attendance in an emergency medicine residency community (attendings and residents) between virtual and live didactic education settings. Answering this foundational question is the first step in the process of evaluating if there is a difference in learner engagement between live and virtual educational settings.

In this retrospective study, there was greater weekly M&M attendance in the virtual setting compared to the live setting. M&M is highly regarded among our residents and faculty, and the virtual setting likely made it easier to attend when participants had commitments that would have limited live participation. This is in keeping with previous literature that theorizes virtual learning encourages attendance when learners are sick, have transportation limitations, have other academic tasks, or cannot attend due to other life demands.^1,3,9, ¹⁸

In contrast to M&M, there was no significant difference in attendance between the live vs virtual settings for the remainder of the didactic education sessions. We expected a higher participation rate in the virtual setting. Didactics are a mixture of small-group/case-based sessions and large-group lectures that cover core emergency medicine topics defined by ABEM in the Model of the Clinical Practice of Emergency Medicine. It is likely that faculty participation drove the difference in attendance during M&M, and their absence during the didactic portion of conference, where there is more steady participation by residents, led to an equalization between the live and virtual settings. This aligns with faculty only being required to attend M&M, but not having a contractual obligation to be present for didactic lectures.

Figure 1 Mean attendance during live and virtual conference days from 7:30 to 12:00

Survey data from prior studies demonstrates that emergency medicine residents have a split preference for live vs virtual didactics. The live preference generally relates to community engagement and lack of distractions, and the virtual preference is driven by convivence.^1,20 Studies in other GME settings highlight both benefits and liabilities of virtual education related to learner engagement and performance.^21,22,23 Exploring the interplay of virtual and live education is important because in 2022, 21% of undergraduate students in the United States strictly took online courses, while 32% engaged in hybrid live/virtual education.²⁴ We expect these trends to move into medical education as well. Finally, there is conflicting data on test performance between live and virtual settings across educational domains. Some studies show equivalence between settings,^25,26 while others show that learners with low academic abilities primarily experience the negative impacts of virtual education.^5,6,27 Given the constellation of findings in these studies, we believe that providing educational material in both the live and virtual settings can be done thoughtfully to limit negative consequences for learners.

Finally, our hypothesis was that greater participation in virtual M&M would translate into greater participation in the didactics lectures that follow. This did not occur, and was likely driven by attending physicians engaging in M&M more robustly in the virtual setting, but residents making up most of the participants who stay for didactic sessions in both settings.

Moving into the future, our residency plans on using a mix of virtual and live conference. In another study, this mix has been demonstrated to be the preference of both emergency medicine residents and attendings.²⁸ The current study demonstrates that resident attendance will likely not be impacted by continuing to educate in both venues.

Limitations

The main limitation of this study is that it does not examine aspects of educational engagement beyond the behavioral element of attendance. Previous studies in emergency medicine education literature have demonstrated that learners in the virtual setting participate in multiple non-educational activities and have distractions while learning at home that would not be present in a live conference.^1,29 This study also relies on self-reported attendance, and both residents and attendings have incentives in their contracts to report attendance. Additionally, the structure of our institution’s conference day is specific, and these findings may not be generalizable to other emergency medicine programs. Future studies can compare other metrics of learner engagement in behavioral, cognitive and affective realms between the virtual and live settings.

Conclusion

The findings of this single center study demonstrated that attendance was similar between the live and virtual didactic education sessions that are part of a structured emergency medicine didactic training curriculum. This is the foundational step in evaluating overall engagement between the two distinct learning environments and supports further investigation of the effectiveness of educational activities in these settings. The integration of virtual M&M may allow more attendings and residents to attend those educational sessions.

1. Weygandt PL, Jordan J, Grabow Moore K. Impact of the COVID-19 pandemic on emergency medicine education: Insights from faculty and residents. Academic Emergency Medicine Education and Training. 2021;5(3):e10603.. DOI:10.1002/aet2.10603
PubMed PMC

4. Wood DB, Jordan J, Gottlieb M. Conference Didactic Planning and Structure: An evidence-based guide to best practices from the Council of Emergency Medicine Residency Directors. Western Journal of Emergency Medicine. 2020;21:999–1007. https://doi.org/10.5811/westjem.2020.5.46762
PubMed PMC

5. Cacault MP, Hildebrand C, Laurent-Lucchetti J, Pellizzari M. Distance Learning in Higher Education: Evidence from a Randomized Experiment. Journal of the European. Economic Association, 2021, 19(4), 2322–2372. https://doi.org/10.1093/jeea/jvaa060. Date of Access 20 August 2024.
Crossref

6. Kofoed MS, Gebhart L, Gilmore D, Moschitto R. Zooming to Class?: Experimental Evidence on College Students’ Online Learning during COVID-19. 2021. IZA Discussion Papers No. 14356, Institute of Labor Economics. https://doi.org/10.2139/ssrn.3846700. Date of Access 18 August 2024.

7. Evans WN, Muchnick K, Rosenlund O. Virtual Learning in Kindergarten Through Grade 12 During the COVID-19 Pandemic and Chronic Absenteeism. JAMA Network Open.. 2024.Open 7, e2429569. https://doi.org/10.1001/jamanetworkopen.2024.29569

8. Martijn M, Bele T, Simone Plak S. College students’ motivation and study results after COVID-19 stay-at-home orders. PsyArXiv. 2020;10–11. DOI:10.31234/osf.io/kn6v9.
Crossref

9. Weijers RJ Ganushchak L, de Koning B, “I’ll be there”: Improving online class attendance with a commitment nudge during COVID-19. Basic Applied Social Psychology. 2022;44(1):12–24. https://doi.org/10.1186/s41073-023-00139-z
Crossref

13. Pekár S, Brabec M. Generalized Estimating Equations: A pragmatic and flexible approach to the marginal GLM modelling of correlated data in the behavioural sciences. Ethology. 2018;124(2):86–93. https://doi.org/10.1111/eth.12713
Crossref

18. Photopoulos P, Tsonos C, Stavrakas I, Triantis D. Remote and In-Person Learning: Utility Versus Social Experience. Springer Nature Computer Science. 2023;4(2):116. https://doi.org/10.1007/s42979-022-01539-6

19. Appleton JJ, Christenson SL, Furlong MJ. Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the Schools. 2008;45(5):369–386. https://doi.org/10.1002/pits.20303
Crossref

20. Jameyfield E, Tesfai S, Olson A. An asynchronous curriculum: learner perspectives on incorporating asynchronous learning into in-person and virtual emergency residency didactics. Cureus. 2023;15(4): e38188. https://doi.org/10.7759/cureus.38188
PubMed PMC

21. Caton J, Chung S, Adeniji N. Student engagement in the online classroom: Comparing preclinical medical student question-asking behaviors in a videoconference versus in-person learning environment. Federation of American Society for Experimental Biology BioAdvances. 2020;3(2):110–117. https://doi.org/10.1096/fba.2020-00089

23. Wilcha R. Effectiveness of virtual medical teaching during the COVID-19 crisis: systematic review. Journal of Medical Internet Research Medical Education. 2020;6(2):e20963. https://doi.org/10.2196/20963

24. National Center for Education Statistics. Digest of Education Statistics. Number and percentage of students enrolled in degree-granting postsecondary institutions, by distance education participation, location of student, level of enrollment, and control and level of institution: Fall 2021 and fall 2022. https://nces.ed.gov/programs/digest/d23/tables/dt23_311.15.asp. Date of Access 9 Sept 2024.

25. Lee B, Zlotshewer B, Kaplan L. Impact of online-only instruction on preclinical medical education in the setting of COVID-19: Comparative analysis of online-only vs. hybrid instructions on academic performance and mental wellbeing. Medical Science Educator. 2022;32(6):1367–1374. https://doi.org/10.1007/s40670-022-01650-6
Crossref PubMed PMC

26. Pei L, Wu H. Does online learning work better than offline learning in undergraduate medical education? A systematic review and meta-analysis. Medical Education Online. 2019;24(1):e1666538. https://doi.org/10.1080/10872981.2019.1666538
Crossref

27. Nilaad S, Lin E, Bailey J. Learning outcomes in a live virtual versus in-person curriculum for medical and pharmacy students. American Thoracic Society Scholar. 2022;3(3):399–412. https://doi.org/10.34197/ats-scholar.2022-0001OC
PubMed PMC

28. Tsyrulnik A, Gottlieb M, Coughlin R. Socially distanced, virtually connected: faculty and resident perceptions of virtual didactics. Academic Emergency Medicine Education and Training. 2021;5(3): e10617. https://doi.org/10.1002/aet2.10617
PubMed PMC

Brief Communication