Impacts of the COVID-19 Pandemic on United States Emergency Medicine Education: A Council of Residency Directors in Emergency Medicine (CORD) Task Force Survey-Based Analysis

Introduction The COVID-19 pandemic presented unpredicted challenges to Emergency Medicine (EM) education. The rapid onset of the pandemic created clinical, operational, administrative, and home-life challenges for virtually every member of the medical education community, demanding an educational and professional response at all levels including undergraduate medical education (UME), graduate medical education (GME), and faculty. The Council of Residency Directors in Emergency Medicine (CORD) COVID-19 Educational Impact Task Force was established in 2021 to examine these effects and the response of the EM educational community. Methods The Task Force utilized consensus methodology to develop the survey instruments, which were revised using a modified Delphi process. Both open- and closed-answer questions were included in the survey, which was initially distributed electronically to attendees of the 2021 Virtual Academic Assembly. Results were analyzed quantitatively and qualitatively. Results Sixty-three individuals responded to the first part of the survey (which addressed issues related to UME and GME) and 41 individuals responded to the second part of the survey (which addressed faculty and wellness). The pandemic’s influence on EM education was viewed in both a positive and negative light. The transition to virtual platforms had various impacts, including innovation and engagement via technology. Remote technology improved participation in didactics and allowed individuals to more easily participate in departmental meetings. However, this also led to a decreased sense of connection with peers and colleagues resulting in a mixed picture for overall engagement and effectiveness. The Task Force has developed a list of recommendations for best practices for EM programs and for EM organizations. Conclusion The survey results articulated the educational benefits and challenges faced by EM educators during the COVID-19 pandemic. Through the challenging times of the pandemic, many institutional and program-based innovations were developed and implemented to address the new educational environment. These approaches will provide invaluable educational tools for future training. This will also prepare the EM academic community to respond to future educational disruptions.


Introduction
The COVID-19 pandemic has presented unpredicted challenges to Emergency Medicine (EM) education. The rapid onset of the pandemic created clinical, operational, administrative, and home-life challenges for 1 2 3 4 5 6 virtually every member of the medical education community. This unprecedented event demanded an educational and professional response at all levels. Effects were felt in undergraduate medical education (UME), the Match process, graduate medical education (GME), faculty development, and wellness [1][2][3][4]. The Council of Residency Directors in Emergency Medicine (CORD) COVID-19 Educational Impact Task Force was established in 2021 to examine these effects and the response of the EM educational community [5].
Through the challenging times of the pandemic, many institutional and program-based innovations were developed and implemented to address education barriers [6]. While individual endeavors were key to overcoming educational constraints, there was no clear coordinated effort within academic EM to examine these endeavors. In order to do this, the CORD Board of Directors convened the COVID-19 Educational Task Force in February 2021 with a charter to investigate the impact of the COVID-19 pandemic on EM education [5]. Membership on the Task Force was determined by an application process and the Task Force first convened in February 2021 to discuss this goal. The Task Force moved quickly to develop a survey to collect data related to experiences during the pandemic as well as an initial needs assessment. The goal was to capture the educational experiences during the pandemic in order to create a roadmap for future endeavors and explore how organizations such as CORD could support their members during similar events in the future.

Materials And Methods
The Task Force utilized consensus methodology to develop the survey instruments [7]. Topic areas were identified using expert consensus and revised using a modified Delphi process [8]. A sub-group of the Task Force developed a draft survey which was then iteratively developed using three cycles of change. Both open-and closed-answer questions were included in the survey, with the former occurring first in order to increase construct validity [8]. Similarly, demographics were entered at the end in order to limit the introduction of bias [9]. To address criterion issues, previously validated means of response were utilized when possible [8]. The pilot version of the survey was distributed to the remaining Task Force members and revised for face validity [8]. Prior to distribution, the project was reviewed and approved by the University of Louisville IRB #21.0323.
The final survey was distributed electronically to attendees of the 2021 Virtual Academic Assembly, held April 11-15, 2021. To improve participation and completion, the survey was conducted in two parts [10]. Part 1 examined the impact of the pandemic on UME and GME. Part 2 examined the impact on faculty and on personal wellness. After the Academic Assembly, the survey was also distributed electronically to CORD membership via the subscribed CORD survey listserv. This survey was subsequently distributed again in May of 2021 to improve the completion rate, and the available survey was also highlighted in the July 2021 CORD electronic newsletter.
Analysis was both quantitative and qualitative. The quantitative analysis was performed using Statistical Product and Service Solutions (SPSS) (IBM SPSS Statistics for Windows, Version 26.0, Armonk, NY). The quantitative analysis was descriptive. We employed qualitative thematic coding mechanisms for all openended questions [11]. This analysis was descriptive and utilized the iterative development of a codebook [12]. Over four cycles, the group thematically analyzed the responses to the open-ended questions. Through this iterative process, responses were grouped thematically with the last cycle resulting in the coding of no more than three thematic categories per response. The presentation of qualitative results is primarily descriptive. Quotations were selected to communicate examples of the central themes for each category.

UME Quantitative Results
Benefits and challenges were identified by survey respondents ( Table 1). The highest-ranked benefit was the reduced financial burden related to decreased travel for rotations and interviews. Other highly ranked benefits included the use of virtual platforms and new educational modalities including asynchronous learning. Survey respondents felt that students having restrictions on clinical experiences was the most important challenge faced during this time period.

UME Qualitative Results
Analysis of the open-ended survey responses demonstrated several themes which were similar to those reflected in the ranking questions ( Table 2). Benefits included changes to the teaching experience, the use of remote platforms, and financial benefits. Overwhelmingly, the greatest challenges named by respondents were related to diminished clinical and educational experiences. Respondents also commented frequently on the negative impact on the acquisition of medical skills and knowledge and on the loss of personal connection.

GME Quantitative Results
Major benefits were improved faculty attendance and participation in virtual conferences, educational innovation, and easier attendance for virtual committee meetings ( Table 3). Other benefits were related to new educational modalities, including the decreased financial burden of using virtual platforms. The most prominent challenges facing GME during the pandemic were loss of personal educational interactions, difficulty for residents to form bonds with peers, and diminished learner participation during conferences.

GME Qualitative Results
GME members also provided substantial qualitative responses regarding the pandemic's impact on training EM residents ( Table 4). Many felt that the transition to remote learning platforms as well as innovation in the clinical environment was beneficial to EM resident learners. Twenty-six percent of respondents perceived an improvement in skills and knowledge, frequently in the areas of airway management, critical care, and disaster preparedness. Challenges in GME included impacts on wellness, disrupted clinical experiences, and loss of personal connection.

Description of Respondents
Forty-one individuals responded to the second part of the survey. Eighteen (43.9%) respondents were PDs, 14 (34.1%) were assistant/associate PDs, four (9.8%) were core faculty, four (9.8%) were clerkship directors, and one individual (2.4%) was involved with faculty development and research. Most respondents were white (87.8%) and women (61%).

Quantitative Results
As the majority (98%) of respondents were faculty, the faculty and personal wellness responses were combined below ( Table 5). Faculty noted the educational benefits and challenges of the new educational environment. Benefits included increased levels of faculty engagement, educational innovation, increased participation in committees, and financial savings. However, faculty also encountered challenges when adapting to the near-total virtual educational environment. The loss of personal connectivity and forming bonds with others was the main faculty challenge. On a personal level, respondents generally felt supported during the pandemic by family, colleagues, and the community and reported an ability to selfreflect. However, they simultaneously noted stress, burnout, and isolation as significant challenges.

Qualitative Results
The open-ended faculty responses identified some positive outcomes of the pandemic ( Table 6). Major benefits included the utilization of remote learning and virtual platforms, the convenience of remote teaching, and the increased ability to participate in educational sessions. Faculty also appreciated increased flexibility related to remote meetings and decreased commute time. Faculty also appreciated the opportunity to teach and acquire new skills and knowledge, such as procedural skills, critical care medicine, and telehealth. Some of the challenges included a sense of emotional insecurity, a general decrease in wellness, concerns for safety, and a loss of personal connectivity with others. Faculty acknowledged that a work/life balance was increasingly difficult to obtain because of the lack of boundaries in the virtual environment.

Discussion
As highlighted by the survey results, the pandemic's influence on EM education was unprecedented, dynamic, and viewed in both a positive and negative light. Similar themes were reported at all levels of medical education and were demonstrated by both quantitative and qualitative measures. The transition to virtual platforms had positive and negative impacts on UME, GME, faculty, and wellness. The virtual platform allowed educators to innovate their curricula, leading to novel uses and opportunities for engagement with technology. Remote attendance at didactic conferences made participation easier for students, residents, and faculty; further, the remote attendance benefit extended beyond engagement in didactics as it allowed residents and faculty to more easily participate in departmental meetings and gatherings without the time commitment for travel.
However, many felt that the move to remote communication led to a decreased sense of connection with peers and colleagues resulting in a mixed picture of overall engagement and effectiveness. Other studies have shown that residents have decreased engagement with lectures in the online format, and often attendees are engaging in increased non-lecture-related activities during lecture time including email, entertainment, self-care, social, or other personal activities [13,14]. While evidence-based techniques for increasing engagement such as team-based or case-based learning may help, these have not yet been studied in the virtual didactic environment. In addition, although actual attendance or remote meetings and conferences is more convenient, many felt remote meetings led to a sense of personal disconnectedness, including the loss of in-person socialization and professional engagement.
Perhaps one area of connectedness that was increased in the virtual environment was the ability to meet one on one with students and residents for advising. Faculty noted that they felt it was easier to connect with students and provide them with more advice and mentorship via a virtual platform. Given the mixed picture of the use of virtual technologies for education and mentorship, it remains to be seen whether virtual platforms and educational uses of technology remain in place after the pandemic.
The impact of the pandemic on clinical education was particularly profound at the UME level, where students were pulled from rotations to mitigate exposure to infection, resulting in a loss of clinical and educational opportunities. However, the shift in environment gave our educators the opportunity to innovate their curriculum and incorporate many previously underutilized tools during the periods of most in-person restrictions. Many programs developed virtual rotations in EM, allowing for innovation resulting in an increase in medical knowledge as well as knowledge in areas not often explicitly taught on clerkships, such as social EM [15]. Despite these benefits, it is recognized that virtual rotations are still not an adequate substitute for clinical learning, with the long-term potential impacts yet to be recognized.
The disruption in the clinical environment also impacted residents, although to a lesser degree. While some residents had modified rotation schedules, they often were redeployed to other clinical areas rather than being pulled completely from clinical interactions with patients. The changes in resident clinical schedules were seen as a source of stress and anxiety, however, the impact on resident clinical education was mixed. Both benefits and challenges were seen within clinical experiences and resident skill and knowledge acquisition. While COVID-19 brought increased skills in airway management, critical care, and disaster medicine, it also led towards a decrease in the breadth of training experience through a loss of rotations and a loss of focus and exposure to other disease processes that are essential to EM trainees. Additionally, the clinical impact seemed to vary based on both location of programs and the phase of the pandemic.
Personal wellness suffered in all three groups, and feelings of safety and security were decreased overall during the pandemic. Disruptions to schedule and inability to connect personally with residents created difficulty for GME leaders to be able to recognize and help manage residents suffering from post-traumatic stress disorder (PTSD), depression, and burnout. While technology was deployed to try and promote camaraderie within programs and departments, it was felt to be a poor replacement for in-person communication and connection. Both residents and faculty appreciated having more time to spend at home due to the elimination of in-person events and meetings, and the time used for commuting was instead used to spend time with family or pursuing other interests, projects, and hobbies, which led to a better work/life integration. However, faculty noted that there was a limit to the convenience of blurring home/work boundaries that could potentially lead to issues of feeling unbalanced, disconnected, and burned out. At the UME level, the inability to forge personal connections led to a decreased understanding of the culture of residency programs, which was a source of stress for medical students during the application cycle. However, the use of the virtual platform created opportunities for students to more easily attend interviews and decrease travel costs during interview season, which was a clear benefit. As we transition away from many of the restrictions implemented, we need to consider a balance between the value of in-person interviewing with the benefits of virtual interviews.
Each of the surveyed groups developed different countermeasures to the challenges of the pandemic. For faculty, the top countermeasures were improving communication, offering remote education and meetings, and putting in place educational and clinical safety measures. These countermeasures were similar to those found in Rodriguez et al. reporting that enhanced availability of personal protective equipment (PPE), rapid and easy access to testing, and clear communication were keys to decreasing academic faculty, resident, and fellow stress and anxiety [16]. In our survey, both UME and GME respondents mentioned countermeasures focused on education interventions, including using technology to increase participation and remote learning for flexibility and safety with the knowledge that further study is needed to evaluate techniques to encourage deep learning and engagement in virtual didactic sessions. The GME survey respondents also noted an increased need to focus on wellness and clinical scheduling adjustments for safety and increased clinical coverage.
Countermeasures to mitigate impacts on wellness (across all survey groups) included offering mental health services, increased time at home, and a focus on teamwork and camaraderie. While attention to mental health and wellness was highlighted during the pandemic, other studies have shown that there were high levels of burnout and PTSD in EM residents pre-pandemic [17]. This suggests that educators should continue to focus on offering mental health services and screening for burnout, depression, and PTSD in EM trainees as the pandemic subsides.
Finally, when directly asked about the role of CORD during the pandemic, we found that respondents reported a variety of needs from CORD as an organizational body. The majority, regardless of their area of focus, looked for support and to provide educational resources. Examples of this included helping respondents to meet Accreditation Council for Graduate Medical Education (ACGME) guidelines and offering platforms to share ideas and innovative educational interventions. UME and GME educators looked to CORD to take on leadership and advocacy roles in the response to the impacts of the pandemic. Examples included an expectation for CORD to help provide recommendations, best practices, and collaboration opportunities among EM residency programs and other organizations.
The most important limitation of our findings is the small sample size. Although the survey links were sent out through various modalities and on several occasions, our response rates were low. This could be due to a combination of survey fatigue and overall stress and fatigue brought on by the pandemic. Both the small sample size and the specific responses gathered may also be impacted by the timing of the survey. The survey was deployed during the early pandemic, which may have affected responses in comparison to another phase, as situations and resources have been extremely dynamic throughout this time period. Although we attempted to describe the learning and work environment for all EM faculty, residents, and medical students, we recognize that this is an extremely heterogenous population and each region experienced very different impacts on learning environments, patient volumes, and percentage of COVID-19 patients. Finally, this survey was only filled out by EM educators (faculty/CORD members), who answered from their perspective, rather than the perspective of medical students and trainees themselves. In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.