Foundational Elements of Applied Simulation Theory: Development and Implementation of a Longitudinal Simulation Educator Curriculum

Simulation-based education has gained popularity, yet many faculty members feel inadequately prepared to teach using this technique. Fellowship training in medical education exists, but there is little information regarding simulation or formal educational programs therein. In our institution, simulation fellowships were offered by individual clinical departments. We recognized the need for a formal curriculum in educational theory. Kern’s approach to curriculum development was used to develop, implement, and evaluate the Foundational Elements of Applied Simulation Theory (FEAST) curriculum. Needs assessments resulted in a 26-topic curriculum; each biweekly session built upon the previous. Components essential to success included setting goals and objectives for each interactive session and having dedicated faculty, collaborative leadership and administrative support for the curriculum. Evaluation data was collated and analyzed annually via anonymous feedback surveys, focus groups, and retrospective pre-post self-assessment questionnaires. Data collected from 32 fellows over five years of implementation showed that the curriculum improved knowledge, challenged thinking, and was excellent preparation for a career in simulation-based medical education. Themes arising from focus groups demonstrated that participants valued faculty expertise and the structure, practicality, and content of the curriculum. We present a longitudinal simulation educator curriculum that adheres to a well-described framework of curriculum development. Program evaluation shows that FEAST has increased participant knowledge in key areas relevant to simulation-based education and that the curriculum has been successful in meeting the needs of novice simulation educators. Insights and practice points are offered for educators wishing to implement a similar curriculum in their institution.


Introduction
Simulation-based education is increasingly used in medical education, yet physicians have very little formal training in delivering these programs [1]. Faculty members have expressed feelings of inadequacy in their ability to teach in a simulated setting and their perceived inadequacy has had a detrimental effect on student learning [2]. Several intensive simulation instructor courses have been developed to address the needs of the faculty; participation in these courses increases the perceived ability of the instructor to use simulation-based techniques to teach trainees [3]. However, the brief nature of these stand-alone courses implies that topics cannot be re-visited for further discussion. A longitudinal approach, with multiple sessions held over time versus stand-alone courses, may allow opportunities for a deepened understanding and application of the principles and techniques learned during the curriculum [4].
Many institutions have begun to offer medical education fellowship training, but there is limited information regarding simulation-specific fellowship training [5][6]. Steinert, et al. reviewed 53 papers describing postgraduate training in medical education, ranging from short workshops to longitudinal fellowship training [7]. Only 10% of the programs reviewed were longitudinal fellowship programs, none of which were focused on simulation-based medical education. Kotal, et al. identified 17 simulation-based fellowship programs in the USA; the program characteristics were described, but no details were provided on the existence of formal educational curricula [8]. Kinnear, et al. described a consensus-based approach to the proposed content and design of a simulation instructor curriculum, but they do not describe implementation of the course or program evaluation [9].
Kern's six-step approach to curriculum development is widely used in medical education and is outlined in Table 1 [10]. We describe, using Kern's framework, the development, implementation, and evaluation of the Foundational Elements of Applied Simulation Theory (FEAST) curriculum. Informed consent was obtained from all participants in this study.

Kern's Six Steps to Curriculum Development
Step 1: Problem Identification and General Needs Assessment Step 2: Learner Targeted Needs Assessment Step 3: Goals and Objectives

Specific objectives
Step 4: Educational Strategies Curriculum content

Educational methods
Step 5: Implementation

Problem identification and needs assessment
Development of the FEAST curriculum began as a series of informal discussions within the Department of Anesthesiology between the simulation fellowship director, simulation faculty, and their simulation fellow. It was felt that beyond hands-on experience, deliberate instruction on various topics in simulation-based education would enhance the experience of the fellow. As the Departments of Emergency Medicine, Obstetrics/Gynecology, Surgery and Anesthesiology began collaborating more actively, it became evident that several departments were launching simulation fellowship programs. All departments identified a need for formal training in educational theory in addition to practical training and were interested in collaborating to generate and deliver a longitudinal curriculum.
To determine curriculum content, needs assessments were conducted amongst current and past fellows, 12 multi-disciplinary simulation educators (representing anesthesiology, surgery, emergency medicine, obstetrics/gynecology, internal medicine, and critical care) and the senior leadership of the University of Ottawa Skills and Simulation Center (uOSSC). Topics taught in

Goals and objectives
The goal of the FEAST curriculum is to provide simulation fellows with a solid foundation in the theory and practice of simulation via interactive teaching about instructional design, curriculum development, and research as it pertains to simulation-based medical education. Each interactive teaching session included topic-specific objectives to be attained (see Appendix).

Educational strategies
Senior uOSSC staff and expert simulation educators from the Departments of Anesthesiology, Critical Care, Emergency Medicine, Obstetrics/Gynecology, and Surgery selected topics in which they had self-identified expertise. Sessions were given bimonthly from September to June of each year. Each interactive 90-minute session was followed by a facilitated 60-minute "Debrief the Debriefer" video review, where fellows could review videos of their own scenario debriefing with their colleagues and a faculty member. Fellows accumulated practical experience as an instructor during weekly simulation teaching sessions within their own specialty. In addition, four months into their fellowship, fellows facilitated simulation sessions outside their specialty area (for example, an anesthesiology simulation fellow would lead the obstetrics/gynecology simulation session). This innovative "fellow swap" allowed fellows to practice their debriefing skills in other content areas.

Curriculum implementation
Personnel and time were essential resources for the success of this curriculum. A designated director and an administrative coordinator were vital. The director was responsible for overseeing implementation and scheduling of the curriculum, while the coordinator was responsible for managing the schedule, evaluation forms, and communication with faculty and fellows. Other key personnel included faculty with simulation expertise and a desire to teach. The faculty needed time to prepare and present their talks, and the fellows needed protected time to attend talks and complete pre-assigned readings. With up to 18 faculty members and 4-10 fellows per year, an effective administrative and communication structure amongst stakeholders was crucial.
FEAST lectures were given at the simulation center in conference rooms with access to computers, internet, and projectors. For some lectures, access to the surgical skills laboratory, simulation mannequins, and audiovisual equipment were necessary.
At the time of our needs assessment, we recognized the continuing need for administrative and political support. At our university, the Faculty of Medicine's Medical Education Research/Innovation Unit is called the Department of Innovation in Medical Education (DIME). Their mandate is to promote educational innovation and scholarship. Partnering with DIME, which had an established fellowship in medical education and mutual interest in providing all fellows with training in learning theory, allowed for the allocation of further resources for administrative support. FEAST was incorporated as a mandatory component of the new DIME/uOSSC Fellowship in Medical Education/Simulation.
Additional support was given by the uOSSC, where senior management participated as faculty and provided access to physical space for training. Simulation educators supported the curriculum by providing their time and expertise in creating and delivering interactive presentations. The faculty recognized that their contributions to FEAST were mutually beneficial, as their time investment would be matched by contributions from colleagues towards the education of their fellows. All faculty received formal evaluations to be included in their teaching dossiers.
Several challenges were encountered during the implementation of this curriculum. Barriers and their solutions are shown in Table 3.

Barrier Solution
Suitable lecture timing Varying clinical schedules amongst different specialties made it challenging to find a suitable time.
Resolution of this issue was achieved via fellowship committee meetings and acknowledgement that FEAST was an essential fellowship component.

Absenteeism
Taking attendance, prompt notification and involvement of the FEAST and fellowship directors was effective at remediating this issue.

Inconsistent
"Debriefing the Debriefer" sessions Fellows reported that the weekly scheduled sessions did not occur regularly. Fellows taking a more active role, and reminders sent to faculty were effective strategies.

Research project initiation
Feedback from the fellows reflected their desire to have the research lectures earlier in the year. In subsequent iterations, these lectures were held over the summer months so that projects could be initiated earlier.
Increasing demand on enrollment In recent years, several requests have been received for ad hoc participation in the program. These requests were denied as the curriculum was designed to build upon preceding lectures and the interactive nature of the presentations required a cohesive small group setting.

Curriculum evaluation
The FEAST Curriculum was introduced in 2011. We collected evaluation data from 32 fellows over the first five years via anonymous feedback surveys, focus groups, and retrospective prepost self-assessment questionnaires. Institutional research ethics board review was waived as this qualified as program evaluation. A 5-point Likert scale was used and the mean scores are reported for selected categories.

Feedback Surveys
The fellows indicated that the FEAST curriculum was given at an "appropriate depth and level" (4.5 out of 5; 1=strongly disagree, 5=strongly agree), the "instructional format of the FEAST curriculum was appropriate" (4.4), and it provided "excellent preparation for a career in simulation-based medical education" (4.5). The fellows rated the faculty highly and felt they were "knowledgeable in the subject matter" (4.7), and effective in "clarity of communication" (4.5), "challenging my thinking" (4.4), and "stimulating enthusiasm" (4.6).

Pre-post Self Assessment Questionnaires
Retrospective pre-post self-assessment questionnaires showed that knowledge and skills in all categories improved. The areas that showed the greatest gains were in meta-debriefing (pooled mean change +2.6), debriefing (+2.5), and simulation centre operations (+2.5). The areas of least improvement were in conflict resolution (+1.4), knowledge of surgical skills props and moulage (+1.5), and knowledge of human factors (+1.6).

Focus Groups and Free-text Survey Comments
Themes arising from focus groups and free text comments on the feedback survey demonstrated that participants appreciated faculty expertise and the structure, practicality, and content of the curriculum. Suggestions for curriculum improvement centered on moving certain topics earlier in the year and proposed lecture topics for subsequent years ( Table 4).

Free-text Comments from Focus Groups and Feedback Surveys
Faculty expertise "Instructors were experts in their topics and had experience to share." "Discussions generated and moderated by multidisciplinary (health care) professionals." "Subject matter experts from multidisciplinary backgrounds … presented a rich portrayal of simulation education."

Discussion
Simulation-based education has gained popularity, yet it must be guided by sound educational theory and practice. The longitudinal nature of the FEAST curriculum allows fellows to gain a profound understanding of topics in educational learning theory underlying simulation-based education, research and techniques for developing and delivering simulation curricula. The success of the program can be attributed to dedicated faculty educators, explicit avenues of communication and strong collaboration between clinical departments and medical education units. FEAST graduates are equipped with the knowledge and skills to develop and conduct simulation-based educational interventions. Some graduates have become consultant staff at our institution; as new FEAST faculty, they give back to the curriculum and perpetuate its evolution. Other benefits of our longitudinal model include sustained networking and "crosspollination" between fellows and faculty at our institution. This collaboration has facilitated the development of additional interdisciplinary and interprofessional simulation-based sessions. Table 5 lists key practice points for educators wishing to implement a similar curriculum at their institution.

Key Practice Points for FEAST Curriculum Development and Implementation
A formal, longitudinal curriculum covering the fundamentals of simulation-based education provides a practical and theoretical foundation for the aspiring simulation educator.
Success requires collaborative leadership to engage multiple disciplines, administrative support and a critical mass of engaged, motivated, and experienced faculty.
Partnership with existing Medical Education Units ensures that educational activities are delivered according to adult-learning theory.
Ongoing benefits of this curriculum include the stimulation of interdisciplinary and interprofessional collaboration.
Graduates have become consultant staff at our institution and have joined the FEAST faculty thereby perpetuating its growth.

TABLE 5: Key Practice Points for FEAST Curriculum Development and Implementation
An important aspect of any educational intervention is evaluation of its effectiveness. Evaluation of this curriculum has been directed at level 1 of Kirkpatrick's model, satisfaction with and perception of the curriculum [7]. In the future, moving to higher levels with pre-and post-written and performance-based assessments (level 2, learning) and portfolios (level 3, behaviour) would further bolster credibility of the program.

Conclusions
We present the successful implementation of a longitudinal simulation educator curriculum that adheres to a well-described framework of curriculum development. The program evaluation shows that FEAST has increased participant knowledge in key areas relevant to simulation-based education and that the curriculum has been successful in meeting the needs of novice simulation educators. Future directions will involve demonstrating impact of the curriculum at higher levels of the Kirkpatrick's model.

Appendices FEAST Curriculum Themes & Session Learning Objectives
Orientation

Welcome to your Fellowship in Simulation & Medical Education
To explain the structure of the DIME Fellowship in Simulation/Medical Education To review the expectations of the DIME Fellowship in Simulation/Medical Education

Introduction to Audio Visual & Patient Simulation Equipment
To identify the capabilities and limitations of mannequin platforms To acquire a baseline knowledge of uOSSC capture and playback software/hardware To acquire a baseline knowledge of uOSSC audiovisual/information technology systems and demonstrate how to operate them

Introduction to Simulation Centre Administration & Operations
To explain the importance of a clearly defined simulation centre organizational structure To explain the roles and responsibilities of the uOSSC administrative staff and personnel To define the uOSSC funding structure

History of Simulation in Medical Education
To identify the key driving forces in medical simulation To describe how simulation has evolved over the last 40 years To name the key players and centres in medical simulation

Educational Theory and Simulation
To describe five components of "education" To describe the role and importance of "learning to teach" in academic centres To explain the key elements of a needs assessment, including the importance of triangulation, gap diagnosis, and its potential solutions To list program objectives based on concepts of cognitive psychology, clinical medicine, educational modalities, and assessment instruments

Conducting Programmatic Evaluation
To list the sequence of activities within your program To explain the steps in building a limited logic model

High Stakes Assessment in Simulation
To explain the factors involved in simulation for high stakes assessment To describe the application of simulation for high stakes assessment

Principles of Crisis Resource Management (CRM)
To define the principles of CRM To define two different approaches to error: personal and systemic To list four characteristics of highly reliable organizations and how they may apply toward error management To demonstrate the ability to analyze cases of error using the systemic approach To provide opportunities to discuss the progress of individual research projects To explain tips and tricks for research dissemination (manuscript writing, submission, and the peer review process)