A Brief Boot Camp for 4th-Year Medical Students Entering into Pediatric and Family Medicine Residencies

The transition from medical student to intern is a challenging process characterized by a steep learning curve. Focused courses targeting skills necessary for success as a resident have increased self-perceived preparedness, confidence, and medical knowledge. Our aim was to create a brief educational intervention for 4th-year medical students entering pediatric, family practice, and medicine/pediatric residencies to target skills necessary for an internship. The curriculum used a combination of didactic presentations, small group discussions, role-playing, facilitated debriefing, and simulation-based education. Participants completed an objective structured clinical exam requiring synthesis and application of multiple boot camp elements before and after the elective. Participants completed anonymous surveys assessing self-perceived preparedness for an internship, overall and in regards to specific skills, before the elective and after the course. Participants were asked to provide feedback about the course. Using checklists to assess performance, students showed an improvement in performing infant lumbar punctures (47.2% vs 77.0%; p < 0.01, 95% CI for the difference 0.2, 0.4%) and providing signout (2.5 vs. 3.9 (5-point scale) p < 0.01, 95% CI for the difference 0.6, 2.3). They did not show an improvement in communication with a parent. Participants demonstrated an increase in self-reported preparedness for all targeted skills, except for obtaining consults and interprofessional communication. There was no increase in reported overall preparedness. All participants agreed with the statements, “The facilitators presented the material in an effective manner,” “I took away ideas I plan to implement in internship,” and “I think all students should participate in a similar experience.” When asked to assess the usefulness of individual modules, all except order writing received a mean Likert score > 4. A focused boot camp addressing key knowledge and skills required for pediatric-related residencies was well received and led to improved performance of targeted skills and increased self-reported preparedness in many targeted domains.


Introduction
The transition from medical student to intern is a challenging process characterized by a steep learning curve as new doctors are suddenly faced with markedly increased clinical 1 2 2 2 responsibilities. These changes require strong communication and organizational skills as well as basic, practical medical knowledge [1].
The final year of medical school within the United States often consists of minimal required content and maximum freedom to choose amongst elective courses and non-clinical pursuits [2]. A few medical schools have instituted brief, mandatory courses before graduation, focused on communication, procedural and life skills, and the management of common medical emergencies. After participation in a three-week capstone course at the University of California, San Francisco, students felt better prepared in several areas including identifying sick patients, obtaining help, communicating, carrying out daily patient-care responsibilities, and maintaining their own well-being [3]. Students who participated in a week-long elective "Internship Boot Camp" course at the Mayo Medical School, consisting of a mix of simulation and problem-based learning sessions, recalled it as the most important aspect of medical school in preparation for the intern year [4]. Senior electives directed at students entering the field of surgery have been shown to increase knowledge and improve confidence in a wide variety of skills including procedures, education, and communication [5][6]. Boot camp-style courses have been demonstrated to increase participant knowledge, skills, and confidence in many fields of medicine [7]. No such courses have been described for students entering pediatric-based residencies, however.
While there are many required skills that overlap with other fields of medicine, pediatric residents will have the specific need to perform age-specific procedures as well as communicate with children and their families. Both immersive, case-based simulation and task/procedural training are particularly useful training techniques for graduating medical students. These can expose participants to realistic challenges and let them experience the consequences of their decisions and actions while providing a safe environment without risk to actual patients. We hypothesize that a focused course addressing the key knowledge and skills required for residency will help to graduate medical students entering into pediatrics, medicine/pediatrics, and family medicine feel better prepared for internship and, ultimately, improve patient safety and satisfaction for those cared for by interns who have completed the course. We conducted a pilot study to evaluate the curricular content and delivery.

Materials And Methods
Fourth-year medical students at the Northwestern University's Feinberg School of Medicine (FSOM) who matched into residencies in pediatrics, medicine/pediatrics, or family medicine were offered an ungraded, five-day elective entitled "Pediatric Internship Boot Camp" in the spring of 2013. The elective used a combination of brief didactic presentations, small group discussions, role playing, facilitated debriefing, and simulation-based education. This study was approved by the Institutional Review Board of the Ann and Robert H. Lurie Children's Hospital of Chicago (approval #2012-14872), and all participants provided written consent prior to participation.

Curriculum
Day 1 and 2 modules were taught in small groups using cased-based role-play with four students per trained facilitator (Tables 1-2). The students were given four pediatric cases each morning for use during all modules during the day. On day 3, students rotated through skill sessions in groups of four students per two instructors ( Table 3).  On day 4 of the course, students rotated through four immersive simulations in small groups. Each was followed by a debriefing session to discuss team dynamics, communication skills, and knowledge gaps around specific medical content. Simulations included a confederate "nurse" and "parent" to provide the history and a nursing student. An additional technician ran the simulations. After the debriefing period, groups were allowed to repeat the scenario to practice leadership, communication, and clinical skills. The four immersive simulations were: Septic Shock in a Child with Known Acute Lymphoblastic Leukemia (ALL): A high-fidelity pediatric simulator was used to present the case of a child with ALL admitted to the oncology service for fever. Clinical goals included identifying septic shock, obtaining the appropriate infectious workup for fever and neutropenia, initiation of appropriate fluid resuscitation in a child with septic shock, and delivery of appropriate antimicrobial therapy.

Respiratory Distress in an Infant with Bronchiolitis:
A high-fidelity infant simulator was used to present the case of an admitted infant with respiratory distress. The clinical goals included identification of respiratory distress, utilization of positioning and suctioning, and identification of different mechanisms for oxygen delivery and non-invasive positive pressure ventilation.

Status Epilepticus in a Febrile Neonate:
A high-fidelity neonatal simulator was used to present the case on a febrile newborn in the emergency department in status epilepticus. Clinical goals included identifying status epilepticus, providing medical management for seizures, recognizing respiratory distress, providing mag-mask ventilation, and identifying the necessary evaluation of a febrile infant.

Supraventricular Tachycardia (SVT) in a Non-Verbal Patient:
A high-fidelity pediatric simulator was used to present the case of a non-verbal child with fussiness and intolerance of feeds who was found to have SVT. Clinical goals included identification and management of stable SVT and anticipation and planning for the management of unstable SVT.
Self-reflection and peer feedback were encouraged and facilitated throughout the course. The final day of the elective was devoted to an individual objective structured clinical exam (OSCE) requiring students to use skills from the previous days to care for a patient throughout an emergency room visit. All students were provided with specific, detailed feedback at the end of the session.

Pre-Boot Camp OSCE:
In the week prior to the elective, students participated in an individual OSCE. The case required students to take a history from a facilitator acting as a standardized parent (SP), discuss the plan with the SP, and sign out the patient to another facilitator acting as a cross-covering resident. The SP completed the Gap-Kalamazoo Communication Skills Assessment (GKCSA) Peer/Facilitator form to assess communication. This instrument uses a five-point Likert scale (1=poor, 2=fair, 3=good, 4=very good, 5=excellent) to assess nine dimensions of communication (building a relationship, opening the discussion, gathering information, understanding the patient's perspective, sharing information, reaching agreement on problems and plans, providing closure, demonstrating empathy, and communicating accurate information) [11]. The second rater completed the I-PASS Study Verbal Handoff Assessment (ISVHA) [8]. Because only one handoff was being observed, the tool was adapted into a yes/no checklist rather than a fivepoint Likert scale assessing frequency.

Post-Boot Camp OSCE:
The course culminated with an individual OSCE. The students were given a novel case that required the synthesis and application of boot camp elements including communication with a trained SP, performing an infant LP and signing the patient out to a cross-covering resident. During the OSCE, a facilitator observed the students interactions with the SP remotely and acted as the cross-covering resident. The facilitator completed the GKCSA Peer/Facilitator form, the SP the GKCSA Parent/Family form, and the student the GKCSA Self form. The handoff was again assessed using the modified ISVHA. Lumbar puncture (LP) performance was reevaluated with the lumbar puncture checklist (LPC).

Preparedness Surveys:
All students matching into the targeted residencies were asked to complete an anonymous online survey to assess their self-perceived preparedness for an internship. They were asked to rate their agreement with the statement "Overall, I feel well prepared for internship" as well as assess their preparedness for specific tasks, skills and challenges encountered in residency using a five-point Likert scale (1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree). Surveys were distributed in the month before the boot camp with an identical survey sent in the fourth month of internship. Boot camp participants were also asked to complete the survey a week after participating in the course. Unique, self-assigned study codes were used so that individuals' responses could be compared across time points.

Student Feedback:
At the end of each day, students were asked to assess the course using a five-point Likert scale (1=strongly disagree, 2= disagree, 3=neutral, 4=agree, 5=strongly agree). For days 1-3, students were asked to provide assessments of the usefulness of individual modules, the balance between didactics and interactive components, the ability of the facilitators to present the material effectively, and whether they would use what they had learned in an internship. On day 4, they were asked to assess the usefulness of the individual cases as well as whether they felt that team communication skills had been addressed. They were also encouraged to give specific feedback throughout. At the end of the week, they were asked about their opinion of the elective overall.

Statistical analysis
Survey responses and OSCE data were analyzed using paired t-tests and repeated measure ANOVA. All tests used p < 0.05 as the significance level. Statistical analysis was performed using Stata (version 11.2, StataCorp, College Station, TX) and IBM SPSS (version 21, IBM Corporation, Armonk, NY).

Participation
Sixteen graduating medical students participated in the boot camp. Twelve were entering into categorical pediatric residencies (75%), three into family medicine (19%), and one into a combined internal medicine/pediatrics program (6%). Six were male. The students would become interns at 14 different institutions across the country with two remaining within our institution. Fifteen students participated in the post-boot camp OSCE (94%).

Resources
The boot camp required four facilitators for days 1-3. For rotating immersive simulations, two facilitators and one simulation technician were used for each case. The OSCE was accomplished with five facilitators and two standardized parents. An additional coordinator was present each day.
Annual costs include medical equipment, including LP trays and needles, sterile gloves, and endotracheal tubes. We used one high-fidelity pediatric simulator, one adult pediatric simulator, and two infant simulators. Two LP task trainers and two infant airway task trainers were also used. Costs for the use of our simulation center were not included as the medical school supports this facility. Our experience is that similar facilities would charge between $250-400/hour for the use of a typical simulation center for each simulation space used for immersive events and perhaps less for task training or room use, although this can vary from site to site.

OSCE results
For the 15 students participating in both pre-and post-intervention LP assessments, the mean pre-intervention LPC score on the checklist was 47.2% (SD 0.1), with a mean post-intervention score of 77.0% (SD 0.1) (p < 0.01; 95% CI for the difference 0.2, 0.4).
For the twelve students who participated in both the pre-and post-boot camp handoff assessments, the mean pre-intervention score on the modified ISVHA for completion of the five elements was 2.5 (SD 1.0). The mean post-intervention score was 3.9 (SD 1.

Survey responses
Fourteen boot camp participants (88%) completed both the pre-boot camp and immediately post-boot camp surveys. Thirteen participants (81%) completed the survey administered during their intern year. Eleven students (69%) completed all three surveys.
Participants reported an increase in self-perceived preparedness for many of the skills targeted and practiced during the course ( Table 4). Regarding the targeted skills, students did not report an increase in preparedness for communication with non-physician health professionals or obtaining a consult from another provider. We also did not see a statistically significant increase in the feeling of overall preparedness

Student feedback
Students overwhelming expressed satisfaction with the individual modules as well as the boot camp as a whole (Tables 5-6). The mean Likert score for the statement, "The module on (each individual module) was useful," exceeded 4.0 (4.1-4.9, SD 0.3-1.0) for all modules in days 1-3, except for order writing (3.9, SD 0.8). Fifty-eight percent of the modules had a mean score > 4.5.
The participants reported that all simulation-based sessions were useful with mean Likert scores of 4.8-4.9 (SD 0.2-0.4). The also reported that they learned about the presentation and medical management of the illnesses (4.7-4.8, SD 0.4-0.6) and team communication skills (4.8-4.9, SD 0.3-0.4) during these modules. In their free-text comments, the most common themes regarding the best aspect of the course included role-playing and skill practice.

Discussion
In Europe, it has been reported that first-year house officers feel unprepared for internship, specifically in basic clinical skills like prescribing, treatment, decision making, and emergency care [12][13]. Residency program directors within the United States report that interns commonly struggle with self-reflection and improvement, organization, application of knowledge, responsibility, and reliability. They feel these skills and attributes, along with communication, should be emphasized in the fourth-year medical school curriculum [14].
There are no reports of boot camps designed specifically for medical students entering into pediatric-based residencies. Using a combination of standardized patients, low-fidelity models, task trainers, and immersive simulations, we allowed students to practice a variety of skills before they are required to use them with actual patients, families, and other medical providers. We found our model to be feasible, both economically and resource-wise, within our institution.
Participants demonstrated significant improvements in infant LP performance as assessed by the LPC. This supports previous evidence that skill improves after an educational intervention, but the translation to actual clinical care by our students is not known at this time [15].
Research evaluating residents' ability to perform LPs successfully on infants after various educational interventions has been mixed [16][17].
Participants also demonstrated improved adherence to the I-PASS handoff tool after the course, as assessed by the modified ISVHA. Previous studies have shown that medical students adhere to handoff guidelines, both immediately after a targeted educational intervention as well as during future clinical rotations [18]. A multi-centered investigation has demonstrated a decrease in preventable medical errors after widespread implementation of the I-PASS tool [19]. Participants in our course may work in locations that use a variety of handoff tools that may or may not be standardized. Our hope is that by learning a commonly accepted tool during the boot camp, they will, at least, have an understanding of the importance of a safe and complete handoff and knowledge of common key components.
We did not see an improvement in communication using the GKCSA. Student scores during the pre-course OSCE were overall high. The lack of improvement likely demonstrates a ceiling effect on this portion of the assessment, as students were highly rated in their pre-intervention assessments. Additionally, the tool may not have the sensitivity to evaluate for subtle changes in regards to communication about difficult topics, such as the need for invasive procedures.
Participants subjectively reported feeling more prepared to perform tasks related to the skills taught during the elective with the exception of obtaining consults and interprofessional communication. We feel that this change was due to the curriculum as we did not see an increase in Likert scores addressing material not specifically addressed during the boot camp (teaching medical students, giving medical students feedback, documentation, working with sick children, handling the consequences of making a medical error, and managing work-life balance). Interestingly, we did not see a significant increase in overall self-reported preparedness suggesting there are other skills or knowledge that students may feel apprehensive about.
The students demonstrated a high level of engagement in the course and felt that the individual modules, as well as the course as a whole, were a valuable experience.
Limitations of this study include the inability to follow long-term outcomes related to intern performance. Because students moved to various institutions, it was not possible for us to compare them to peers in an unbiased manner. Although we attempted to contact all graduates entering into the targeted residencies to participate in the follow-up survey, we had a very poor response from people who did not participate in the boot camp. We were, therefore, unable to compare self-reported preparedness between students who did and did not participate in the boot camp during their intern year. Given the elective nature of the course, we had a small sample size of participants. There is also likely selection bias given that the students selfselected to participate. All evaluations were unblinded and performed by instructors who had been working with students throughout the course, which may have introduced bias into the evaluation process. Finally, the assessment tools, including the LPC, ISVHA, and GKCSA, have all been used to evaluate medical students and/or residents in other contexts but were not specifically validated in our population.
Future directions include continuing to revise the curriculum based on student feedback and assessment outcomes. We would like to continue to gather data from graduated participants of the boot camp and their program directors in order to continue to focus and develop the elective.

Conclusions
A brief boot camp-style course for graduating medical students entering into pediatric or family medicine internships may lead to improved self-confidence and skills. Time constraints may be less severe at the end of medical school compared to the beginning of an internship, making this an ideal time to engage students in preparatory, hands-on learning prior to transitioning roles.