Development and National Validation of a Musculoskeletal Emergency Medicine Assessment Tool

Introduction Musculoskeletal (MSK) complaints and injuries are the fourth most common primary diagnosis in the emergency department in the United States (US). Despite the prevalence and economic impact on the US healthcare system, new emergency medicine (EM) residency graduates report feeling unprepared to treat MSK complaints. Currently, there are no reported means to assess MSK knowledge in EM resident physicians. The purpose of this study is to develop a validated and peer-reviewed multiple-choice assessment tool focused on MSK knowledge relevant to EM to allow us to better assess the knowledge of resident physicians. Methods A group of EM/Sports Medicine subject-matter experts (SMEs) created an initial list of the most important MSK topics in EM to generate a relevant question bank. The questions were validated by a different group of SMEs using a three-round modified Delphi method to obtain consensus on the importance of each question. Based on these results, the assessment was formed. Results From a list of 99 MSK topics, SMEs developed a final list of 37 MSK topics relevant to EM. Following round one, free-marginal kappa was 0.58, 95% CI [0.50, 0.66], with a moderate overall agreement of 71.95%. Following round two, the calculated free-marginal kappa increased to 0.88, 95% CI [0.83, 0.92], with an overall agreement of 91.79%. Using a five-point Likert scale, a threshold of an average score less than four was used to exclude questions in round three of validation and to create a final 50-question assessment tool. Conclusion Our proposed exam, titled Musculoskeletal Emergency Medicine Assessment Tool (MEAT), was successfully validated by experts in our field. It evaluates clinically important topics and offers a tool for assessing MSK knowledge in EM resident physicians. Future studies are needed to determine the feasibility of administering the tool and to establish a baseline score among different populations within the practicing field of EM.


Introduction
Musculoskeletal (MSK) complaints and injuries account for 20% of presenting chief complaints and are the fourth most common primary diagnosis in United States (US) emergency departments (ED) [1].Despite the prevalence, reports demonstrate a deficiency in MSK education in medical schools based on a validated MSK examination (FB-MSK) for graduating medical students [2][3][4][5][6][7] which may translate to incoming emergency medicine (EM) resident physicians.Comer et al. assessed EM attending and resident physicians at a Level 1 Trauma Center with the FB-MSK and reported that 35% of residents and 43% of attendings did not demonstrate proficiency [8].In addition, 23% of participants reported dissatisfaction with their MSK education.In 2017, new EM residency graduates reported not feeling well prepared to care for MSK complaints [9].
Various national organizations provide guidelines and resources to aid EM residency programs with MSK educational interventions.A national EM task force from eight recognized organizations updates the Model of Clinical Practice of EM every two years [10].This model describes EM clinical practice and is used by organizations, including residency programs, around the nation to structure core competencies and educational curricula including MSK education.However, the MSK section is limited, and the American Medical Society for Sports Medicine (AMSSM) recently recommended significant additions to the Model of Clinical Practice of EM and provided further curricular guidelines for MSK and Sports Medicine (SM) in EM residency [11].The EM Council of Residency Directors (CORD) published an online SM toolkit offering modules, videos, and case presentations that offer a "plug and play" SM curriculum for EM residency programs [12].
Despite curricular guidelines and educational modalities being recommended to address a possible deficiency in MSK education of EM resident physicians, there is no documented assessment tool to determine if a deficiency exists.In addition, if a perceived or objective deficiency in MSK knowledge does exist, there are no prior reports of barriers to implementing an educational intervention to address it.We recently performed a national needs assessment of EM residencies for MSK education and identified time, interdepartmental relations, and funding as the top three barriers [13].Most respondents believed their curriculum could be improved and they would utilize a standardized MSK assessment for EM residency training.Although validated MSK examinations exist, they were validated for medical students in 1998 (FB-MSK) and for medical trainees in a primary care context (MSK30) in 2019 [2,7].Although both are useful, the FB-MSK was developed over two decades ago for medical students, and the MSK30 was developed for medical students entering the outpatient primary care field.As there is no relevant MSK assessment for EM resident physicians, the purpose of this study is to develop a validated and peer-reviewed MSK assessment tool to assess EM resident physician MSK knowledge.

Materials And Methods
A modified Delphi technique by means of three rounds of validation was completed by subject-matter experts (SMEs) across the US.In 2021, 16 SMEs practicing in nine different states were recruited by author consensus.These SMEs were all dual board-certified in EM and SM and felt to best represent the intersection of EM and MSK medicine.Although MSK SMEs exist in other specialties, the recruited demographic has unique clinical experience in both the ED and outpatient MSK clinics.These SMEs were separated into two groups, Group A and Group B, to perform topic and assessment development and assessment review and validation.
A subsection of Group A compiled a comprehensive list of MSK topics deemed most clinically relevant to EM based on the Model of Clinical Practice and AMSSM EM Curricular Guidelines.Group A reviewed the list and voted to combine, add, or remove topics.Any topic that had a greater than 50% vote was modified.
Group A assigned a blinded five-point Likert scale (one -not at all important, two -less important, threesomewhat important, four -more important, five -very important) to all topics and provided a recommended number of questions for the assessment tool.Any topic with a Likert score less than four was initially excluded.
Following these results, Group A discussed the results and reviewed the topic list.Several topics that were initially excluded were added to the list if the majority agreed.Group A then assigned a percentage weight to each topic recommended for the assessment tool.Based on the average weight, each member of the group was randomly assigned topics to develop questions and was assigned to independently review and modify questions written by others within Group A.
Group B provided feedback through the modified Delphi technique with three iterative rounds.In the first and second rounds, Group B was asked if each question should be included, modified, or deleted with the option to provide suggested edits.A free marginal kappa coefficient was calculated following each round.In the third round, a blinded five-point Likert scale (one -strongly disagree, two -disagree, three -neither agree nor disagree, four -agree, five -strongly agree) was utilized to determine if each question was appropriate and clinically relevant to EM.This project was reviewed by the University of Arizona Institutional Review Board and approved as an exemption (2104683405, dated May 14, 2021).

Results
A total of 99 MSK topics were determined to be clinically relevant to EM (Table 1).After a topic review with Group A, 16 changes were recommended to the included topics list.Following a majority vote, eight topics were modified, and one topic was removed (sternum fracture).A total of 98 topics were ranked by each blinded SME in Group A. Of the 98 topics, 37 topics were determined to have a Likert importance scale equal to or higher than four.After a final discussion with Group A, three topics that did not make the list were added by majority vote.Four topics, "dermatome/myotome", "musculoskeletal radiograph interpretation", "musculoskeletal ultrasound interpretation", and "splinting", were agreed to be concurrent themes assigned to topics in the final list of 40 topics (Table 3).After question and assessment development, free-marginal kappa was calculated in rounds one and two until an overall agreement of 91.79% was achieved (Table 4).

TABLE 4: Calculated free-marginal kappa for rounds one and two of assessment development
In round three of the modified Delphi method, responses were compiled from all SMEs and an average score was calculated for each question.A final threshold of an average score of less than four was decided upon to exclude questions in round three of validation.After group discussion and consensus, the validated 50question assessment tool was created (see Appendices).

Discussion
Currently, there is no established gold standard for training and educating EM resident physicians in MSK knowledge.To our knowledge, this 50-question Musculoskeletal Emergency Medicine Assessment Tool (MEAT) is a novel assessment for EM developed through a rigorous process by an expert cohort.Through MEAT, EM residency programs can assess relevant MSK knowledge and utilize the results to address any notable deficiencies.The efficacy of residency-wide interventions such as an orthopedic or SM rotation or an MSK block in grand rounds can be evaluated through MEAT.We previously reported on pre-and postorthopedic rotation MSK knowledge acquisition through FB-MSK assessment but can replace the FB-MSK with MEAT to compare results [14,15].Incoming resident physicians can be evaluated as a baseline and then retested upon graduation to determine if MSK knowledge was gained.Evaluators can administer MEAT through online or paper distribution and assign a one-point score to each question for a total score of 50.Programs can also reference the topic list MEAT is based on to create their own questions or educational interventions.This should allow for the development of more individualized and robust curriculums for EM residents.
EM residency programs have a significant amount of physician tasks, medical knowledge, patient care, and procedural skills to learn over the course of a three-or four-year residency.Although MSK pathologies may not constitute a majority of critical conditions, one in five patients presenting to an ED in the US will have an MSK-related chief complaint [1].The ability to differentiate "sick" from "not sick" starts with a fundamental understanding of a domain, such as MSK.Family medicine identified this deficit in the outpatient setting and developed the MSK30, an assessment for graduating medical students and primary care resident physicians [7].With MEAT, EM educators and clinicians may have a starting point to identify deficiencies in MSK knowledge.Once deficiencies are identified, residency-wide or individualized solutions can be employed from published resources [11,14].Focusing solutions on identified deficiencies may save time in the residency curriculum, which was identified as a barrier to implementing the MSK curriculum [13].
The next step is to determine the feasibility of administering MEAT to EM resident physicians.Our goal is to assess this tool in a single and then multi-institutional study.Based on these results, we hope to establish a baseline score among EM resident physicians entering residency to help determine the success of MSK educational interventions.Subsequently, comparing MEAT to clinically relevant endpoints would allow for potential future iterations and improvements to the assessment tool.
Given MEAT was developed to assess MSK knowledge in EM resident physicians, it is possible to extrapolate and utilize this tool for board-certified EM physicians.The EM-relevant MSK topics chosen for MEAT would not change in clinical practice from resident to attending physician.It may even be used to reassess MSK knowledge after a period following board certification.
There are several limitations to this study.Although we recruited from multiple states across the country to incorporate different training backgrounds and practice patterns, we did not recruit a specific breakdown of SMEs (e.g., academic versus community physicians) nor did we randomly select SMEs from a prepopulated list.Although the decision to recruit dual-board certified EM/SM physicians was felt to be the most relevant to EM MSK knowledge due to clinical experience, other specialty SMEs could be involved in future studies.
The top 50 list of topics is still expert opinion, and therefore subject to bias and susceptible to future emerging topics.While SMEs were blinded to each other's responses, it is possible that they communicated with each other about their responses or discussed the study with an outside party.

Conclusions
Our Fight bite Flail chest Greenstick fracture Hamstring rupture High-pressure injection injury Figure 7

FIGURE 2 :
FIGURE 2: Anteroposterior radiograph of the wrist Image courtesy of Ian Bickle, Radiopaedia.org

FIGURE 3 :
FIGURE 3: Longitudinal ultrasound view of the left hindfoot Image courtesy of Matthew Negaard

FIGURE 4 :
FIGURE 4: Lateral radiograph of the wrist Image courtesy of Will Denq

TABLE 3 : Final list of musculoskeletal topics relevant to Emergency Medicine with a Likert importance score of greater than or equal to four, including percentage weight and number of questions assigned to each topic.
concurrent theme -topics assigned to be concurrent with other topics in the final list; SC: sternoclavicular; SCFE: slipped capital femoral epiphyses; AVN: avascular necrosis proposed assessment, the MEAT, fills a gap in the current EM MSK curriculum.It evaluates clinically important topics and offers a specific tool to assess clinical MSK knowledge in EM.Educators may be able to use this tool to develop further educational interventions.Future studies are needed to assess the feasibility of administration and establish a baseline score of MSK knowledge for different subject populations across EM practice.Harris classification would this child have?17.A 10-year-old female presents with wrist pain following a FOOSH off her bicycle.On exam there is swelling and tenderness over the physis of the distal radius.There is no tenderness to the anatomic snuff box.Radiograph is shown below (Figure2).What would be the suspected diagnosis and treatment?Figure2: Image courtesy of Ian Bickle, Radiopaedia.orgYouareevaluating a patient with atraumatic knee pain and no systemic symptoms.He has normal vital signs.He has an effusion and history of all fields.There are no open wounds of the chest wall noted.Chest radiograph below (Figure5).What is your immediate next step?