Quality Improvement Projects and Anesthesiology Graduate Medical Education: A Systematic Review

Quality improvement (QI) projects are essential components of graduate medical education and healthcare organizations to improve patient outcomes. We systematically reviewed the literature on QI projects in anesthesiology graduate medical education programs to assess whether these projects are leading to publications. A literature search was conducted in July 2023, using PubMed, Embase, and the Central Register of Controlled Trials (CENTRAL) for articles describing QI initiatives originating within the United States and applicable to anesthesiology residency training programs. The following data were collected: intervention(s), sample size (number of participants or events), outcome metric(s), result(s), and conclusion(s). One hundred and fifty publications were identified, and 31 articles met the inclusion criteria. A total of 2,259 residents and 72,889 events were included in this review. Educational modalities, such as simulation, training sessions, or online curricula, were the most prevalent interventions in the included studies. Pre-intervention and post-intervention assessments were the most common outcome metrics reported. Our review of the literature demonstrates that few QI projects performed within anesthesiology training programs lead to published manuscripts. Further research should aim at increasing the impact of required QI projects within the sponsoring institution and specialty.


Introduction And Background
The Accreditation Council for Graduate Medical Education (ACGME) establishes Common Program Requirements, or shared goals, for all specialty training programs.Since the addition of quality improvement (QI) projects to these requirements in 2012, residents have been expected to meet the following objectives upon graduation: demonstrate competency in analyzing the quality of care they provide, achieve organizational patient safety goals, prioritize activities and implement changes with the goal of care improvement, and evaluate the impact of interventions [1].In 2018, the Clinical Learning Environment Review (CLER) Program highlighted the underrepresentation of trainees on organizational QI committees and a discrepancy between resident projects and organizational quality of care goals [2].
The reasons for lack of resident engagement with organizational quality initiatives are likely multifactorial.Traditional QI methods, such as statistical process control (SPC) charts, require substantial time to collect data and properly evaluate an intervention.Still, these methods may not achieve organizational goals or generate positive results.Furthermore, because QI projects may focus on decreasing the incidence of rare events, collecting sufficient data may preclude completion of projects within the time constraints of residency training.
To evaluate relevant QI projects for United States anesthesiology trainees from the time of the ACGME common program requirement change, we conducted a systematic review of the literature.Our objective was to answer the following research question: In the United States between 2012 and 2022, what are the characteristics of QI projects that have progressed to publication?This review was not registered.

Population of Included Studies
A total of 31 studies were identified for data extraction, which included 2,259 residents and 72,889 events (variable of primary interest) .Residents accounted for, at least, a portion of the research or study team in 48.39% (15/31) of the studies [3,[5][6][7][8][9][10][11]13,15,16,[18][19][20][21]23,24,28,30].Other represented participants included anesthesiology attending faculty, as well as staff and trainees from otolaryngology, trauma surgery, emergency medicine, intensive care unit (ICU) nursing, nurse anesthesia students, nurses, and respiratory therapy [5,10,21,27,29].Additional data regarding the post-graduate years of residents were inconsistently reported, and the study participants were, on occasion, nonspecifically identified as "physicians"; thus, it was unclear how many attendings or residents were included among the participants.In the studies that included other professionals, residents were the most represented group at 42.94%, 65.52%, and 71.79% of participants [5,10,21].We allowed the inclusion of other specialties and professional groups because QI projects commonly involve multidisciplinary teams.

Incidence of nerve injury and local anesthetic systemic toxicity
Nerve injury 6-12 months: 3 vs 1, p = 0.003 Nerve injury >12 months: 1 vs 0, p = 0.24; seizure: 1 vs 0, p = 0.24 Nerve blocks are safe when performed by trainees with using ultrasound and nerve stimulator.More preoperative anesthesia visits are not an economically useful focus for the Perioperative Surgical Home.

Risk of Bias
Most non-randomized studies demonstrated some risk for bias as measured with the Risk Of Bias In Nonrandomized Studies-of Exposure (ROBINS-E) tool from confounding as demonstrated in the traffic-light plot shown in Figure 2 [34,35].The risk of bias for the only randomized controlled trial was assessed with the risk-of-bias tool for randomized trials (ROB-2) tool, as shown in Figure 3 [34,36].The randomized-controlled design leads to a low risk for bias due to confounding.The risk of bias due to missing data was common among all studies.The overall risk of bias was rated as 'some concern' for each of the included studies as presented in Figure 4 [34].

Discussion
According to the Association of American Medical Colleges (AAMC), there were 6,371 anesthesiology residents in 2021-2022 [37].Therefore, a large number of resident QI projects presumably do not progress to publication.This may serve as additional evidence of trainee underrepresentation in organizational quality of care goals as published by CLER [2].
Projects to improve knowledge are important components of residency training and education was the most represented intervention in the publications identified.These projects are relatively straightforward to design and complete and do not require complex statistical analyses.However, because these projects are commonly evaluated with surveys and questionnaires taken by providers, they are unlikely to be impactful with respect to broad organizational quality improvement goals, which are measured with patient outcome metrics.For anesthesiology departments to consistently provide high-quality patient care, educational interventions are required, but these projects, by themselves, are unlikely to produce objective, measurable improvements that healthcare organizations commonly desire.If the intent of the Common Program Requirements is to prepare trainees to grow as leaders in their respective healthcare systems, current efforts may not be consistent with that expectation.
A few published QI projects aligned with common organizational QI goals for patient care, OR and clinic efficiency and hand hygiene [4,14,17,18].QI projects in support of organizational goals are more likely to require lengthy data collection protocols, and traditional analysis methods may not sufficiently evaluate these goals within the time constraints of residency training.Substantial projects can be passed on between graduating resident classes, yet this practice may deprive residents of valuable experiences gained from attempting their own quality initiatives.
This review of the literature characterizes the quality improvement projects that have been published by departments with GME curricula in the United States since the inclusion of QI in the ACGME Common Program Requirements in 2012.Each project has been described with respect to the key participants of the project and the interventions and outcomes.These data highlight that published QI projects performed within United States departments with GME curricula tend to focus on educational interventions and usually do not comment on their respective organization's QI goals.

Limitations
The authors recognize several limitations in this systematic review.The search query used to identify articles may not be inclusive of all relevant studies.For example, projects conducted outside of the United States may be relevant to general anesthesiology quality initiatives, particularly those that originate in Canada, which has similar graduate medical education requirements.We decided against the inclusion of projects conducted outside of the United States in an effort to best characterize the impact of QI's inclusion to the Common Program Requirements by the ACGME (an organization that governs GME programs within the United States) in 2012.
PubMed, Embase and CENTRAL may not sufficiently account for all relevant studies.We did not include meta-analyses, systematic reviews or Cochrane reviews in our database searches.Although the inclusion of such publications may have identified additional relevant studies, we sought only primary research so that our search strategy could be replicated and to eliminate the subjective inclusion of studies referenced in other reviews.Again, not reviewing the references from other reviews may have limited the number of relevant articles identified.However, we suspect this objective search strategy in this study's design allows for replication.
Most non-randomized studies demonstrate a high risk of bias due to confounding, which is likely related to the observational design of many projects.All studies demonstrated 'some concern' for the risk of bias, thus serving as a limitation for qualitative synthesis of the results.
The absence of publications is not interchangeable with the absence of work.While there is a dearth of publications highlighting resident quality improvement projects, this should not be confused with a lack of high-quality work and may alternatively suggest that publications are not being submitted for other reasons (such as lack of novelty of the project or negative result).

Future Research
Future research should aim to enhance the impact of quality improvement projects.Statistics software has been used previously to simulate the results of quality initiatives prior to their implementation.For example, investigators attempting to reduce PACU lengths of stay utilized the Monte Carlo simulation to quantify the impact of eliminating administrative delays [38].In their study, 8.4% of patients experience an administrative delay leading to a prolonged PACU course (0.82 ± 0.58 hours vs. 0.29 ± 0.65 hours).Using baseline data, the authors created two hypothetical patient groups and simulated delayed and non-delayed PACU lengths of stay for 8.4% and 91.6% of patients, respectively.The Monte Carlo simulation allowed the investigators to predict the effect of their proposed intervention (elimination of administrative delays) to determine where resources should be dedicated based on the simulated effect (decreased PACU length of stay).Because the Monte Carlo simulation estimated total PACU time to decrease by less than 5% if administrative delays were eliminated, the authors sought alternative interventions with greater impacts.A similar method should be employed within the context of a graduate medical education program to maximize the impact of potential projects.

Conclusions
QI projects must be performed to meet residency training goals.In order for these same projects to also meet organizational goals, advanced planning potentially using simulation may also be required.Because the most readily published QI projects are designed to improve trainee or provider knowledge, these projects are unlikely to meet organizational quality goals, by themselves.

FIGURE 2 :FIGURE 3 :FIGURE 4 :
FIGURE 2: Traffic-light plot for the risk of bias of non-randomized studies

(First author) Intervention(s), observation(s), or initiative(s) Sample size Outcome metric(s) Selected result(s) Conclusion(s)
Table 1 displays the characteristics of the included studies.The substantial heterogeneity of the results precluded the metaanalysis.