Failure of Real-time Passive Notification about Radiation Exposure to Influence Physician Ordering Behavior

Objectives To determine whether real-time passive notification of patient radiation exposure via a computerized physician order entry system would alter the number of computed tomography scans ordered by physicians in the Emergency Department (ED) setting. Methods When a practitioner ordered a computed tomography scan, a passive notification was immediately and prominently displayed via the computerized physician order entry system. The notification stated the following: the amount of estimated radiation in millisieverts (mSv), the equivalent number of single-view chest radiographs, and equivalent days of average environmental background radiation to which a patient during a specific computed tomography scan would be exposed. The primary outcome was changed in the number of computed tomography scans ordered when comparing data collected before and after the addition of the notification. Results Before the dosimetry notification (“intervention”) was instituted, 1,747 computed tomography scans were performed on patients during 11,709 Emergency Department visits (14.9% computed tomography scan rate). After the intervention had been instituted, 1,827 computed tomography scans were performed on patients during 11,582 Emergency Department patient visits (15.8% computed tomography scan rate). No statistically significant difference was found for all chief complaints combined (p = 0.17), or for any individual chief complaint, between the number of computed tomography scans performed on Emergency Department patients before versus after the intervention. Conclusions Passive real-time notification of patient radiation exposure displayed in a computerized physician order entry system at the time of computed tomography scan ordering in the Emergency Department did not significantly change the number of ordered scans.


Introduction
Use of all types of diagnostic radiologic examinations in Emergency Departments (ED) in the United States has increased ten-fold between 1950 and 2006, with an estimated 67 million computed tomography (CT) scans completed in 2006, and a 330% increase in CT utilization from 1996 to 2007 [1][2]. In 2010, 80 million CT scans were performed, with a projected yearover-year increase of 10% [1]. In large part because of the increase in CT scan utilization, potential adverse effects of ionizing radiation originating in health care settings are a growing concern, especially in the pediatric population [3]. It is estimated that up to 2% of all cancers may be caused by radiation from medical CT scanning [4].
For many reasons, which include increasing patient visits, efficiency, inpatient physician and patient expectations, and litigation avoidance, ED physicians are under increasing pressure to accurately evaluate, diagnose, and treat increasing numbers of patients in an expeditious and thorough manner. Implementation of computerized physician order entry (CPOE) systems assists physicians in adapting to these pressures by decreasing costs, shortening the length of stay, increasing the timeliness of critical test result reports, and decreasing medical errors in the inpatient setting [5][6].
CPOE facilitates a study of physician ordering practices, use of clinical decision rules, and overall knowledge. However, there has not yet been a study to evaluate whether CPOE systems utilizing passive notification (defined as the display of information without requiring acknowledgment) can decrease unnecessary CT scans, and therefore radiation exposure, in patients in the ED setting.
The purpose of the present study was to determine whether real-time passive notification of patient radiation exposure via a CPOE system at the time of order entry would alter the number of CT scans requested by physicians in the ED setting.

Study design and setting
The study was performed at a tertiary academic medical center with level-one adult and pediatric trauma designations, a dedicated pediatric ED, and an annual census of more than 62,000 visits. The ED is staffed by board-certified adult and pediatric emergency physicians, emergency medicine residents, and residents from other specialties (e.g., surgery, internal medicine, obstetrics and gynecology, and pediatrics). The study was determined to be exempt from informed consent by the Stanford Institutional Review Board.
The communication method to inform ordering practitioners of the estimated amount of radiation exposure for studies was to prominently post a notification via the existing CPOE system (EPIC, Verona, WI). This notification (the "intervention"), an example of which is shown in Figure 1, reported the amount of estimated radiation in millisieverts (mSv), equivalent number of single-view chest radiographs, and days of average environmental background radiation. The notification appeared immediately whenever a practitioner ordered a CT scan.

FIGURE 1: Example of Notification Bar as Displayed in EPIC
The notification, in this case for a CT scan of the head, states, "This study will expose your patient, on average to 2 (1.5-3.8) mSv of radiation. This is equivalent to 100 chest X-rays and 200 days of average background radiation." This study was powered to detect a 10% relative reduction in CT scan ordering or a 1.5% absolute reduction in CT scan ordering using a two-tailed significance level. This was based on a CT scan rate of 15 scans per 100 ED patients. To power and demonstrate this effect size, a sample size of 8,524 patients in each group was needed. A P-value of less than 0.05 was used to denote statistical significance. This number was chosen to be certain that the goal was met to have 80% power to detect a 10% relative drop in scan rates. The notification was added on May 17, 2012, to all login contexts within EPIC at the study hospital, including ED and inpatient settings. CT scan ordering data were then collected for 10,000 ED patient visits from March 10 to May 16, 2012. The increase in the number of subjects over the minimal number needed to power the study could be expected to increase power. Data was collected from the Stanford Translational Research Integrated Database Environment (STRIDE), which houses clinical information on patients seen at the study hospital since 1995.

Study population
Participants in this study included all ED patients who underwent CT scanning during the study period of March 10 to July 23, 2012. They included adult, pediatric, trauma, and pregnant patients. Patients were excluded if they did not undergo CT scanning.

Interventions and data collection and processing
To determine the estimated amount of radiation exposure for each study to be messaged to ordering physicians, radiation dosimetry data for each type of included CT scan were analyzed. These data were reviewed and analyzed by the study hospital's Department of Radiology. The method was to review 14 days of CT scan dosimetry data for each type of study ordered for patients in the ED. If fewer than 20 CT scans of a particular type were performed in the 14 day period, the period was continued until 20 CTs scans were completed. The mean amount of radiation in millisieverts (mSv) was calculated for each CT scan type. Because the dose of radiation administered to each patient for the same diagnostic test is not the same, doses were calculated to within confidence intervals of 95%. These determinations ( Table 1) served as the basis for the notifications placed in the CPOE system that informed ordering physicians of the amount of radiation to which each patient would be exposed by a particular type of CT scan.  Data were collected from the hospital's electronic medical record (EPIC). These data included radiologic tests ordered, estimated ionized radiation administered to patients, final patient disposition, and length of stay. Retrospective data were then extracted from the STRIDE database.
Patient nonclinical demographics were de-identified by STRIDE. Each patient was assigned a "de-identified patient number," and each visit was assigned a "de-identified patient encounter number." For each patient ED visit, the associated chief complaint, ICD-9 diagnosis code, admitting department, and CT scan ordered. For each CT scan, a "de-identified order number" was assigned and associated with a particular ED visit. Summary impressions, which are standardized codes that indicate the significance of the diagnostic findings contained within the study, were extracted for each CT scan.
Chief complaints from each visit during the study period were assigned to one of 20 chief complaint categories. Sixteen percent (3,224) of visits contained more than one chief complaint; these visits were assigned a single chief complaint based on the complaint most likely to have triggered the CT scan order. For 164 of these 3,224 ( Table 2) visits with multiple complaints, the assignment was based upon the actual CT scan ordered because more than one of the chief complaints could potentially have triggered the order. Six hundred and twenty-four visits did not have a chief complaint recorded; these visits were excluded from the overall analysis.

Outcome measures
The primary outcome of this study changed in the number of CT scans ordered when comparing data collected before and after the addition of the radiation dosimetry notification delivered at the time of order entry via EPIC.

Data analysis
To assess the impact of displaying dosimetry information, the rates of images ordered, both overall and as a function of chief complaint body system, before and after the intervention were assessed. For patients with multiple chief complaints, two authors (Lauren AP, Paul SA) reviewed the image orders and attributed each imaging study to a single complaint. Frequency counts of images done, both overall and by chief complaint and body system, before and after the intervention were calculated and the rates were compared using Poisson regression methods. A priori, the body system for which the study was ordered was deemed most likely to be impacted by the availability of dosimetry information and was therefore considered a primary endpoint. Other sites were considered as secondary. Therefore, these exploratory analyses were not subject to a correction for multiple comparisons. Tertiary analyses explored the overall rates for finding abnormalities. Rates of findings (categorized as non-diagnostic, no significant abnormality, previously reported abnormality, possibly significant abnormality, and critical result) before and after the intervention were assessed. Differences in these rates were evaluated using the Freeman-Halton extension of Fisher's Exact test. All analyses were done in SAS 9.4. Two-tailed p-values less than 0.05 were considered statistically significant.

Results
No statistically significant difference was found for any chief complaint between the number of CT scans performed on ED patients, before and after the intervention introduced by this study ( Tables 3, 4

TABLE 4: Number of CT scans performed before and after the intervention by chief complaint category
This table also shows the number of patients evaluated within each chief complaint category before and after the intervention. A p-value of <0.05 when comparing the rates of CT scans before and after the intervention was considered statistically significant.

Discussion
CT utilization in inpatient and outpatient health care settings, and especially in the ED setting, continues to increase. National CT use in EDs increased 330% from 1996 to 2007 [2,4,7]. At one United States tertiary care hospital from 2000 to 2004, CT use increased 27% in the outpatient setting and 48% in the inpatient setting, while ED CT uses increased 131% [7]. As a result of this increase in CT scanning, patients are receiving increasing doses of radiation from diagnostic and therapeutic medical imaging. The United States per capita annual effective radiation dose from medical procedures increased six-fold from 0.5 mSv in 1980 to 3.0 mSv in 2006 [1]. In our study, the average CT of the abdomen and pelvis exposed patients to 15 mSv of radiation, while the average CT angiogram imaging of the chest, abdomen, and pelvis delivered 26 mSv of radiation.
Multiple studies, based on radiation exposure rates of Japanese atomic bomb detonation survivors, have demonstrated a theoretical increased cancer risk attributable to exposure to even low-dose radiation. The Board on Radiation Effects Research VII report states that a single population dose of 10 mSv of radiation is associated with a lifetime attributable risk of 1 in 1000 of developing solid cancer or leukemia [8]. This dose of 10 mSv is well within the range of radiation doses delivered by modern CT scanners. Many patients have or will receive multiple CT scans during their lifetimes. For example, a patient with a history of pulmonary embolism (PE) who presents repetitively to EDs with symptoms suspicious for PE may undergo multiple chest radiographs and CT pulmonary angiogram studies. Each CT scan would expose the patient to on average 15 mSv of radiation, more than the amount that has been observed to increase the lifetime risk of cancer. Pediatric patients, who are ten times more sensitive to radiation than are adults, are undergoing more CT scans due to increased speed of CT scanning and decreased the need for sedation, among other factors [3,9].
Two studies have demonstrated an increased risk of radiation exposure, which is actual and rather not theoretical. Both studies included children and adolescents. The first assessed the risk from CT scan exposure in persons from birth to age 22 years in the United Kingdom from 1985 to 2002, excluding patients previously diagnosed with a malignancy [10]. This study found an excess relative risk of 0.036 per mSv for leukemia and 0.023 per mSv for brain malignancy [10]. The second study evaluated the risk from CT scan exposure in patients from birth to age 19 years in Australia and found a 24% relative higher cancer incidence in CT scan-exposed patients [11]. Average effective radiation dose in this study was 4.5 mSv, and an absolute excess incidence for all cancers was 9.38 per 1,00,000 person years at risk [11]. These studies and logic suggest that as CT scan utilization increases, so will the overall population risk of developing cancer from radiation exposure.
Several studies demonstrated that physicians consider CT overutilization to be a problem and desire decision support to guide ordering practices [12 -14]. Despite this desire and the fact that physicians are the persons ordering CTs, their knowledge of CT radiation doses and the associated cancer risk is not adequate [12,[15][16]. At the same time, physicians, especially emergency physicians, are under pressure to evaluate efficiently and accurately diagnose increasing numbers of patients. To improve efficiency, charting, and ease of ordering, CPOE systems embedded in electronic medical records (EMR) are instituted at many hospitals. CPOE systems have been shown to improve patient care and overall safety, being especially effective at improving provider adherence to guidelines [5][6]. CPOE systems also create the opportunity to embed standard-of-care guidelines to aid physicians in ordering and patient care. Despite this, few studies have evaluated the effects of adding guidelines for radiologic ordering to CPOEs. One recent study placed a passive reminder in its CPOE system to inform physicians if the patient had undergone five or more CT scans in the last 365 days; no significant change in the absolute number or rate of CT scan ordering was found [17].
In the present study, a passive notification was embedded into the order entry component of EPIC. This notification, at the time of a request for a CT scan, informed the attending and resident physicians of the quantity of radiation to which their patient would be exposed. We wished to evaluate if this specific message, method and timing of communication, and type of intrusion into the workflow process would influence ordering behavior. We did not observe a significant difference in the number of CT scans ordered from the ED related to the intervention.
The above study sufficiently powered to detect even a small difference in CT scan ordering between the pre-and post-intervention groups. We propose several reasons for not finding a statistically significant difference. First, the notification embedded into the CPOE system was passive rather than active. Attending and resident physicians were not required to acknowledge the notification before placing a CT scan order. Therefore, physicians may not have noticed or read the information, found it to be of little or no value at that particular moment (e.g., appeared too late) in their decision-making process, or decided that the benefits of ordering a CT scan outweighed the radiation exposure risks.
Our study was conducted at a level-one trauma center. Trauma patients overall undergo more diagnostic imaging, including more CT scans, and therefore are exposed to more radiation than are non-trauma patients [2]. In the current study, the CT scan rate for all patients with nontrauma-related chief complaints was 11.4% before the intervention and 11.1% after the intervention, while for trauma patients, it was 38% before the intervention and 40.1% after the intervention. In our hospital, we observe little or, more commonly, no discussion of radiation exposure during patient trauma treatment.
The current study showed that passively written notation about radiation exposure displayed to an ordering ED physician in the manner that we deployed did not decrease CT scan utilization. If we order too many tests, and some of these tests may reasonably be foregone, and if knowledge of radiation exposure might sway decision making, then we need to find a more effective way to interact with the ordering physicians. Algorithms designed to expedite workflow processes that include CT scans perhaps lead busy doctors attempting to manage a hectic ED away from contemplating the risk: benefit ratio of any individual test.
It is possible that mandatory acknowledgment of radiation exposure before placing an order will affect behavior. Behavioral interventions that include accountable justification and peer comparison have been shown to lower inappropriate antibiotic prescribing in the primary care setting [18]. As shown in this study, passive notification alone does not influence ordering behavior, active types of interventions deserve to be studied for CT scan ordering in the ED setting.
Several studies have demonstrated poor physician knowledge of radiation dosages from CT scans and the risks associated with this radiation. These studies also determined that physicians view CT scan overutilization as a problem and welcome guidelines to reduce CT scan ordering [12][13][15][16]. It is our observation, along with others, that ED physicians generally believe that advanced imaging is overutilized [19]. It is our empirical observation that they further believe that CT scans lead to unnecessary radiation exposure and increased health care costs. However, changing ordering behavior has not been solved.
If we wish to influence physicians to order fewer CT scans, for reasons of patient safety or otherwise, then we need to find a solution. Perhaps there is a more persuasive messaging technique that would cause them to order safer. The messaging could include, e.g., less radiation exposure, substitutes, e.g., ultrasound, magnetic resonance imaging, clinical observation, provide more precise and persuasive indications for CT scan ordering, e.g., promulgation of clinical decision support rules, and in a useful fashion. Such messaging would give practicing ED clinicians a fighting chance of integrating all of this knowledge into their standard workflow processes.

Limitations
Certain limitations may have influenced this study. It is possible that education and advertising needed to be conducted in advance of the intervention for it to become effective. It is possible that a more prominently displayed notification in the EMR might have been more effective. It is possible that the physicians might not understand the significance of radiation exposure as it was presented.
We were not able to control for the fact that while attending (faculty) physicians remain constant, different groups of residents rotate through the ED each month, including EM and off-service residents. It is possible that, had the entire physician group remained constant. This intervention might be effective after a sufficiently long enough period, that the providers became aware of and responsive to the information provided to them. It is possible that the two months pre-and post-intervention was too short a period to draw definitive conclusions. It may be that this intervention would require a longer period to become effective.
Patient volume and case mix in the ED fluctuates by time of year. Because the data collected in the pre-and post-intervention periods were not matched by time of year, it is possible that the type and number of patients in the ED at any given time may have influenced ordering behavior. This data matching is particularly important when considering the age of the patients as it relates to the propensity to order a CT scan.
Specific chief complaints were assigned to one of 20 chief complaint categories. Visits with multiple chief complaints were assigned to a single chief complaint category based on the complaint most likely to have triggered ordering of a CT scan. Our determination of these assignments may have influenced the number of CTs attributed to each chief complaint category. It is also possible that the assignments were incorrect, although we do not believe this to be the case.
It is possible that the intervention occurred too late in the decision-making process, perhaps after other clinical decision rules had been deployed to determine the need for a CT scan. We did not control for whether or not a clinical decision rule was used. Perhaps radiation exposure will not be effectively addressed unless it is a factor integrated into a clinical decision rule.
We did not control for the precise time of availability of other imaging modalities, such as MRI or ultrasound, but during the period of this study, modalities which acted as alternatives to CT scan were readily available.
This study was performed at an urban academic ED. It may not be generalizable to the community or rural hospitals, or to other inpatient or outpatient settings. As noted above, factors that might improve the ability of a "passive" notification system to be helpful in decreasing the number of CT scans ordered are: a stable staff, a different visual design (e.g., red flashing warning), mandatory acknowledgment or acceptance of the notification (accountable justification) before being allowed to proceed with placing a test order, periodic feedback to individual ordering physicians of their ordering profiles i.e., either in isolation or in comparison to peers, a different set of guiding information (perhaps to include the financial expense of the studies), or presentation of clinical pathways or decision rules (e.g., risks versus benefits).

Conclusions
Real-time passive notification of patient radiation exposure displayed in a CPOE system at the time of CT scan ordering in the ED did not significantly change the number of ordered scans. It remains to be determined whether or not there is a notification or other method that would