Preliminary Reports in the Emergency Department: Is a Subspecialist Radiologist More Accurate Than a Radiology Resident?

doi:10.1016/j.acra.2006.11.001

Academic Radiology

Volume 14, Issue 2, February 2007, Pages 201-206

https://doi.org/10.1016/j.acra.2006.11.001 Get rights and content

Rationale and Objectives

To determine whether emergency department (ED) preliminary reports rendered by subspecialist attending radiologists who are reading outside their field of expertise are more accurate than reports rendered by radiology residents, and to compare error rates between radiologists and nonradiologists in the ED setting.

Materials and Methods

The study was performed at a large academic medical center with a busy ED. An electronic preliminary report generator was used in the ED to capture preliminary interpretations rendered in a clinical setting by radiology residents, junior attendings (within 2 years of taking their oral boards), senior attendings, and ED clinicians between August 1999 and November 2004. Each preliminary report was later reviewed by a final interpreting radiologist, and the preliminary interpretation was adjudicated for the presence of substantial discordances, defined as a difference in interpretation that might immediately impact the care of the patient. Of the 612,890 preliminary reports in the database, 65,780 (11%) met inclusion criteria for this study. A log-linear analysis was used to assess the effects of modality and type of author on preliminary report error rates.

Results

ED clinicians had significantly higher error rates when compared with any type of radiologist, regardless of modality. Within the radiologists, residents and junior attendings had lower error rates than did senior attendings, but the differences were not statistically significant.

Conclusion

Subspecialized attending radiologists who interpret ED examinations outside their area of expertise have error rates similar to those of radiology residents. Nonradiologists have significantly higher error rates than radiologists and radiology residents when interpreting examinations in the ED.

Section snippets

Setting

The Radiology Department at the University of Pittsburgh Medical Center has 52 radiology residents, and performs more than 1.4 million radiology studies per year. The ED at the main teaching hospital treats more than 50,000 patients per year and is a Level I trauma center. All of the attending radiologists who interpret examinations from this ED are subspecialized by body part. ED examinations performed during daylight hours (8:00 am to 5:00 pm), both on weekdays and weekends, are interpreted

Results

The log-linear analysis of error rates showed statistically significant variation (P < .05) for each association and for the saturation interaction. Thus no backward elimination was achieved.

Rates of substantial discordance are summarized in Table 1. The overall rate of substantial discordances was 2.2% (1,449/65,780). There was a strong correlation between author and modality (clinicians were more likely to place preliminary reports on CR examinations than on CT examinations). CT error rates

Discussion

This analysis of more than 65,000 emergency room preliminary reports indicates that subspecialist radiology staff who are interpreting examinations outside their area of expertise are no more accurate than radiology residents.

Radiology residents are educated best when they participate in overnight call without an attending radiologist immediately available (3). Residents in the overnight setting get exposed to a different mix of cases than they see during daylight hours, thus expanding their

Conclusion

Subspecialized attending radiologists who interpret ED examinations outside their area of expertise have error rates similar to those of radiology residents. Although there are many reasons, both political and practical, that may require attending radiologists in academic medical departments to provide in-house overnight coverage for the ED, the claim of improved quality of service is not valid for replacing on-call residents with subspecialist radiologists. Additionally, nonradiologists have

References (13)

T.B. Hunter et al.
Emergency department coverage by academic departments of radiology
Acad Radiol
(2000)
B.F. Branstetter
Are residents’ on-call errors worse than the alternatives?
J Am Coll Radiol
(2005)
J.M. McDonald et al.
Radiology residency call: residents’ perceptions of the on-call experience
J Am Coll Radiol
(2005)
P.N. Cascade et al.
Evaluation of competence in the interpretation of chest radiographs
Acad Radiol
(2001)
G.C. Velmahos et al.
Around-the-clock attending radiology coverage is essential to avoid mistakes in the care of trauma patients
Am Surg
(2001)
O. Ojutiku et al.
Radiology residents’ on-call interpretation of chest radiographs for pneumonia
Acad Radiol
(2005)

There are more references available in the full text version of this article.

Cited by (66)

Imaging Information Overload: Quantifying the Burden of Interpretive and Non-Interpretive Tasks for Computed Tomography Angiography for Aortic Pathologies in Emergency Radiology
2022, Current Problems in Diagnostic Radiology
Over the past decade, technological advances have provided new tools for radiologists. However, the effect of these technological advances on radiologist workload and detecting pathologies needs to be assessed.
The purpose of this study is to assess the workload, including non-interpretative tasks, associated with Computed Tomography Angiogram (CTA) of Aorta exams performed in the Emergency Department (ED) over a 10-year period and their relationship to detection of aortic pathology.
This is a retrospective analysis of CTAs of Aorta performed on adults with suspected acute aortic pathology within the ED at an academic level I quaternary care hospital from January 1, 2005, through December 31, 2015. Data assessed included (1) Interpretive tasks: total number of images, number of reformat series, number of radiology reports with positive aortic pathologies; and (2) Non-interpretative tasks: recommendations and documentation of verbal communication with requesting providers. Statistical analyses were performed to assess temporal trends of variables. P values less than 0.05 are considered significant.
A total of 4368 examinations (mean age: 69.8, M/F: 56.8%/43.2%) were performed. Studies per year increased significantly from 2005 (n = 278) to 2007 (n = 445), but not significantly after. The number of images and reformat series per scan increased from 487 to 2819 and 6.4 to 13.7, respectively (both P-value < 0.01). The proportion of exams with aortic findings did not significantly change (28.1% in 2005 and 24.9% in 2015). However, The proportions of exams with verbal communication increased from 9.3% to 24.6% and with recommendations from 1.8% to 28.9% (both P-value < 0.01).
During a 10-year period, CTAs performed in the ED for suspected aortic pathology were associated with a significant increase in images created, reformat series generated, recommendations, and verbal communications with ordering providers without a concomitant increase in the rate of aortic pathologies. To completely capture the complexities of CTA workloads, non-interpretive tasks such as radiologist recommendations and verbal communications should also be included.
Balancing the Scales: An Analysis of Social Determinants of Health, Radiology Report Acuity, and Radiology Staffing Models in an Academic Health System
2022, Journal of the American College of Radiology
Social determinants of health, including race and insurance status, contribute to patient outcomes. In academic health systems, care is provided by a mix of trainees and faculty members. The optimal staffing ratio of trainees to faculty members (T/F) in radiology is unknown but may be related to the complexity of patients requiring care. Hospital characteristics, patient demographics, and radiology report findings may serve as markers of risk for poor outcomes because of patient complexity.
Descriptive characteristics of each hospital in an urban five-hospital academic health system, including payer distribution and race, were collected. Radiology department T/F ratios were calculated. A natural language processing model was used to classify multimodal report findings into nonacute, acute, and critical, with report acuity calculated as the fraction of acute and critical findings. Patient race, payer type, T/F ratio, and report acuity score for hospital 1, a safety net hospital, were compared with these factors for hospitals 2 to 5.
The fraction of patients at hospital 1 who are Black (79%) and have Medicaid insurance (28%) is significantly higher than at hospitals 2 to 5 (P < .0001), with the exception of hospital 3 (80.1% black). The T/F ratio of 1.37 at hospital 1 as well as its report acuity (28.9%) were significantly higher (P < .0001 for both).
T/F ratio and report acuity are highest at hospital 1, which serves the most at-risk patient population. This suggests a potential overreliance on trainees at a site whose patients may require the greatest expertise to optimize care.
Bosniak classification of cystic renal masses version 2019 does not increase the interobserver agreement or the proportion of masses categorized into lower Bosniak classes for non-subspecialized readers on CT or MR
2020, European Journal of Radiology
In a condition so prevalent as renal cysts, classifications should display satisfactory reproducibility not only among subspecialized readers. We aimed to compare the interobserver agreement of the Bosniak classification version 2019 (BC19) and current Bosniak classification (CBC) for non-subspecialized readers on CT and MRI and to verify whether BC19 leads to a downgrade of renal cystic masses when compared to CBC.
We evaluated 50 renal cystic masses in 47 patients on MR and CT (25 per method). Eighteen readers (nine third-year radiology residents and nine abdominal imaging fellows) assessed the images using BC19 and CBC with an eight-week interval. Kappa statistic was used to assess agreement. An average score of Bosniak categories across all raters evaluated if there was downgrading of lesions on BC19.
The highest values of Kappa were found for fellows on CBC-MR (κ = 0.51), and the lowest values were found for residents on CBC-MR and fellows on BC19-MR (both κ = 0.36). On CBC, interobserver agreement was moderate for MR and CT (κ = 0.42 and 0.43, respectively), whereas on BC19, it was fair (κ = 0.38 and 0.40, respectively). The best agreements were in categories I (κ = 0.49−0.69) and IV (κ = 0.45−0.51). The poorest agreements occurred at IIF (κ = 0.18 on BC19-CT). There was a moderate median increase from CBC to BC19 in terms of Bosniak categories for both methods (MR [Z=-2.058, p = 0.040] and CT [Z=-2.509, p = 0.012]).
BC19, when compared to CBC, did not improve interobserver agreement nor diminished the proportion of masses categorized into lower Bosniak classes among non-subspecialized readers.
Tales from the Night:: Emergency MR Imaging in Pediatric Patients after Hours
2019, Magnetic Resonance Imaging Clinics of North America
Resident Case Volume Correlates with Clinical Performance: Finding the Sweet Spot
2019, Academic Radiology
To determine whether the total number of studies interpreted during radiology residency correlates with clinical performance as measured by objective criteria.
We performed a retrospective cohort study of three graduating classes of radiology residents from a single residency program between the years 2015–2017. The total number of studies interpreted by each resident during residency was tracked. Clinical performance was determined by tracking an individual resident's major discordance rate. A major discordance was recorded when there was a difference between the preliminary resident interpretation and final attending interpretation that could immediately impact patient care. Accreditation council for graduate medical education milestones at the completion of residency, Diagnostic radiology in-training scores in the third year, and score from the American board of radiology core exam were also tabulated. Pearson correlation coefficients and polynomial regression analysis were used to identify correlations between the total number of interpreted films and clinical, test, and milestone performance.
Thirty-seven residents interpreted a mean of 12,709 studies (range 8898–19,818; standard deviation [SD] 2351.9) in residency with a mean major discordance rate of 1.1% (range 0.34%–2.54%; stand dev 0.49%). There was a nonlinear correlation between total number of interpreted films and performance. As the number of interpreted films increased to approximately 16,000, clinical performance (p = 0.004) and test performance (p = 0.01) improved, but volumes over 16,000 correlated with worse performance.
The total number of studies interpreted during radiology training correlates with performance. Residencies should endeavor to find the “sweet spot”: the amount of work that maximizes clinical exposure and knowledge without overburdening trainees.
Radiologist Quality Assurance by Nonradiologists at Tumor Board
2018, Journal of the American College of Radiology
To explore the use of nonradiologists as a method to efficiently reduce bias in the assessment of radiologist performance using a hepatobiliary tumor board as a case study.
Institutional review board approval was obtained for this HIPAA-compliant prospective quality assurance (QA) effort. Consecutive patients with CT or MR imaging reviewed at one hepatobiliary tumor board between February 2016 and October 2016 (n = 265) were included. All presentations were assigned prospective anonymous QA scores by an experienced nonradiologist hepatobiliary provider based on contemporaneous comparison of the imaging interpretation at a tumor board and the original interpretation(s): concordant, minor discordance, major discordance. Major discordance was defined as a discrepancy that may affect clinical management. Minor discordance was defined as a discrepancy unlikely to affect clinical management. All discordances and predicted management changes were retrospectively confirmed by the liver tumor program medical director. Logistic regression analyses were performed to determine what factors best predict discordant reporting.
Approximately one-third (30% [79 of 265]) of reports were assigned a discordance, including 51 (19%) minor and 28 (11%) major discordances. The most common related to mass size (41% [32 of 79]), tumor stage and extent (24% [19 of 79]), and assigned LI-RADS v2014 score (22% [17 of 79]). One radiologist had 11.8-fold greater odds of discordance (P = .002). Nine other radiologists were similar (P = .10-.99). Radiologists presenting their own studies had 4.5-fold less odds of discordance (P = .006).
QA conducted in line with tumor board workflow can enable efficient assessment of radiologist performance. Discordant interpretations are commonly (30%) reported by nonradiologist providers.

View all citing articles on Scopus

View full text

Original investigationPreliminary Reports in the Emergency Department: Is a Subspecialist Radiologist More Accurate Than a Radiology Resident?

Rationale and Objectives

Materials and Methods

Results

Conclusion

Section snippets

Setting

Results

Discussion

Conclusion

Acad Radiol

J Am Coll Radiol

J Am Coll Radiol

Acad Radiol

Around-the-clock attending radiology coverage is essential to avoid mistakes in the care of trauma patients

Am Surg

Radiology residents’ on-call interpretation of chest radiographs for pneumonia

Acad Radiol

Original investigation
Preliminary Reports in the Emergency Department: Is a Subspecialist Radiologist More Accurate Than a Radiology Resident?