Disagreement in interpretation: a method for the development of benchmarks for quality assurance in imaging
Introduction
Radiologists do not currently have an objective benchmark for an acceptable level of missed diagnoses to meet hospital accreditation and proctoring requirements [1]. Also, there is no accepted measure by which to judge other nonradiologist physicians’ imaging interpretations. Government regulations, accreditation requirements, and the movement toward consumerism in the marketplace are placing ever increasing demands on the need to demonstrate quality. Residency training programs in radiology expose residents to studies in optical physics regarding perceptual errors. Although they also are familiarized with existing studies on rates of disagreement between multiple reads of the same film, there is a paucity of recent literature on the subject that is applicable to actual practice situations and that can serve as a credible benchmark. This study seeks to address that gap.
Section snippets
Sources of data
The International Radiology Group (IRG) is a radiology practice in Dallas, Texas, currently reading between 1200 and 1500 cases per day. It operates a highly automated, streamlined reading center. Since its inception, IRG has maintained a quality assurance (QA) program. Its cases come largely from outpatient settings in 26 states and include studies from nighttime emergency and off-hours coverage. Over the past several years, the center has developed its digital and film transmission
Results
Table 1 shows the overall disagreement rate, 3.5%, and disagreement rates by modality. Screening mammography had the highest disagreement rate, 5.8%, of all the modalities.
Table 2 shows disagreement rate by radiologist by modality disregarding whether the radiologist was the first or second reader on the case. It also shows actual versus expected overall disagreement rates for each radiologist. Comparing actual and expected disagreement rates overall across all modalities, radiologist I had a
Conclusions
This study reports the results of a QA program involving a double reading of 2% of more than 300,000 cases that produced statistically valid results. In this study, a 5% or less disagreement rate was recorded. Another study of similar magnitude [3] also reported disagreement rates in a similar range. We believe that our study provides important input toward building a credible benchmark that could be used to measure accuracy of interpretation of plain film, mammography and ultrasound.
Comparison with other studies
Acknowledgements
We would like to acknowledge the contributions of Brian Hall, vice president of operations at IRG, Dallas, Texas.
This study received assistance in statistical analysis from the ACR’s Technology Assessment Studies Assistance Program, which provided the professional services of Jonathan H. Sunshine, PhD, Mythreyi Bhargavan, PhD, and Rebecca Lewis, MPH.
References (9)
- et al.
Rates of disagreement in imaging interpretation in a group of community hospitals
Acad Radiol
(1998) Quality improvement in diagnostic radiology
Am J Roentgenol
(1990)Physicians’ Current Procedural Terminology
(2001)- et al.
Quality control in radiologyvariance analysis
(1999)
Cited by (74)
Impact of Shift Volume on Neuroradiology Diagnostic Errors at a Large Tertiary Academic Center
2023, Academic RadiologyNeuroradiology diagnostic errors at a tertiary academic centre: effect of participation in tumour boards and physician experience
2022, Clinical RadiologyCitation Excerpt :Reported error rates for diagnostic radiology range between 1.7–9%.1–11
Peer learning in breast imaging
2022, Clinical ImagingTransitioning From Peer Review to Peer Learning: Report of the 2020 Peer Learning Summit
2020, Journal of the American College of RadiologyCitation Excerpt :Score-based peer review was introduced to radiology nearly 20 years ago as a quality assurance mechanism for evaluating radiologist performance [1-3].