Risk Factors for Perceptual-versus-Interpretative Errors in Diagnostic Neuroradiology
=====================================================================================

* S.H. Patel
* C.L. Stanton
* S.G. Miller
* J.T. Patrie
* J.N. Itri
* T.M. Shepherd

## Abstract

**BACKGROUND AND PURPOSE:** Diagnostic errors in radiology are classified as perception or interpretation errors. This study determined whether specific conditions differed when perception or interpretation errors occurred during neuroradiology image interpretation.

**MATERIALS AND METHODS:** In a sample of 254 clinical error cases in diagnostic neuroradiology, we classified errors as perception or interpretation errors, then characterized imaging technique, interpreting radiologist's experience, anatomic location of the abnormality, disease etiology, time of day, and day of the week. Interpretation and perception errors were compared with hours worked per shift, cases read per shift, average cases read per shift hour, and the order of case during the shift when the error occurred.

**RESULTS:** Perception and interpretation errors were 74.8% (*n* = 190) and 25.2% (*n* = 64) of errors, respectively. Logistic regression analyses showed that the odds of an interpretation error were 2 times greater (OR, 2.09; 95% CI, 1.05–4.15; *P* = .04) for neuroradiology attending physicians with ≤5 years of experience. Interpretation errors were more likely with MR imaging compared with CT (OR, 2.10; 95% CI, 1.09–4.01; *P* = .03). Infectious/inflammatory/autoimmune diseases were more frequently associated with interpretation errors (*P* = .04). Perception errors were associated with faster reading rates (6.01 versus 5.03 cases read per hour; *P* = .004) and occurred later during the shift (24th-versus-18th case; *P* = .04).

**CONCLUSIONS:** Among diagnostic neuroradiology error cases, interpretation-versus-perception errors are affected by the neuroradiologist's experience, technique, and the volume and rate of cases read. Recognition of these risk factors may help guide programs for error reduction in clinical neuroradiology services.

Medical errors are a substantial cause source of patient morbidity and mortality.1 Given the important role that diagnostic imaging plays in patient management, errors in diagnostic imaging interpretation likely contribute to this burden of medical error.2 To improve patient care, radiologists should analyze errors to identify conditions that may alter the risk of future errors. Such analysis may uncover contributory factors that can inform appropriate systematic corrective steps and improve the overall quality of patient care.3⇓–5

Errors in diagnostic radiology take 2 broad forms: errors of perception and errors of interpretation. A perception error occurs when a radiologist fails to identify an abnormality that is clearly present to most readers on a diagnostic image (false-negative errors). An interpretation error occurs when a radiologist identifies a finding on a diagnostic image but misinterprets its significance or meaning. Interpretation errors may take several forms such as attributing the wrong diagnosis to a finding, misjudging the severity of a finding, interpreting a normal finding as an abnormality, or interpreting an abnormal finding as a normal finding.

Prior studies of errors in radiology demonstrated that errors of perception compose most radiologic errors (60%–80%).6⇓–8 Little is known about potential risk factors that might predispose radiologists to perceptual-versus-interpretive errors. We hypothesized that the imaging technique, specific imaging finding, underlying diagnosis, radiologist's experience, and environmental conditions would influence the type of error made by neuroradiologists. Our study characterized whether case-specific metrics differed between perception and interpretation errors among a cohort of recognized neuroradiology error cases from a large academic medical center.

## Materials and Methods

### Case Selection and Categorization

This study was Health Insurance Portability and Accountability Act–compliant and was approved by the local institutional review board.

Cases of diagnostic errors were collected by staff and trainees in the neuroradiology division at a large tertiary care medical center as part of an ongoing practice-quality-improvement initiative. Each attending physician, fellow, and rotating resident in the neuroradiology division was instructed to place any error case they identified during clinical practice into a specified digital folder in our PACS. This requires a single-click drag-and-drop function facilitating addition of relevant cases to the cohort. A monthly divisional “M&M” conference was held where error cases were presented in a confidential manner, and the error was adjudicated by consensus among board-certified staff neuroradiologists.

Cases for this study accrued from July 2014 to January 2016 (19 months) during which time 75,804 CT and MR imaging neuroradiology examinations were interpreted at our institution. The cases and radiologist reports were retrospectively reviewed by a neuroradiology fellow and 2 board-certified attending neuroradiologists (with 5 and 7 years of experience, respectively) confirming that an error had occurred by consensus, with incorporation of any relevant clinical information available in the electronic medical record or PACS before and subsequent to the case in question. Errors were defined as findings or interpretations that both neuroradiologists thought should have been in the impression of the radiology report and that both neuroradiologists agreed would potentially change clinical management (eg, missed aneurysm, but not inflammatory maxillary sinus changes in an immunocompetent patient). Because not all cases have criterion standard final diagnoses (such as surgical pathology distinguishing meningioma from schwannoma), for interpretation errors, we emphasized missing differential diagnoses that both neuroradiologists thought should have been included in the radiology report. A total of 254 errors were discovered. Each error was categorized as either a perception error (the relevant finding was not identified) or an interpretation error (the relevant finding was identified, but not interpreted correctly) (Figs 1 and 2).

![Fig 1.](http://www.ajnr.org/https://www.ajnr.org/content/ajnr/40/8/1252/F1.medium.gif)

[Fig 1.](http://www.ajnr.org/content/40/8/1252/F1)

Fig 1. 
Perception error. A 55-year-old woman undergoing brain MR imaging for multiple sclerosis. *A*, Contrast-enhanced T1WI (using a 3D magnetization prepared rapid acquisition of gradient echo sequence) reveals an incidental aneurysm of the right internal carotid artery (*arrow*), which was not identified on this examination. It was detected on subsequent MR imaging. *B*, A cerebral angiogram confirms the aneurysm (*arrow*).

![Fig 2.](http://www.ajnr.org/https://www.ajnr.org/content/ajnr/40/8/1252/F2.medium.gif)

[Fig 2.](http://www.ajnr.org/content/40/8/1252/F2)

Fig 2. 
Interpretation error. A 17-year-old adolescent boy undergoing brain MR imaging for headache. A round mass (*arrows*) was identified in the right parietal lobe, demonstrating peripheral hypointense “blooming” on the T2*WI (*A*) and both central and peripheral contrast enhancement on the contrast-enhanced T1WI (*B*). The MR imaging finding was detected but incorrectly interpreted as representing a cavernous malformation. *C*, A subsequent angiogram revealed that this was, in fact, a partially thrombosed aneurysm arising from the right middle cerebral artery (*arrow*). (In *A* and *B*, there are artifacts from the patient's braces).

Each error case was categorized by the imaging technique (CT, MR imaging, radiograph), anatomic location of the relevant finding (cerebral gray matter, cerebral white matter, infratentorial brain, vasculature, meninges/ventricles, calvaria/scalp, face/skull base, neck or spine/paraspinal), and etiology (neoplastic, vascular/ischemic, congenital, trauma, degenerative, infection/inflammation/autoimmune, or other). The diagnoses were tabulated for each case. The experience level of the radiologist who made the error was noted (trainee; non-neuroradiology attending physician; attending neuroradiologist with ≤5 years, 6–10 years, 11–20 years, or >20 years of experience). There were 12 attending neuroradiologists who made at least 5 errors included in this study, and each attending neuroradiologist experience level comprised exactly 3 such attending neuroradiologists. The remaining attending neuroradiologists who made <5 errors included in our study were part-time or per diem neuroradiologists, neuroradiologists who previously worked at our institution but have since left, and community neuroradiologists affiliated with our institution. Trainee errors were only included if the trainee created and signed a “preliminary report” that was available in the electronic medical record for viewing by the referring clinicians. The cases were classified as those reported during regular weekday hours (Monday through Friday, 8 am to 5 pm) and those reported at all other times (ie, on-call situations). Characteristics of the specific radiologist's work shift when the error occurred were available in a subset (*n* = 217) of the cohort and included the number of cases read during the total shift, the shift length (in hours), and the average number of cases read per hour during that shift. Finally, the sequence in which the error case occurred during the shift was recorded (ie, was it the fifth case of the day or the 30th?).

### Statistical Analysis

Perception errors and interpretation errors were summarized by frequencies and percentages. Logistic regression was used to examine whether interpretation-versus-perception errors were systematically related to the neuroradiologist's experience, imaging technique, anatomic location, disease etiology, day of the week, and time of day. The logistic regression outcome variables were binary variables that were assigned the value 1 if the diagnostic error was classified as an interpretation error and value 0 if the diagnostic error was classified as a perception error. We tested the null hypothesis that there was no systematic relationship between the categories of the predictor variable and the log-odds of cognitive error. The Wald statistic served as the pivotal quantity of the null hypothesis test, with *P* < .05 defined a priori as a statistically significant difference. The confidence interval was based on the Wald large-sample approximation method. Additionally, 2-tailed *t* tests at the *P* < .05 significance level were performed to compare the hours worked per shift, cases read per shift, cases read per shift hour, and the order of the error case during the shift, between perception and interpretation errors.

## Results

A total of 254 recognized diagnostic neuroradiology errors were included. During the same time as error collection, approximately 75,000 diagnostic neuroradiology CT and MR imaging studies were performed at our institution. There were a total of 190 perception errors (74.8%) and 64 interpretation errors (25.2%). The On-line Table lists the errors by various categories. The most common diagnoses among the errors were the following: metastasis (*n* = 23), infarction (*n* = 20), vascular thrombus (*n* = 14), intracranial aneurysm (*n* = 14), and malformation of cortical development (*n* = 11).

There was no association among the 3 radiologist subgroups, trainees, non-neuroradiology attending physicians, and attending neuroradiologists for interpretation-versus-perception errors (*P* = .57). We note that during the shifts when errors occurred, the trainees (versus attending physicians) worked slightly longer hours per shift (9.0; 95% CI, 7.14–10.86 versus 7.3; 95% CI, 6.92–7.68; *P* = .04) and read fewer cases per shift hour (3.83; 95% CI, 2.91–4.74 versus 5.88; 95% CI, 5.53–6.24; *P* < .001). However, we found no significant difference in the average total cases read per shift (33.5 versus 41.3; *P* = .22) and sequential error occurrence during the shift (14th case versus 23rd case; *P* = .08) between trainees and attending physicians.

Examining only attending neuroradiologists did not reveal a statistical association between attending physician experience and interpretation-versus-perception error, when attending physician experience was categorized as 1–5 years, 6–10 years, 11–20 years, and >20 years of experience (*P* = .18). However, a post hoc comparison did reveal that the errors made by those with ≤5 years of experience were more likely to be interpretation errors (OR, 2.09; 95% CI, 1.05–4.15; *P* = .04) compared with the errors made by those with >5 years of experience.

There were 161 MR imaging cases (including 20 MRA/MRV), 89 CT cases (including 9 CTA/CTV), and 4 plain film cases. Errors associated with MRI examinations were more likely to be interpretation errors (OR, 2.10 [1.09, 4.01], *P* = 0.03) compared to errors associated with CT examinations. There was a significant association between interpretation-versus-perception error and disease etiology (*P* = .04), driven by the infectious/inflammatory/autoimmune category of which interpretation errors represented 52% of the errors (for all other disease categories, interpretation errors represented <30% of total errors).

There was no significant association between interpretation-versus-perception errors and the anatomic location of the relevant imaging findings (*P* = .48). There was no association between the type of error and the time of the week when the case was read (comparing routine weekday hours with all other times, *P* = .92).

Perception errors were associated with a 16% higher mean number of cases read per shift (42.5; 95% CI, 39.0–46.0 versus 36.7; 95% CI, 32.0–41.3; *P* = .05) and a 19% higher mean number of cases read per shift hour (6.01; 95% CI, 5.6–6.4 versus 5.03; 95% CI, 4.5–5.5; *P* = .004). There was no difference in the mean total hours per shift between the occurrence of perception errors or interpretation errors (7.3; 95% CI, 6.9–7.7 versus 7.6; 95% CI, 6.8–8.5; *P* = .45); however, perception errors did occur sequentially later during the shift (23.5; 95% CI, 20.8–26.2 versus 17.9; 95% CI, 13.7–22.0 previous cases read; *P* = .04).

## Discussion

We characterized the relative proportion of perceptual and interpretive errors and the underlying conditions associated with these errors among a cohort of known, recognized error cases interpreted predominantly by academic subspecialty-trained neuroradiologists. To our knowledge, this is a novel approach to studying diagnostic error. Prior studies of diagnostic errors in neuroradiology have focused on determining the rate and significance of errors when interpreting emergency imaging studies9⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓–21 or discrepancy rates between general practice radiologists and academic subspecialists.22⇓⇓–25 Discrepancy rates can be misleading because the incidence can be confounded by discovery or recall bias and were not the focus of this study.

Our results identified certain specific case, neuroradiologist, and reading situation features that were associated with interpretative or perception errors in neuroradiology practice. Less experienced neuroradiologists made a higher proportion of interpretation errors, while more experienced neuroradiologists made a higher proportion of perception errors. Errors were more likely to be perceptual with CT studies and interpretative with MR imaging studies. Errors were more likely to be interpretative during evaluation of patients with infectious, inflammatory, or autoimmune diseases. Perception errors were associated with a higher case volume, faster reading rates, and later parts of a work shift compared with interpretation errors.

Our results regarding imaging technique should be intuitive to most radiologists. Compared with CT, MR imaging is known to produce far greater sensitivity for many common pathologic processes seen on neuroimaging studies (eg, acute infarcts,26 cerebral neoplasms,27 demyelinating disease28), which should decrease the number of perception errors among MR imaging cases. Conversely, MR imaging has more artifacts, there are more MR imaging sequences per study to evaluate, and the greater complexity inherent to MR imaging findings relative to CT all might have contributed to the relatively higher proportion of interpretation errors among MR imaging-versus-CT cases.

Prior research in medical imaging perception might help to explain our results concerning the radiologist's experience level. Studies using eye-tracking technology indicate that more experienced radiologists have shorter image-viewing times, fewer visual fixations, fewer visual fixations on nondiagnostic areas of the image, and less dwell time per visual fixation when evaluating diagnostic imaging studies.29 Although such tendencies correlate with greater efficiency in image search, they might also explain the higher proportion of perceptual-type errors (ie, not seeing a pertinent imaging finding) among more experienced neuroradiologists. We stress, however, that while the proportion of errors among more experienced neuroradiologists compared with less experienced neuroradiologists was more heavily weighted toward perceptual errors, our results have not established that more experienced radiologists have a higher overall rate of perceptual errors.

Gaining an appreciation for predisposing conditions toward error type (interpretive-versus-perceptual) might aid future efforts towards error reduction in diagnostic neuroimaging. Perceptual errors (ie, nonidentification of a relevant imaging finding) might be reduced when radiologists use consistent search patterns or checklists during image evaluation,30 have improved accessibility to an accurate clinical indication,31 and read studies from state-of-the-art imaging systems.32 Clinical coverage schedules can be tailored to avoid overly long work shifts that may predispose to fatigue-related perceptual errors.33,34 Specific practice measures can also be implemented to reduce interpretation errors (ie, misinterpretation of a detected imaging finding), though such measures are less well-established in the literature. In theory, these practices may include multidisciplinary case discussion,35,36 selective second reading of studies associated with a higher risk for error,37,38 and application of a “slow-thinking” metacognition approach to avoid heuristic pitfalls when interpreting imaging abnormalities.39

This study has limitations. We collected 254 errors during a time when 75,000 CT and MR imaging studies were performed at our institution. This discovered error rate (<0.5%) is lower than previous clinically significant error rates reported from similar academic radiology practices.40 This finding likely reflects differences in the discovery mechanism and/or differences in what was considered a clinically significant error and does not indicate differences in interpretation accuracy between institutions. The subsequent study analysis implicitly assumes that error discovery was random and the subset used was a true representation of all errors that occurred. Error detection depended on a second radiologist's review of the images or a follow-up study; diagnostic errors that occur in outpatients without follow-up or image review during multidisciplinary conferences may be under-represented in our sample. Yet, our error-collection mechanism appeared to provide an externally valid representative cohort of cases; the percentage of perceptual errors and interpretive errors (74.8% and 25.2%, respectively) here is similar to that in previous reports.6⇓–8 Moreover, prior studies have shown that neoplastic and vascular etiologies are the most common neuroradiology error cases,25,40 concordant with our dataset (On-line Table). It would also be helpful in a future study to match error cases to similar cases that were interpreted correctly (ie, controls) to better resolve specific factors that contribute to errors. Such a measure may allow determination of differences in the error rate based on case-specific, work shift–specific, or radiologist-specific metrics and would be important for future study of this topic. The 2 neuroradiologists who adjudicated errors for this study would be considered less experienced in the above analytic framework, but these readers each had 5+ years reading only neuroradiology studies at an academic institution, adjudicated the studies by consensus (after prior group discussion), and had access to the full medical record (including subsequent clinical data).

Finally, we chose a simple, binary, and mutually exclusive scheme for classifying the error cases (ie, each case was either a perception or an interpretation error). Although this form of radiologic error categorization is commonly accepted in the literature,30 errors can take far more specific or overlapping forms than simply errors of perception or interpretation. For instance, research on medical imaging perception using eye-tracking technology indicates that perceptual errors broadly take 3 forms: 1) search error: the lesion is never fixated on; 2) recognition error: the lesion is fixated on, but below the threshold necessary for recognition; and 3) decision error: the lesion is fixated on for long periods, but its features are not fully appreciated and/or are dismissed.41 Interpretation errors also may vary, for instance, depending on whether a specific heuristic bias was involved in a given misinterpretation or a whether lack of sufficient knowledge contributed to the error.42,43 Understanding error trends at these higher levels of nuance is an important challenge for future research efforts.

## Conclusions

Among diagnostic neuroradiology error cases, interpretation-versus-perception errors are associated with neuroradiology attending physician experience, imaging technique, disease etiology, case volume, reading rate, and case order. Our results highlight differences between error types that may inform future situation-specific effort toward error reduction in clinical neuroradiology services.

## Footnotes

*   Disclosures: Sohil H. Patel—*UNRELATED*: *Expert Testimony*: medicolegal work*; *Grants/Grants Pending*: Radiological Society of North America Research Scholar Grant (RSCH1819)*. Jason N. Itri—*UNRELATED*: *Expert Testimony*: medicolegal work; *Grants/Grants Pending*: Coverys Community Healthcare Foundation, *Comments*: grant entitled, “Strategies to reduce diagnostic errors in imaging.”* Timothy M. Shepherd—*UNRELATED*: *Expert Testimony*: medicolegal work; *Grants/Grants Pending*: National Institutes of Health, National Institute on Aging K23-AG048622; *Stock/Stock Options*: MICroStructure Imaging (MICSI), Inc, *Comments*: start-up focused on advanced MRI processing and applications. *Money paid to the institution.

*   Paper previously presented, in part, at: Annual Meeting of the American Society of Neuroradiology and the Foundation of the ASNR Symposium, April 22–27, 2017; Long Beach, California.

## REFERENCES

1.  1. Makary MA, Daniel M. Medical error-the third leading cause of death in the US. BMJ 2016;353:i2139 doi:10.1136/bmj.i2139 pmid:27143499
    
    [FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE4OiIzNTMvbWF5MDNfMTIvaTIxMzkiO3M6NDoiYXRvbSI7czoyMDoiL2FqbnIvNDAvOC8xMjUyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

2.  2. Bruno MA, Walker EA, Abujudeh HH. Understanding and confronting our mistakes: the epidemiology of error in radiology and strategies for error reduction. Radiographics 2015;35:1668–76 doi:10.1148/rg.2015150023 pmid:26466178
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1148/rg.2015150023&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=26466178&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

3.  3. Brook OR, O'Connell AM, Thornton E, et al. Quality initiatives: anatomy and pathophysiology of errors occurring in clinical radiology practice. Radiographics 2010;30:1401–10 doi:10.1148/rg.305105013 pmid:20622192
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1148/rg.305105013&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=20622192&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

4.  4. Fitzgerald R. Error in radiology. Clin Radiol 2001;56:938–46 doi:10.1053/crad.2001.0858 pmid:11795921
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1053/crad.2001.0858&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=11795921&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=000172857500001&link_type=ISI) 

5.  5. Itri JN, Tappouni RR, McEachern RO, et al. Fundamentals of diagnostic error in imaging. Radiographics 2018;38:1845–65 doi:10.1148/rg.2018180021 pmid:30303801
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1148/rg.2018180021&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=30303801&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

6.  6. Donald JJ, Barnard SA. Common patterns in 558 diagnostic radiology errors. J Med Imaging Radiat Oncol 2012;56:173–78 doi:10.1111/j.1754-9485.2012.02348.x pmid:22498190
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1111/j.1754-9485.2012.02348.x&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=22498190&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

7.  7. Kabadi SJ, Krishnaraj A. Strategies for improving the value of the radiology report: a retrospective analysis of errors in formally over-read studies. J Am Coll Radiol 2017;14:459–66 doi:10.1016/j.jacr.2016.08.033 pmid:27884632
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.jacr.2016.08.033&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=27884632&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

8.  8. Renfrew DL, Franken EA Jr., Berbaum KS, et al. Error in radiology: classification and lessons in 182 cases presented at a problem case conference. Radiology 1992;183:145–50 doi:10.1148/radiology.183.1.1549661 pmid:1549661
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1148/radiology.183.1.1549661&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=1549661&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=A1992HJ87400026&link_type=ISI) 

9.  9. Cooper VF, Goodhartz LA, Nemcek AA Jr., et al. Radiology resident interpretations of on-call imaging studies: the incidence of major discrepancies. Acad Radiol 2008;15:1198–204 doi:10.1016/j.acra.2008.02.011 pmid:18692761
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.acra.2008.02.011&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=18692761&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

10. 10. Erly WK, Berger WG, Krupinski E, et al. Radiology resident evaluation of head CT scan orders in the emergency department. AJNR Am J Neuroradiol 2002;23:103–07 pmid:11827881
    
    [Abstract/FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpuciI7czo1OiJyZXNpZCI7czo4OiIyMy8xLzEwMyI7czo0OiJhdG9tIjtzOjIwOiIvYWpuci80MC84LzEyNTIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

11. 11. Filippi CG, Schneider B, Burbank HN, et al. Discrepancy rates of radiology resident interpretations of on-call neuroradiology MR imaging studies. Radiology 2008;249:972–79 doi:10.1148/radiol.2493071543 pmid:19011191
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1148/radiol.2493071543&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=19011191&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

12. 12. Funaki B, Szymski GX, Rosenblum JD. Significant on-call misses by radiology residents interpreting computed tomographic studies: perception versus cognition. Emerg Radiol 1997;4:290–94 doi:10.1007/BF01461735
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1007/BF01461735&link_type=DOI) 

13. 13. Goradia D, Blackmore CC, Talner LB, et al. Predicting radiology resident errors in diagnosis of cervical spine fractures. Acad Radiol 2005;12:888–93 doi:10.1016/j.acra.2005.04.004 pmid:16039542
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.acra.2005.04.004&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=16039542&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=000230867600011&link_type=ISI) 

14. 14. Guimaraes CV, Leach JL, Jones BV. Trainee misinterpretations on pediatric neuroimaging studies: classification, imaging analysis, and outcome assessment. AJNR Am J Neuroradiol 2011;32:1591–99 doi:10.3174/ajnr.A2567 pmid:21835948
    
    [Abstract/FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpuciI7czo1OiJyZXNpZCI7czo5OiIzMi85LzE1OTEiO3M6NDoiYXRvbSI7czoyMDoiL2FqbnIvNDAvOC8xMjUyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

15. 15. Hillier JC, Tattersall DJ, Gleeson FV. Trainee reporting of computed tomography examinations: do they make mistakes and does it matter? Clin Radiol 2004;59:159–62, discussion 157–58 pmid:14746785
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/S0009-9260(03)00309-X&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=14746785&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=000188712600006&link_type=ISI) 

16. 16. Lal NR, Murray UM, Eldevik OP, et al. Clinical consequences of misinterpretations of neuroradiologic CT scans by on-call radiology residents. AJNR Am J Neuroradiol 2000;21:124–29 pmid:10669236
    
    [Abstract/FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpuciI7czo1OiJyZXNpZCI7czo4OiIyMS8xLzEyNCI7czo0OiJhdG9tIjtzOjIwOiIvYWpuci80MC84LzEyNTIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

17. 17. Meyer RE, Nickerson JP, Burbank HN, et al. Discrepancy rates of on-call radiology residents' interpretations of CT angiography studies of the neck and circle of Willis. AJR Am J Roentgenol 2009;193:527–32 doi:10.2214/AJR.08.2169 pmid:19620452
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.2214/AJR.08.2169&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=19620452&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

18. 18. Miyakoshi A, Nguyen QT, Cohen WA, et al. Accuracy of preliminary interpretation of neurologic CT examinations by on-call radiology residents and assessment of patient outcomes at a level I trauma center. J Am Coll Radiol 2009;6:864–70 doi:10.1016/j.jacr.2009.07.021 pmid:19945042
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.jacr.2009.07.021&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=19945042&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

19. 19. Strub WM, Leach JL, Tomsick T, et al. Overnight preliminary head CT interpretations provided by residents: locations of misidentified intracranial hemorrhage. AJNR Am J Neuroradiol 2007;28:1679–82 doi:10.3174/ajnr.A0653 pmid:17885236
    
    [Abstract/FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpuciI7czo1OiJyZXNpZCI7czo5OiIyOC85LzE2NzkiO3M6NDoiYXRvbSI7czoyMDoiL2FqbnIvNDAvOC8xMjUyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

20. 20. Walls J, Hunter N, Brasher PM, et al. The DePICTORS Study: discrepancies in preliminary interpretation of CT scans between on-call residents and staff. Emerg Radiol 2009;16:303–08 doi:10.1007/s10140-009-0795-9 pmid:19184142
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1007/s10140-009-0795-9&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=19184142&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

21. 21. Wysoki MG, Nassar CJ, Koenigsberg RA, et al. Head trauma: CT scan interpretation by radiology residents versus staff radiologists. Radiology 1998;208:125–28 doi:10.1148/radiology.208.1.9646802 pmid:9646802
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1148/radiology.208.1.9646802&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=9646802&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=000074296300020&link_type=ISI) 

22. 22. Erly WK, Ashdown BC, Lucio RW 2nd., et al. Evaluation of emergency CT scans of the head: is there a community standard? AJR Am J Roentgenol 2003;180:1727–30 doi:10.2214/ajr.180.6.1801727 pmid:12760951
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.2214/ajr.180.6.1801727&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=12760951&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=000183149200039&link_type=ISI) 

23. 23. Jordan MJ, Lightfoote JB, Jordan JE. Quality outcomes of reinterpretation of brain CT imaging studies by subspecialty experts in neuroradiology. J Natl Med Assoc 2006;98:1326–28 pmid:16916131
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=16916131&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

24. 24. Loevner LA, Sonners AI, Schulman BJ, et al. Reinterpretation of cross-sectional images in patients with head and neck cancer in the setting of a multidisciplinary cancer center. AJNR Am J Neuroradiol 2002;23:1622–26 pmid:12427610
    
    [Abstract/FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpuciI7czo1OiJyZXNpZCI7czoxMDoiMjMvMTAvMTYyMiI7czo0OiJhdG9tIjtzOjIwOiIvYWpuci80MC84LzEyNTIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

25. 25. Zan E, Yousem DM, Carone M, et al. Second-opinion consultations in neuroradiology. Radiology 2010;255:135–41 doi:10.1148/radiol.09090831 pmid:20308451
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1148/radiol.09090831&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=20308451&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

26. 26. González RG, Schwamm LH. Imaging acute ischemic stroke. Handb Clin Neurol 2016;135:293–315 doi:10.1016/B978-0-444-53485-9.00016-7 pmid:27432672
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/B978-0-444-53485-9.00016-7&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=27432672&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

27. 27. Higer HP, Pedrosa P, Schuth M. MR imaging of cerebral tumors: state of the art and work in progress. Neurosurg Rev 1989;12:91–106 doi:10.1007/BF01741480 pmid:2674770
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1007/BF01741480&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=2674770&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

28. 28. Young IR, Hall AS, Pallis CA, et al. Nuclear magnetic resonance imaging of the brain in multiple sclerosis. Lancet 1981;2:1063–66 pmid:6118521
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=6118521&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=A1981MQ10100001&link_type=ISI) 

29. 29. Krupinski EA. The role of perception in imaging: past and future. Semin Nucl Med 2011;41:392–400 doi:10.1053/j.semnuclmed.2011.05.002 pmid:21978443
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1053/j.semnuclmed.2011.05.002&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=21978443&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

30. 30. Bahrami S, Yim CM. Quality initiatives: blind spots at brain imaging. Radiographics 2009;29:1877–96 doi:10.1148/rg.297095123 pmid:19734470
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1148/rg.297095123&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=19734470&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

31. 31. Mullins ME, Lev MH, Schellingerhout D, et al. Influence of availability of clinical history on detection of early stroke using unenhanced CT and diffusion-weighted MR imaging. AJR Am J Roentgenol 2002;179:223–28 doi:10.2214/ajr.179.1.1790223 pmid:12076941
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.2214/ajr.179.1.1790223&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=12076941&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

32. 32. Knake S, Triantafyllou C, Wald LL, et al. 3T phased array MRI improves the presurgical evaluation in focal epilepsies: a prospective study. Neurology 2005;65:1026–31 doi:10.1212/01.wnl.0000179355.04481.3c pmid:16217054
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1212/01.wnl.0000179355.04481.3c&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=16217054&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

33. 33. Krupinski EA, Berbaum KS, Caldwell RT, et al. Long radiology workdays reduce detection and accommodation accuracy. J Am Coll Radiol 2010;7:698–704 doi:10.1016/j.jacr.2010.03.004 pmid:20816631
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.jacr.2010.03.004&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=20816631&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

34. 34. Krupinski EA, Berbaum KS, Caldwell RT, et al. Do long radiology workdays affect nodule detection in dynamic CT interpretation? J Am Coll Radiol 2012;9:191–98 doi:10.1016/j.jacr.2011.11.013 pmid:22386166
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.jacr.2011.11.013&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=22386166&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

35. 35. Maldonado MD, Batchala P, Ornan D, et al. Features of diffuse gliomas that are misdiagnosed on initial neuroimaging: a case control study. J Neurooncol 2018;140:107–13 doi:10.1007/s11060-018-2939-9 pmid:29959694
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1007/s11060-018-2939-9&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=29959694&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

36. 36. Lee CS, Nagy PG, Weaver SJ, et al. Cognitive and system factors contributing to diagnostic errors in radiology. AJR Am J Roentgenol 2013;201:611–17 doi:10.2214/AJR.12.10375 pmid:23971454
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.2214/AJR.12.10375&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=23971454&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

37. 37. Lauritzen PM, Andersen JG, Stokke MV, et al. Radiologist-initiated double reading of abdominal CT: retrospective analysis of the clinical importance of changes to radiology reports. BMJ Qual Saf 2016;25:595–603 doi:10.1136/bmjqs-2015-004536 pmid:27013638
    
    [Abstract/FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI1LzgvNTk1IjtzOjQ6ImF0b20iO3M6MjA6Ii9ham5yLzQwLzgvMTI1Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

38. 38. Lauritzen PM, Stavem K, Andersen JG, et al. Double reading of current chest CT examinations: clinical importance of changes to radiology reports. Eur J Radiol 2016;85:199–204 doi:10.1016/j.ejrad.2015.11.012 pmid:26724666
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.ejrad.2015.11.012&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=26724666&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

39. 39. Busby LP, Courtier JL, Glastonbury CM. Bias in radiology: the how and why of misses and misinterpretations. Radiographics 2018;38:236–47 doi:10.1148/rg.2018170107 pmid:29194009
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1148/rg.2018170107&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=29194009&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

40. 40. Babiarz LS, Yousem DM. Quality control in neuroradiology: discrepancies in image interpretation among academic neuroradiologists. AJNR Am J Neuroradiol 2012;33:37–42 doi:10.3174/ajnr.A2704 pmid:22033725
    
    [Abstract/FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpuciI7czo1OiJyZXNpZCI7czo3OiIzMy8xLzM3IjtzOjQ6ImF0b20iO3M6MjA6Ii9ham5yLzQwLzgvMTI1Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

41. 41. Krupinski EA. Current perspectives in medical image perception. Atten Percept Psychophys 2010;72:1205–17 doi:10.3758/APP.72.5.1205 pmid:20601701
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.3758/APP.72.5.1205&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=20601701&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

42. 42. Itri JN, Patel SH. Heuristics and cognitive error in medical imaging. AJR Am J Roentgenol 2018;210:1097–105 doi:10.2214/AJR.17.18907 pmid:29528716
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.2214/AJR.17.18907&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=29528716&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

43. 43. Patel SH, Ambrosino MM, McGuinness G. The challenging case conference: initial observations and feedback. J Am Coll Radiol 2012;9:666–68 doi:10.1016/j.jacr.2012.04.010 pmid:22954551
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.jacr.2012.04.010&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=22954551&link_type=MED&atom=%2Fajnr%2F40%2F8%2F1252.atom) 

*   Received March 21, 2019.
*   Accepted after revision June 9, 2019.


*   © 2019 by American Journal of Neuroradiology