Rationale and objectives: This study examines the joint effect of several factors on radiology resident performance in the task of interpreting after-hours neuroradiology examinations.
Materials and methods: As part of a quality assessment process, we conducted a prospective evaluation of all (N = 21,796) after-hours preliminary readings of neuroradiology examinations performed by radiology residents over a 62-month period at our academic medical center. Each reading was scored by the interpreting neuroradiologist as "agree," "disagree with minimal clinical impact," and "disagree with significant clinical impact." Coded resident and attending identities were also recorded for each case along with modality, body area studied, and the date of examination. These raw data were used to create an analytic data set with level of resident/attending agreement as the outcome and six predictors, including two date-derived variables: months 1-62 representing when the case occurred during the study and quartiles 1-4 accounting for the timing of the case in each resident's own experience. Cross tabulations, plots, bivariate statistics, and logistic regression were used to examine the relationships between study variables and the outcome (level of agreement).
Results: Over about 5 years of the study, the absolute number of significant disagreements remained stable at about three per month. The total caseload increased at a rate of 4.1 per month with most of the increase falling into the agree category, whereas the minimal disagreements actually decreased slightly (0.2 per month). In the logistic model for disagreement, three of the factors accounted for most of the variance: attending (61%), resident (15%), and month (15%). Study type (modality and area examined) accounted for another 10%. There was no significant contribution from the variable (quartile) constructed to test for individual resident learning during the on-call experience.
Conclusion: Although residents differ somewhat in the extent of attending agreement with their on-call work, evaluation or remediation made on the basis of simple comparison of these rates should be done with caution. Improved agreement over time seems to be a collective experience shared by residents.