Interobserver Variability in Retreatment Decisions of Recurrent and Residual Aneurysms

BACKGROUND AND PURPOSE: The degree of variation in retreatment decisions for residual or recurrent aneurysms among endovascular therapists remains poorly defined. We performed a multireader study to determine what reader and patient variables contribute to this variation.

MATERIALS AND METHODS: Seven endovascular therapists (4 neuroradiologists, 3 neurosurgeons) independently reviewed 66 cases of patients treated with endovascular coil embolization for ruptured or unruptured aneurysm. Cases were rated on a 5-point scale recommending for whether to retreat and a recommended retreatment type. Reader agreement was assessed by intraclass correlation coefficient and by identifying cases with a “clinically meaningful difference” (a difference in score that would result in a difference in treatment). Variables that affect reader agreement and retreatment decisions were examined by using the Wilcoxon signed-rank test, Pearson χ² test, and linear regression.

RESULTS: Overall interobserver variability for decision to retreat was moderate (ICC = 0.50; 95% CI, 0.40–0.61). Clinically meaningful differences between at least 2 readers were present in 61% of cases and were significantly more common among neuroradiologists than neurosurgeons (P = .0007). Neurosurgeons were more likely to recommend “definitely retreat” than neuroradiologists (P < .0001). Previously ruptured aneurysms, larger remnant size, and younger patients were associated with more retreat recommendations. Interobserver variability regarding retreatment type was fair overall 0.25 (95% CI, 0.14–0.41) but poor for experienced readers 0.14 (95% CI, 0–0.34).

CONCLUSIONS: There is a large amount of interobserver variability regarding the decision to retreat an aneurysm and the type of retreatment. This variability must be reduced to increase consistency in these subjective outcome measurements.