Observer Agreement Regarding the Necessity of Retreatment of Previously Coiled Recurrent Cerebral Aneurysms

Discussion

In this study, we demonstrated substantial variability among experienced operators when deciding, on morphologic features alone, whether a series of aneurysms should be retreated and how. This finding is important for several reasons. First, it suggests that patient management following aneurysm recurrence may vary widely among operators and centers. Second, these findings suggest that retreatment may not be an ideal outcome in a clinical trial, given wide interobserver variability. Third, it sheds light on a previously neglected aspect of patient care—that is, whom should we retreat and why?

The underlying reasons behind the wide variation among our readers remain unknown but likely are multifactorial. Issues such as ease of retreatment probably impacted individual ratings—that is, rounded narrow-necked recurrences, irrespective of size, might have prompted some reviewers to recommend treatment. Difficult wide-necked cases might have prompted those who favor stents to recommend retreatment. This tendency is shown by the fact that in cases in which readers were uncertain about retreatment, surgery was recommended, while in cases with high degrees of certainty, readers were amenable to coil embolization without the need for a stent.

In our study, roughly two-thirds of cases showed differences of ≥2 points. A 2-point difference indicates a difference of “not sure” to “definitely treat” or “definitely do not treat”, or a difference of “probably treat” to “probably not treat.” Thus, even a 2-point difference might lead to differences in patient management. Thirty-seven percent of cases varied by 3 points. A 3-point variation indicates a difference between “definitely treat” to “probably not treat” or vice versa, again indicating substantial differences in patient management.

One might theorize that treating physicians may be more likely to recommend retreatment for ruptured recurrent aneurysms over unruptured recurrences in certain cases. To prevent this possible bias in the current study, we told the reviewers to assume that all patients were 50-year-old women who had SAHs from the index aneurysm and who had made full clinical recoveries, and that their follow-up imaging was performed at their 6-month return visit. This information was included to prevent retreatment bias based on rupture status, age, and recovery status; however, these likely play a strong role in a provider's decision about recurrent aneurysm retreatment. More research is needed in this area. Likewise, we cannot be certain that our small cohort is typical of all recurrent aneurysms. It remains possible that these cases were particularly challenging. Future studies are needed to address this issue.

Several studies have been performed evaluating the occlusion rates of aneurysms and their recurrence.^13,15–19 Previous evaluations of intraobserver agreement in the assessment of aneurysm recurrence have found that agreement among observers expectedly improved as the number of response options decreased.¹² Likewise studies have demonstrated that recurrence depends on the initial size of the aneurysm as well as previous rupture status.^13,17 A large meta-analysis by Ferns et al²⁰suggested that approximately 20% of all coiled aneurysms recur and half of these go on to be retreated, but there are no good guidelines that can help a clinician decide whether observation or retreatment is more appropriate. Much of our decision-making about retreatment of recurrent aneurysms relies on experience and is likely affected by previous cases and poor outcomes.

This study has several limitations. First, 4 of the readers had overlapping training often with similar training styles. Nevertheless, the reviewer most likely to recommend observation and the reviewer most likely to recommend retreatment worked together. Second, previous studies from our group¹² have suggested, as is evident in our current data, that the increase in number of rating options leads to a decrease in the intraobserver agreement (κ = 0.27) as seen in our data. Likewise, the current study was not designed to look at which part of aneurysm morphology is more likely to evoke a response advocating retreatment. Additional studies are needed to further elucidate morphometric differences that would aid in defining the need for retreatment.