Validity of the Meyer Scale for Assessment of Coiled Aneurysms and Aneurysm Recurrence

BACKGROUND AND PURPOSE: Both the Meyer and Raymond scales are commonly used to report angiographic outcomes following coil embolization of intracranial aneurysms. The objectives of this study were the following: 1) to assess the interobserver agreement of the Meyer and Raymond scales, and 2) to evaluate and compare their performance in predicting major recurrence at follow-up. MATERIALS AND METHODS: A retrospective series of 120 coiled aneurysms was included. Four investigators independently graded DSA images immediately posttreatment and at follow-up according to the Meyer and Raymond scales. On follow-up DSA, readers also evaluated recurrence outcome. Interobserver agreement was assessed via the intraclass correlation coefficient. The ability of posttreatment Meyer and Raymond scales to predict major recurrence was modeled by using logistic regression and assessed by using receiver operating characteristic analysis. RESULTS: For the Meyer scale, interobserver intraclass correlation coefficients were 0.58 (95% CI, 0.46–0.68) on posttreatment and 0.78 (95% CI, 0.72–0.83) on follow-up evaluations. For the Raymond scale, interobserver intraclass correlation coefficients were 0.50 (95% CI, 0.39–0.61) and 0.69 (95% CI, 0.62–0.76), respectively, for posttreatment and follow-up. The areas under the curve for the receiver operating characteristic analyses regarding the performance to predict major recurrence at follow-up were 0.69 (95% CI, 0.60–0.79) for the Meyer and 0.70 (95% CI, 0.61–0.78) for the Raymond scale. CONCLUSIONS: The Meyer scale appears consistent and reliable with observer agreement as high or higher than that of the Raymond scale. Performance of both scales in predicting the risk of major recurrence at follow-up is adequate, with no statistical difference between the scales.

T he most commonly used grading scale to assess aneurysm occlusion after coiling is the Raymond and Roy scale. 1 This grading scale divides the arteriographic outcomes of endovascularly treated saccular aneurysms into 3 categories: 1) complete occlusion, 2) residual neck, and 3) residual aneurysm. 1 However, despite its simplicity, its application in large clinical registries demonstrated important interobserver variability, which may limit its application and efficacy as an outcome-assessment tool. 2 Moreover, important variability in aneurysm morphology and occlusion may occur within the same category by using this 3-category scale. To standardize the reporting on the degree to which volumetric occlusion has been achieved, a new consensual 6-point grading system, the Meyer scale, has been proposed jointly by multiple societies across neurointerventional radiology, neurosurgery, and neurology. [3][4][5][6] This proposed scale, published in 2012, was put forth to standardize the reporting of the degree of volumetric occlusion and to properly assess technical outcomes in a standardized report. The rationale for this new scale was that the commonly used Raymond scale demonstrated important interobserver variability that may limit its application and efficacy as an outcome-assessment tool. 2 However, despite the necessity for high reproducibility and interobserver rates, this new grading scale has never been evaluated, to our knowledge. As stated by the consensus writing group, there is no literature support for the proposed grading system, though this writing group believes it is the best available grading scale and offers a greater likelihood of reporting additional degrees of aneurysm recurrence. [3][4][5][6] In addition, none of the scales have been validated as predictive of aneurysm recurrence. 7,8 Possible risk factors for reopening of a coiled aneurysm with time, such as the initial degree of occlusion, 9 seem to have an influence on the likelihood of recurrence, with subtotal initial treatment leading to remnant regrowth. 10,11 Using a large retrospective series of patients with coiled aneurysms, the objectives of this current study were the following: 1) to assess interobserver agreement of the Meyer scale, and 2) to evaluate its performance in predicting recurrence at follow-up and compare the results with those of the Raymond scale.

Study Cases
Mayo Clinic institutional review board approval was obtained for this retrospective study. Records for patients treated with endovascular coil embolization for either ruptured or unruptured aneurysms at our institution were evaluated. Some of the cases were used in a prior publication, but there is no overlap in scope with the current article. 12 These images of 120 cases of coiled aneurysms were compiled, de-identified, and placed into a 120-page PDF file. Each page included a DSA of the initial uncoiled aneurysm if available, the final postcoiling working-projection DSA, and the follow-up DSA. The PDF file was then distributed to 4 interventional neuroradiologists in different international centers. Two readers had 6 years' experience, and 2 readers had 4 years' experience.

Angiographic Evaluations
The Meyer scale is a 6-point grading scale based on the percentage of the aneurysm filled by contrast on DSA. Grade zero indicates complete and total aneurysm occlusion without remnant or interstitial filling within the aneurysm. Grade 1 represents Ͼ90% volumetric occlusion of the aneurysm based on planar imaging assessment; grade 2, 70%-89% aneurysm occlusion; grade 3, 50%-69%; grade 4, 25%-49%; and grade 5, Ͻ25% volumetric aneurysm occlusion. Two other "modifier" criteria are combined with this scale: The modifier "I" may be used to describe interstitial opacification within the confines of the coil mass and the modifier "G" may be used to describe interval growth in the overall dimensions of the aneurysm as a separate phenomenon from coil compaction with recurrence of the aneurysm.
The Raymond scale is a commonly used outcome grading scale that divides the angiographic outcomes of endovascularly treated saccular aneurysms into 3 categories: complete occlusion, residual neck, and residual aneurysm. 1 Complete occlusion and residual neck are considered adequate occlusion; and residual aneurysm, inadequate occlusion. [13][14][15][16] The 4 investigators independently and retrospectively examined each set of images of the posttreatment and follow-up DSAs to grade occlusion status according to the Raymond scale, the Meyer scale, and the modifier criteria. The investigators also evaluated the recurrence status of the aneurysm at follow-up DSA as follows: no recurrence, minor recurrence, or major recurrence. Major recurrence was defined as a theoretically or technically retreatable aneurysm and was considered an unfavorable outcome.
To evaluate the performance of posttreatment DSA evaluation to predict the risk for major recurrence, we used a consensus rating based on the most common grade among all readers. In case of disagreement among readers, 2 readers (A.R., W.B.) did a consensus reading.

Statistical Analysis
Reader agreement was assessed via the intraclass correlation coefficient (ICC) by using the ICC method (2,1), as specified by Shrout and Fleiss, 17 for each of the 7 outcomes (posttreatment and follow-up Raymond scale, Meyer scale, Meyer modifier criteria, and recurrence status at follow-up). For the ICC description, we used the verbal description proposed by Portney and Watkins 18 : 0 -.2, poor agreement; 0.3-0.4, fair agreement; 0.5-0.6, moderate agreement; 0.6 -0.7, substantial agreement; 0.7-0.8, strong agreement; and Ͼ0.8, almost perfect agreement. Aneurysm recurrence at follow-up was modeled as a function of either the Raymond scale or the Meyer scale (including the Meyer modifier criteria). Univariable (for Raymond) or multivariable (for Meyer) logistic regression by using the Firth penalized likelihood was performed by using the consensus rating of favorable or unfavorable recurrence at follow-up as the outcome. Aneurysm recurrence was modeled and receiver operating characteristic (ROC) curves were constructed both for each of the 4 raters individually and by using a consensus rating for each scale.

Prediction of Recurrence According to the Posttreatment Angiographic Evaluation
For the Raymond

DISCUSSION
Our current study demonstrated that the Meyer scale has moderate interreader agreement immediately after treatment and strong agreement at follow-up, and for both immediate posttreatment and follow-up, the agreement for the Meyer scale is comparable with that of the Raymond scale. Furthermore, the performance of the Meyer and Raymond scales for predicting major recurrence risk, based on immediate posttreatment results, was fair and similar between scales. These data indicate that the Meyer and Raymond scales have similar performance and consistency levels for aneurysm occlusion evaluation and prediction of a major recurrence.
The interobserver agreement in our study for the Raymond scale is higher than that previously reported by Tollard et al, 19 who reported a fair statistic at 0.276. Interobserver agreement statistics for the Raymond scale ranged from 0.28 to 0.83 in prior studies. [19][20][21][22][23][24] Most interesting, the Meyer scale agreement score was higher than that of the Raymond scale despite the higher chance for better agreement when grading with fewer responses. 8 This outcome is likely because the Meyer score allows a little more flexibility in the assessment of angiographic occlusion. For example, in a case in which an aneurysm is well-occluded with the exception of filling in the interstices, some may mark the status as complete occlusion on the Raymond scale and others may mark the status as aneurysm remnant. However, with the Meyer scale, a visual assessment of the percentage of angiographic filling may be more reproducible. The interstitial opacification criteria have previously been taken into account in the recently introduced modified Raymond scale, with a higher performance in predicting recurrence at follow-up. 20 Beyond assessing an adequate or inadequate operator result, grading aneurysm occlusion at the end of the coiling is of interest for predicting the long-term outcome and managing follow-up. Subtotal initial treatment leads to a higher likelihood of potential bleeding risk. 10,11 Raymond et al 22 reported major recurrence rates of 9% for aneurysms initially completely occluded, 23% for aneurysms with a residual neck, and 47% for aneurysms with residual aneurysm. Thornton et al 25 reported a recurrence rate of 1.8% for aneurysms that were 100% occluded and 26% for aneurysms with a residual neck. To our knowledge, this study is the first to evaluate the Meyer scale for predicting the risk of major recurrence at follow-up. Both the Raymond and the Meyer scales, combined with the interstitial opacification criteria, have a fair performance level in predicting the risk of unfavorable outcome at follow-up. This is important because in cases of an inadequate posttreatment angiographic results, given the higher risk for major recurrence, physicians could plan to perform surveillance angiography earlier or more frequently. 12,19,26,27 Further studies are needed to determine a cutoff value for adequate-versus-inadequate posttreatment occlusion according to the Meyer scale. In our study, we focused on posttreatment angiographic grading, but this single criterion is not the only one that influences the recurrence outcome. Other factors such as aneurysm size, 9,22 smoking status, 28,29 and rupture status 22 should be taken into account to have a model for predicting recurrence.

Limitations
Our study is limited by its retrospective nature and the use of only selected images for the DSA readers' assessment. Readers did not have access to the complete angiographic run when assessing angiographic occlusion. Another limitation of this study is that DSA was performed at different time points, which can modify the outcomes, depending on the length of follow-up. 22

CONCLUSIONS
This study highlights the consistency and the reliability of both the Meyer and Raymond occlusion scales for DSA evaluation of coiled aneurysms. Both scales have substantial interobserver agreement and fair performance levels in predicting major recurrence at follow-up. These findings are very important in managing patients at follow-up.