Evaluation of the Angiographic Grading Scale in Aneurysms Treated with the WEB Device in 80 Rabbits: Correlation with Histologic Evaluation

BACKGROUND AND PURPOSE: The WEB Occlusion Score has been proposed to assess angiographic outcomes for intracranial aneurysms treated with the Woven EndoBridge (WEB) device. Using a large series of experimental aneurysms treated with the WEB, we had the following objectives: 1) to compare angiographic outcomes as measured by the WEB Occlusion Scale with histologic results, and 2) to assess interobserver and intraobserver agreement of the WEB Occlusion Scale. MATERIALS AND METHODS: Intracranial aneurysms were created in 80 rabbits and treated with WEB devices. Animals were sacrificed at last follow-up for histologic evaluation. DSA was performed just after the deployment of the device and at follow-up. Four investigators independently and retrospectively graded the DSA twice according to the WEB Occlusion Scale. One histopathologist blinded to the angiographic results graded the occlusion according to a 4-point scale patterned on the WEB Occlusion Scale. Intra- and interobserver agreement were evaluated for DSA. Follow-up angiographic grading and histologic reference were compared to determine the WEB Occlusion Scale accuracy for complete (with or without recess filling) versus incomplete occlusion and adequate (complete occlusion or neck remnant) versus inadequate occlusion. RESULTS: Inter- and intraobserver weighted κ for the angiographic WEB Occlusion Scale were, respectively, 0.76 and 0.76, indicating substantial agreement. The sensitivity and specificity of the WEB Occlusion Scale for complete occlusion at follow-up compared with the histologic reference standard were, respectively, 75% and 83.3%, with an overall accuracy of 80%. Similarly, for adequate occlusion at follow-up, sensitivity was 97.7%, specificity was 64.9%, and overall accuracy was 82.5%. CONCLUSIONS: The WEB Occlusion Scale appears to be consistent, reliable, and accurate compared with a histologic reference standard.

T he Woven EndoBridge (WEB) device (Sequent Medical, Aliso Viejo, California) is a new and promising tool in the treatment of wide-neck and bifurcation intracranial aneurysms. [1][2][3][4] The endosaccular implant conforms to the aneurysm wall and spans the aneurysm neck to essentially function as a flow diverter at the aneurysm neck-parent artery interface. Initial experimental and clinical studies have demonstrated that the WEB is both safe and effective in the treatment of intracranial aneurysms. [1][2][3][4][5][6][7][8][9] Because the mechanism of action of the device differs from that in flow diverters and endosaccular coils, developing tools to assess angiographic occlusion has become a recent priority among many neurointerventionists, and several grading scales have been proposed. 4,[7][8][9][10] Perhaps the most commonly used scale is the WEB Occlusion Scale (WOS). 7 However, none of these scales, including the WOS, has previously been validated with histologic studies, to our knowledge. In a large series of elastase-induced aneurysms in rabbits treated with the WEB device, the objectives of this study were the following: 1) to compare angiographic outcomes as measured by the WOS with histologic results, and 2) to assess interobserver and intraobserver agreement of the WOS.

In Vivo Experiments
The Institutional Animal Care and Use Committee at our Institution approved the animal procedures. Elastase-induced aneu-rysms were created in 80 New Zealand white rabbits. Aneurysm creation procedures were performed as previously described by our study group. 11 Aneurysms were treated at least 3 weeks after aneurysm creation. 12 No antiplatelet therapy was used before or after treatment. During the device-deployment procedure, animals were anesthetized, the right femoral artery was exposed, and a 5F sheath was inserted, followed by injection of 500 U of heparin through the sheath. A 5F guide catheter (Envoy; Codman & Shurtleff, Raynham, Massachusetts) was advanced into the aortic arch. Digital subtraction angiography was performed with contrast injection through the guide catheter. A microcatheter (Renegade Hi-Flo; Boston Scientific, Natick, Massachusetts) was advanced into the aneurysm lumen over a microguidewire (Transend EX; Stryker, Kalamazoo, Michigan) through the guide catheter. The WEB size was selected according to the aneurysm size. After deployment of the device, the microcatheter was removed and DSA was performed through the guide catheter at the brachiocephalic trunk immediately and 5 minutes following device implantation. Follow-up angiographic evaluation was performed at the sacrifice end point according to the WEB Occlusion Scale as described by Fiorella et al. 7 Animals were sacrificed with a lethal injection of pentobarbital. Aneurysm and parent artery tissue were immediately fixed in 10% neutral buffered formalin.

Angiographic Evaluations
Four investigators independently and retrospectively examined selected images of the posttreatment and follow-up DSA to grade the occlusion status according to the WEB Occlusion Scale. These readings were performed twice by each of the investigators at 2-month intervals to analyze the intraobserver correlation in the readings. The WEB Occlusion Scale is a 4-point scale using the following grades: complete aneurysm occlusion, complete occlusion with recess filling, aneurysm neck remnants, and aneurysm remnants. The investigators also evaluated the modification of the aneurysm occlusion status between posttreatment and follow-up DSA as follows: improvement, stable, or recurrence. To compare follow-up DSA readings and histologic findings, we dichotomized DSA results as complete occlusion (with or without proximal recess filling) or incomplete occlusion (neck remnant or aneurysm remnant); similarly, we dichotomized DSA results as adequate occlusion (complete occlusion or neck remnant) or inadequate occlusion (aneurysm remnant), according to previous studies. 8,13 In case of disagreement among readers, a fifth reader adjudicated between adequate or inadequate occlusion.

Histopathologic Processing and Analysis
One histopathologist blinded to the angiographic results did the processing and analysis for healing evaluation. Aneurysm samples were processed at 1000-m intervals in a coronal orientation, permitting long-axis sectioning of the aneurysm neck, with use of an IsoMet Low Speed Saw (Buehler, Lake Bluff, Illinois). After the device segments were carefully removed under a dissecting microscope, the samples were then re-embedded in paraffin, sectioned at 5-6 m, and stained with hematoxylin-eosin. The sections were evaluated by using our previously reported evaluation criteria. 14 The histologic results for each aneurysm were evaluated according to a 4-point histologic scale, patterned on the angiographic WEB Occlusion Scale with the same items: complete aneurysm occlusion, complete occlusion with recess filling, aneurysm neck remnants, and aneurysm remnants.

Statistical Analysis
Statistical analysis was performed by using the statistical software package SAS 9.0 (SAS Institute, Cary, North Carolina). Interobserver and intraobserver agreement was assessed by using the quadratic weighted statistic. statistics for interobserver agreement were calculated between each observer. Mean values were calculated as well. The ANOVA intraclass correlation between readers was calculated. The sensitivity, specificity, and accuracy of DSA evaluations at last angiographic follow-up were calculated by using the histologic findings as a reference standard. These were calculated on a per-reading basis.

Population
Eighty consecutively treated rabbits were included in this study. The length of follow-up varied from 30 to 365 days, with a mean length of follow-up of 101.4 days. Rabbits were sacrificed at day 30 (n ϭ 27), day 50 (n ϭ 5), day 90 (n ϭ 30), day 180 (n ϭ 12), or day 365 (n ϭ 6). A Single-Layer WEB was used in 55% of cases (n ϭ 44), and a Dual-Layer WEB was used in 45% of cases (n ϭ 36).
To compare the DSA results with histologic findings, we determined a consensus value to allocate in the adequate versus inadequate occlusion group, according to the most present value without need for a fifth reader to adjudicate. The consensus dichotomized DSA evaluation yielded 68.8% (55/80) adequate occlusion and 31.2% (25/80) inadequate occlusion.

Correlation between Angiographic and Histologic Results
Illustrative correlations between follow-up DSA and histology are presented in Figs 1 and 2.

Correlation for the Diagnosis of Complete Occlusion
We observed 24 (30%) cases for which DSA and histologic evaluations agreed for complete occlusion (complete occlusion with or without recess filling) and 40 (50%) cases for which DSA and histology agreed for incomplete occlusion. In 8 (10%) cases, DSA suggested a complete occlusion but histology identified incomplete, and similarly, 8 (10%) cases that had been misassessed as incomplete by DSA were found to have complete occlusion on histology ( Table 2). Sensitivity and specificity of the DSA for the diagnosis of complete versus incomplete occlusion at follow-up compared with the histologic results as the reference standard were respectively 75% (95% CI, 56%-88%) and 83.3% (95% CI, 69%-92%). The overall accuracy of the DSA evaluation com-  pared with the reference standard histology was 80% (95% CI, 65%-90%).

Correlation for the Diagnosis of Adequate Occlusion
We observed 42 (52.5%) cases for which DSA and histologic evaluations agreed for adequate occlusion (complete occlusion or neck remnant) and 24 (30%) cases for which DSA and histology agreed for inadequate occlusion. In 13 (16.2%) cases, DSA suggested an adequate occlusion but histology identified inadequate occlusion, and similarly 1 (1.3%) case was misassessed as inade-quate by DSA but was found to have adequate occlusion on histology (

DISCUSSION
Using a histologic reference standard, our study performed on a large series of experimental aneurysms demonstrates that the angiographic WOS is sensitive, specific, and accurate in assessing aneurysm occlusion following treatment of intracranial aneurysms with the WEB device. In addition, we found a substantial level of inter-and intraobserver agreement for the WEB Occlusion Scale. These findings suggest that the WEB Occlusion Scale is an easily reproducible and accurate tool in assessing aneurysm occlusion following treatment with the WEB device.
Our current results are in accordance with the clinical study published by Fiorella et al, 7 which reported a value statistic at 0.779 (95% CI, 0.70 -0.86). Our study is the first to compare angiographic assessment of aneurysm occlusion after treatment with the intrasaccular WEB device with histologic controls. Assessment of occlusion is challenging following treatment with the WEB because the angiographic appearance differs from that of coiled or intraluminal flow-diverter-treated aneurysms. 7 The proximal surface of the WEB is slightly recessed into the body of the device, thus forming a concave "marker recess" at the parent artery-aneurysm interface, to avoid any protrusion of the device in the parent artery. This marker recess opacification can be mistaken for residual filling of the aneurysm neck.
The WEB Occlusion Scale (WOS), 15 based on the modified Raymond Scale, has been developed for the standardized reporting of angiographic occlusion assessment achieved with intrasaccular mesh implants, taking into account the distinction between recess opacification and aneurysm neck remnant. In this scale, complete aneurysm occlusion with or without opacification of the proximal recess is considered complete occlusion 7 ; complete occlusion and neck remnant are considered adequate angiographic outcomes according to previous studies. 13,15,16 This scale has  (blue arrow, B). An all-mount histologic sample shows a lack of filling of the aneurysm by the conjunctive tissue (blue arrow, C). Photomicrograph (hematoxylin-eosin, original magnification ϫ100) shows a lack of coverage of the neck by endothelial tissue and filling of the aneurysm pouch with a partial thrombosis in the aneurysm sac (D). No other large-scale study has been published comparing histologic findings with DSA in the setting of the evaluation of aneurysm occlusion either for coiled or flowdiverter-treated aneurysms. However, several studies have reported that interobserver and intraobserver agreement rates for the assessment of aneurysm occlusion following both coiling and flow-diverter treatment are moderate to substantial. 13,[16][17][18][19] The degree of agreement seen in our study for the WOS is similar to that reported in prior studies of coiling and flow-diverter-treated aneurysms. At a time when consensus definitions for reporting angiographic outcomes following endovascular treatment of intracranial aneurysms are required, scales that are both easily reproducible and histologically validated are of the utmost importance. 20,21

Limitations
Our study is limited by its retrospective nature and the use of only selected images for the DSA readers' assessment. Readers did not have access to the complete angiographic run when assessing angiographic occlusion. Furthermore, only 1 experienced reader evaluated the histologic samples. However, this reader was blinded to the DSA outcomes. Another limitation of this study is that rabbits were sacrificed at different time points, which can modify the outcomes after WEB implantation, depending on the length of follow-up.

CONCLUSIONS
This study confirms the consistency and reliability of the WEB Occlusion Scale for DSA evaluation of WEB-treated aneurysms with substantial interobserver and intraobserver agreement. Furthermore, the WEB Occlusion Scale appears to be accurate compared with a histologic reference standard, which is of great importance to justify its use in clinical studies for the evaluation of the WEB device.