Abstract
BACKGROUND AND PURPOSE: The aim of this study was to evaluate 2D-digital subtraction angiographic (DSA) and 3D-time-of-flight (TOF) MR imaging in assessment of aneurysmal residue by using a pulsating silicon aneurysm model. For each imaging system, we studied intra- and interobserver reproducibility and the agreement between interpretations and reference measurements. We also examined how each imaging technique affected the operator’s therapeutic decision.
METHODS: Two silicon aneurysm models depicting subarachnoidal aneurysms were used, one with a wide neck and one with a narrow neck. Each aneurysm model was placed in series on a pulsed flow circuit and was filled with Guglielmi detachable coils to simulate a clinical case. Each aneurysm was then gradually filled with silicon gel in increments of 10%, up to 100% to simulate different levels of occlusion (residual neck or dog ear, partial, complete) at each filling level. For each level of filling, we performed conventional 2D-DSA and 3D-TOF MR imaging. We submitted the images for examination by 2 senior medical staff with 2 readings per image. A combined reading of the 2 images was submitted to each expert to determine whether the 2 examinations were complementary.
RESULTS: The 2D-DSA analysis showed good reproducibility (k = 0.8 and k = 0.57) and agreement (k = 0.71) in describing “complete” treatments. The distinction between a “residual neck” and “partial treatment,” however, was not reliable. The 2D-DSA provided a good description of the coil and silicon protrusion into the parent artery. The 3D-TOF analysis of the residual aneurysm, however, was not reproducible, though it was more effective than the 2D-DSA in evaluation of partially wide-necked aneurysms (k = 0.68 MR imaging vs k = 0.041 2D-DSA; P = .018). At the same filling level, the 2D-DSA analysis indicated repeat treatment more often than 3D-TOF analysis (P = .059).
CONCLUSION: The 2D-DSA remains the gold standard, but MR imaging is more effective in evaluating a “partial treatment.” The 2D-DSA analysis indicated repeat treatment more often than the 3D-TOF for the same occlusion level. The distinction between “partial treatment” and a “residual neck” was not reliable with either method of evaluation.
Endovascular therapy of intracranial aneurysms is gradually becoming the preferred treatment of choice because of its low risk of complications. Coil embolization with Guglielmi detachable coils (GDCs; Boston Scientific Neurovascular, Fremont, Calif) is currently the most widely used endovascular procedure. Postoperative angiographic follow-up is essential in light of the high percentage of aneurysm recurrence found in recent studies.1,2 Evaluation of aneurysm occlusion is a fundamental parameter in postoperative follow-up because it can be used to evaluate the risk of recurrence and the need for additional treatment.
Two imaging techniques are currently used in clinical practice to assess this parameter: 2D-digital subtraction angiography (DSA) is an invasive technique requiring selective arterial catheterization currently considered the reference method; sequential 3D-time-of-flight (TOF) MR imaging is a noninvasive technique typically used in our institution to perform a first examination at 6 months. In case of anomaly, 2D-DSA is performed soon afterward.
The practitioner evaluates filling of the aneurysm subjectively. At present, the 2 methods for assessing aneurysm filling are the occlusion rate by percentage grouped into 3 categories: <70%, 70%–90%, and 90%–100%; and the classification described by Raymond et al,3 known as the Jean Raymond grading scale, that distinguishes 4 classes of aneurysmal filling level—complete obliteration, partial, residual neck, and dog ear. Residual neck and dog ear, defined by the persistence of residual flow at the neck of the aneurysm, were combined into one single group in this study. Residual aneurysm was defined as a “partial treatment” with a residual flow within the mass of coils.
Recent work has examined the agreement between 3D-TOF evaluations and the gold standard (2D-DSA).4–7 These in vivo studies, however, do not allow verification of the correspondence of the images with the anatomic reality of the aneurysmal residual.
We did not find in the literature any comparative studies of the 2 imaging systems by using a reference model covering the modalities of clinical evaluation.
The aim of our work was to evaluate the correlation between subjective evaluations by interventional practitioners and objective measurements made by using a reference silicon model. We filled 2 aneurysms, one wide-necked and one narrow-necked, with GDC coils to obtain loose but realistic packing. Each aneurysm was then gradually filled with silicon gel to simulate different levels of occlusion. For each level of occlusion, we performed 2D-DSA and 3D-TOF MR imaging, which were subjected to appraisal by 2 experienced interventional neuroradiologists.
Materials and Methods
Silicon Model and Characterization of Flow Dynamics
We used a silicon model of the left internal carotid artery mounted in series under pulsatile flow (Flowtek System, San Diego, Calif). The model had 3 aneurysms, including one with a broad neck and one with a narrow neck (Fig 1).8,9
The rate and frequency of pulsation were calibrated by using an echography probe in Doppler mode and were dependent on the mean values found at the base of an internal carotid artery (300 mL/min; systolic speed, 80–100 cm/s; section, 7 mm).10 The echogenicity of the solution was increased for calibration purposes by the addition of microbubbles.
We used an isotonic solution of macromolecule (hydroxyethylstarch 130,000 d and sodium chloride [Voluven]) to simulate blood viscosity as closely as possible (viscosity of Voluven at 24°C, 2.2 mPaS).
For the needs of the 3D-TOF MR imaging sequence, the R1 relaxivity (1/T1) of the solution was calibrated to that of fresh blood (T1 = 1300 milliseconds) by the addition of gadolinium (Prohance 1396.5 mg/5 mL) at a concentration of 0.16 mL/L Voluven.
In all, our reference model met the flow constraints of flow rate, pulsability, viscosity, and relaxivity needed for both types of imaging: 3D-TOF MR and 2D-DSA.
Aneurysm Filling Protocols
For each aneurysm, we performed a series of reference measurements of the volume by using a microcatheter and micrograduated syringe. The reference volumes were compared with the values obtained after postprocessing of a 3D-DSA image. There were no significant differences between the direct measurements and the 3D-DSA quantification (P < .05).
Because our model did not allow thrombus formation, after loosely packing the aneurysms with GDC, we gradually filled them with silicon gel injected in sections of 10% up to 100% to simulate different levels of aneurysmal occlusion.
Each injection was quantified by using a micrograduated syringe to determine the real percentage of filling of the aneurysm at each stage. At each level of filling quantified on the reference model, we performed a 3D-TOF sequence for MR imaging studies and a 4-plane conventional radiographic acquisition for 2D-DSA.
For the narrow-necked aneurysm, we measured packing by using a volume-embolization ratio (VER; total coils volume/aneurysm volume) of 20%. After the final stage of filling (90%–100%), we intentionally injected a small amount of silicon into the parent artery simulating a packing protrusion (Table 1).
For the wide-necked aneurysm, we performed packing at a VER of 12%. We deliberately pulled out a loop of coil into the parent artery at 2 levels of filling (Table 2).
At each stage of filling, we used silicon of a different color. At the end of the manipulations, we prepared 2-mm-thick sections of each packed aneurysm and studied via transillumination the distribution of each silicon injection (Fig 2). We were able to characterize the real topography of the aneurysmal residue (residual neck, dog ear, partial, complete) at each filling level.
TOF MR Imaging Sequence
The MR imaging exploration was performed by using a 1.5T magnet (Intera, Release 10, Philips Medical System, Best, the Netherlands; 33 mT/m hypergradients) with a phase array head coil. The aneurysm model was placed in the center of the antenna with 2 bags of isotonic saline serum to obtain a satisfactory signal intensity.
The 3D-TOF was identical to the one used routinely in clinical practice for the diagnosis and follow-up of treated aneurysms: axial 3D gradient echo sequence (T1 weighted with TR, 23 milliseconds; TE, 3 milliseconds; α = 20°; Nex, 1; field of view, 180 mm; 140 sections, 0.5 mm thick, were reconstructed; matrix, 256 × 202; sense factor, 2). In all, the voxel was anisotropic and measured 0.35 × 0.35 × 0.50. The acquisition time was 5 minutes, 41 seconds.
Visualization Protocol
The examinations were visualized on a Philips PC, View Forum, version 3.2 (Best, the Netherlands). The visualization protocol for 3D-TOF acquisitions included visualization of native images and reconstructions by maximum intensity profile (MIP). We generated 2 series of 12 views, which included rotations around the major axes and rolled around the small axis of the volume under examination. The intensity threshold was, as in clinical practice, dependent on the operator. The silicon gel had a lower signal intensity compared with the high signal intensity of the circulating fluid.
2D-DSA Acquisition
We used a Siemens Neurostar Plus/T.O.P (Siemens Corporation, Munich, Germany) biplanar apparatus. The 2D acquisitions were obtained in the subtraction mode.
The acquisition conditions were identical to the parameters generally used in clinical practice during injection into the internal carotid artery: matrix, 1024 × 1024, by using orthogonal biplane acquisitions. Four projections (anteroposterior, lateral, anterior, and oblique views), were properly selected to accurately demonstrate, at least in one instance, the relationship of the aneurysm with the parent artery with an imaging field of 22 cm. In each case the anteroposterior view was found to be the best incidence, but the other views were helpful to judge the presence of residual flow within the mass of coils.
Injection parameters were 4 mL/s of a total of 10 mL Hexabrix with a 2-second acquisition delay.
Visualization Protocol
The acquisitions were reproduced on film (3 × 4) with a reference mask for each view.
Medical Expert Examination
Each level of filling of each aneurysm was given an anonymous order (run number from 1 to 16 for 2D-DSA and 1 to 16 for 3D-TOF MR imaging).
Two blinded experienced interventional neuroradiologists made 2 independent readings of each image in a randomized order. A total of 4 sessions of 16 interpretations were performed.
2D-DSA and 3D-TOF studies of the model without coil or silicon were available to the experts during the readings.
A nominal scoring grid was completed after each reading. The criteria assessed were the percentage of aneurysmal occlusion from 0%–100% by 10% step intervals; the filling level according to Raymond grading scale3 combined “dog ear” and “residual neck” into one group; the presence of a coil extrusion into the parent artery, coded yes/no; the presence of a packing protrusion into the parent artery, coded yes/no; the image quality (very good, good, poor, not interpretable).
At the end of the analysis, we asked each physician to indicate whether further treatment was required on the basis of the images presented.
A third combined reading of the MR imaging and 2D-DSA studies for each filling level was performed by each expert to judge the complementary nature of the 2 imaging systems.
Statistical Analysis
The data from the readings were compared with the reference measurements made on the model and the agreement estimated by using the Kappa coefficient with calculation of the 95% confidence interval (CI 95%). The comparison between the kappa values was performed by using a z test.11 A P value ≤.05 was considered to be statistically significant.
Depending on the MR imaging and angiographic results being paired, any therapeutic decisions made by the experts were done by using McNemar’s χ2 test on dissimilar pairs after verification that the conditions of application were met (theoretical population >5).
Results
All images were graded very good or good by the 2 experts.
Intra- and Interobserver Variations
Estimation of the aneurysmal residue by percentage class was not reproducible between readers, whatever acquisition technique was used. In our study, the classification using the Raymond grading scale showed greater inter- and intraobserver reproducibility. Therefore, this is the reference scale we chose to compare reproducibility between 2D-DSA and 3D-TOF.
Intraobserver Variation
In 2D-DSA, intraobserver reproducibility was good with kappa values of 0.801 and 0.589 with no significant difference between the 2 readers (P = .47).
The kappa values for the MR imaging evaluation were low, ranging from 0.061 to 0.253, also with no significant difference between readers (P = .49).
Interobserver Variation
In 2D-DSA, reproducibility was excellent for evaluation of packing or coil extrusion (k = 1.0 and k = 0.765) into the parent artery and for assessment of complete treatment. In contrast, the kappa values were very low when it came to evaluating the aneurysmal residue (residual neck or dog ear from residual aneurysm).
In 3D-TOF, the kappa value was excellent for showing packing protrusion into the parent artery (k = 1.0), but poor in evaluating the extrusion of the coils (k = 0.103), residual aneurysm (k = 0.3), and residual neck or dog ear (k = −0.333).
Agreement between the Evaluation of Filling by Percentage Class (%) and the Reference Measurements
There was no agreement between the evaluation of filling by percentage class and the reference measurements whatever the method of acquisition despite a grouping by class used in clinical practice (<70%, 70%–90%, 90%–100%; Table 3).
Agreement between the Evaluation of Filling by the Raymond Grading Scale and the Reference Measurements
2D-DSA.
Evaluation of filling in 2D-DSA, according to the Raymond classification, showed that for both readers whatever the neck and filling level there was poor agreement. Depending on the type of filling, however, significantly different kappa values were obtained (P < .05), with good agreement in the evaluations of complete treatment with kappa values at 0.71 versus only 0.14 and 0.11 for the evaluations of partial treatment and residual neck (Table 4; Figs 3 and 4).
3D-TOF.
The 3D-TOF evaluation of filling showed, for both readers, irrespective of neck size and occlusion level, moderate agreement (k = 0.447; Table 5). The z score test,11 however, did not show any significant difference from the 2D-DSA evaluation (P = .139).
Differences in the kappa value were noted at different filling levels. When treatment was defined as partial on the reference (residual aneurysm), the kappa value was 0.50 (Table 5) versus 0.145 (Table 4) for the 2D-DSA evaluation, though no significant difference could be shown at this level (P = .068).
The kappa value in evaluation of the complete treatment was moderate (0.483), lower than that of 2D-DSA, with no significant difference (P = .238). The kappa value in evaluation of a residual neck was moderate as in 2D-DSA (Tables 4 and 5).
Combined Reading.
The combined reading of the 2 acquisitions showed no significant difference between the kappa values obtained by 2D-DSA and 3D-TOF.
Study of the Agreement between Evaluations and the Reference Depending on the Neck Type
Wide Neck.
In wide-necked aneurysms, irrespective of the occlusion level obtained, a broad range of kappa values was recorded by using either 3D-TOF or 2D-DSA evaluations (Table 6); however, 3D-TOF assessment of partial treatment on the wide-necked aneurysm was significantly better than that by 2D-DSA (respective kappa values, 0.679 vs 0.041; P = .018; Fig 5).
Narrow Neck.
The kappa values for all occlusion levels together ranged from 0.30 to 0.39 for the 3D-TOF evaluation versus 0.066 to 0.612 for the 2D-DSA evaluation. The differences observed were not significant. Nevertheless, the kappa value for evaluation by 2D-DSA of “complete” treatment was higher (Table 7).
Evaluation of Packing Protrusion.
For both readers, the evaluation of packing protrusion was good in MR imaging (k = 0.70) and excellent in 2D-DSA (k = 0.85), with no significant difference between the 2 techniques (P < .44).
Evaluation of Coil Extrusion into the Parent Artery.
The evaluation of coil extrusion into the parent artery was excellent (k = 0,84) when 2D-DSA was used (Fig 6), whereas it was poor when the 3D-TOF evaluation was used (k = 0.10; P < .001).
Evaluation of the Difference in Therapeutic Decision Depending on the Acquisition Mode.
Of the 64 decisions made, 7 depended on imaging mode. In cases of disagreement in the therapeutic decision, depending on the type of imaging used, 2D-DSA tended to influence the decision toward retreatment, whereas MR imaging tended to favor absence of therapy (P = .059; McNemar’s test).
In 6 cases, additional treatment was thought to be necessary in 2D-DSA and unnecessary in 3D-TOF. In only one case did the experts consider treatment necessary in 3D-TOF and unnecessary in 2D-DSA (Table 8).
Discussion
Comparison with Other Studies Performed
To the best of our knowledge, no studies have used expert evaluation of aneurysmal residue by using 2D-DSA and 3D-TOF imaging in comparison with a reference model. Recent in vivo studies compared a reference 2D-DSA with MR imaging.4,5,7 These studies do not allow a precise comparison of the images with a verifiable reference of aneurysmal residue. In some studies comparing the 2 techniques, the time intervals between the 2 studies vary from a few days to as long as several months, and modification of the aneurysmal packing between the 2 investigations is therefore possible.7 Other recent studies did not allow determination of the interobserver reproducibility essential to a critical evaluation of a technique.5,6,12 3D-DSA that has proved to be beneficial in assessing aneurysmal occurrence13–17 could not be included in our aneurysmal post-treatment evaluation. In our model, artifacts preclude its use as a method of aneurysm-filling evaluation.
Reproducibility
2D-DSA.
Intraobserver reproducibility of 2D-DSA evaluations was good. Interobserver reproducibility was good to excellent in analysis of a coil extrusion or packing protrusion into the parent artery as well as for evaluation of “complete” treatment. Analysis of a “partial treatment” or “residual neck” aneurysm showed little reproducibility from one reader to another. The distinction between a “residual neck” and “residual flow” within the mass of coils was often difficult (Fig 3).
As frequently noted in the literature,2,18 acquisition in 2 dimensions gives superimposition of the aneurysm sac and the parent artery and does not allow formal recognition of an additional image as flowing blood within the mass of coils.
Furthermore, arterial pulsation may be responsible for a slight discrepancy between acquisition of the mask and the injected series (Fig 4) and mimic an additional image difficult to differentiate from a “residual neck.”
3D-TOF.
In MR imaging, intra- and interobserver reproducibilities were low in analyses of the aneurysmal residue, irrespective of occlusion level. This result agrees with Leclerc et al19 and Derdeyn et al,7 who also found low reproducibility in MR imaging evaluation. This result is explained by the variability in choice of intensity and contrast levels between readings and readers in MIP study responsible for dispersion of the results. Van Hoe et al20 have already demonstrated similar results in an MIP study in arterial stenosis with an increase in dispersion depending on the decrease in vessel size.
Packing Protrusion.
Evaluation of the packing protrusion into the parent artery was well identified by both imaging modalities. We did, however, note the presence of false-positives in 3D-TOF studies, despite the absence of observable artifacts related to the mass of the coils. Leclerc et al19 and Derdeyn et al7 did not identify a “subjectively identifiable” loss of signal intensity around the packing, agreeing with observations by Hennemeyer et al,21 who only found “minor artifacts” of magnetic susceptibility. Therefore, we explain this loss of signal intensity as probable disturbed flow around the residual neck sufficient to affect flow in the parent artery (Fig 7).
Coil Extrusion.
Evaluation of coil extrusion was excellent in 2D-DSA because of the quality of definition (1024 × 1024) and speed of acquisition. MR imaging analysis was not reliable by using either the MIP reconstructions or source images (Fig 6). The lower image definition (matrix, 256 × 202) and variability in contrast thresholds may explain these differences.
Evaluation of the Aneurysmal Residue: Classification by Percentage
The mode of evaluation by percentage classes was found to show little agreement, despite grouping into very broad classes (<70%, 70%–90%, 90%–100%). The experts systematically overestimated the filling level whatever the mode of imaging. In the absence of silicon and despite a 12% packing for a wide-necked aneurysm, no expert gave an evaluation <60%–70% of filling. For the narrow-necked aneurysm with a 20% packing, no expert gave an evaluation <80%–90%.
2D-DSA.
According to Piotin et al,18 2D-DSA systematically overestimates of the aneurysm filling level. A 2D-DSA evaluation of 100% occlusion corresponds at best to a real filling of only one third of the aneurysm sac. This result is due to complete attenuation of the radiograph by the mass of coils, which forms a radiopaque cage, making it impossible to evaluate its contents.
3D-TOF.
Disturbed flow in an empty aneurysm leads to a loss of signal intensity that predominates in the center and proximal border of the sac and neck (Fig 8A). Gobin et al,22 Kerber and Heilman,23 and Sorteberg et al24 have shown that coils cause a major disturbance in the flow with stagnation in the dome even when the VER is low. These features easily explain overestimation of filling noted by the experts (Fig 8B).
Evaluation of the Aneurysmal Residue: Jean Raymond Grading Scale
2D-DSA.
By using the Raymond grading scale, “complete” treatments were correctly assessed by the experts. In contrast, the distinction between “residual aneurysm” and a “residual neck or dog ear” was not reliable.
The experts also overestimated the filling level, classing as a “residual neck” or “dog ear” treatments formally identified as “residual aneurysm” on the reference (Figs 3 and 5).
3D-TOF.
3D-TOF MR imaging studies did better than 2D-DSA in the identification of “partial” treatments on the wide-necked aneurysm (Fig 5). Anzalone et al,4 Leclerc et al,19 and Derdeyn et al7 reported cases of increased intensity MR imaging signals within the packing of aneurysm, which were considered entirely occluded by 2D-DSA.
In the absence of a better reference than angiography, these observations were reported as false-positive cases for the 3D-TOF MR imaging.
We postulate that in view of the significant differences in the kappa values that we obtained (0.68 MR imaging vs 0.041 2D-DSA; P = .018) it is valid to consider increased intensity MR signals as the manifestation of a residual flow within the mass of coil, especially if the aneurysm is a wide-necked and the initial VER is low.
In MR imaging, asymmetric underestimation of the neck size is observed when the aneurysm is empty. Flow is well perceived at the distal border of the parent artery and underestimated at the proximal border25–27 (Fig 8A). Because of the mass of coil, asymmetric flow dynamics were not observed during analysis and “residual neck” was described by the 2 experts, with no difference between distal and proximal border (Fig 8B).
Combined Reading
Combined reading of the 2 types of image did not improve agreement between the evaluations and reference measurements. The practitioners tended to suppress data supplied by the MR imaging in favor of the 2D-DSA angiographic images.
Repeat Treatment
Theoretically all “partial treatments” verified on cross-sections may be eligible for a second stage treatment. Although partial treatment is a potential indication for therapy, experts were concerned with treatment feasibility.
Only “partial treatments” were found suitable for retreatment by the experts. Above 50% filling, no additional treatment decision was carried out. At the same filling level, 2D- DSA analysis indicated additional treatment more often than 3D-TOF analysis (P = 0.058). A better description of the filling status of the aneurysmal neck in relation to the parent artery allows a greater confidence in the evaluation of whether an additional treatment is needed and possible.
Conclusion
In our in vitro study, the Raymond classification proved to be a more reproducible and precise evaluation tool than the percentage classes. 2D-DSA is the best imaging modality, because of better intra- and interobserver reproducibility and good agreement with the reference measurements in evaluating completely occluded aneurysms. Although the benefits of MR imaging were obvious to depict “residual aneurysm,” the distinction between “residual aneurysm” and “residual neck” was not reliable with either method of evaluation. 2D-DSA analysis led to more repeat treatments than 3D-TOF analysis.
This study confirms the need for angiographic control in the postoperative follow-up of coiled aneurysms according to the high level of concordance in complete treatment evaluation and stresses, at the same time, the benefits of MR imaging in the detection of partially treated aneurysms.
Acknowledgments
The authors would like to thank Boston Scientific, Signe Haughton, and Dr. Levy for their contribution in this work. This study has been done in the section “Biomorphology” of the Master of Science of University of Paris V, VII, and XIII.
References
- Received January 27, 2005.
- Accepted after revision May 18, 2005.
- Copyright © American Society of Neuroradiology