Endovascular Coiling of Cerebral Aneurysms Using “Bioactive” or Coated-Coil Technologies: A Systematic Review of the Literature

Materials and Methods

Data Extraction

All studies that met the primary criteria were then formally assessed, and data were extracted by 1 of the authors (P.M.W.) by using a standardized critical appraisal and data-extraction form (Fig 1). As a validation of the data-extraction process, a second author (J.R.) reviewed a random sample of 20% of the studies. The second author (J.R.) independently reviewed all the data extracted and assigned a second score to each study.

• Download figure
• Open in new tab
• Download powerpoint

Fig 1.

Data extraction proforma.

The data-extraction form was subdivided into 3 basic sections: 1) design and baseline population characteristics, 2) procedure, and 3) outcomes. A score of 0–3 (0 = inadequate data to assess, 1 = poor, 2 = acceptable, 3 = good) was assigned for each category (Fig 1). This proforma method has been well described and enabled an objective and reproducible assessment to be made of each article.^7-10 The quality scoring system was intrinsically weighted so that some items on the data-extraction form carried more weight than other items (Table 1). Weighting was used because some items relate more strongly to the reliability of results obtained than others—for instance, it is likely that authors will reliably report the coil studied (low weighting) but independent core lab assessment of angiographic outcomes is more reliable than operator self-assessment (high weighting).¹¹ To score full (3) marks in a category, a study had to meet most of the highly weighted criteria and most of the lower weighted ones. To score 2 in a category, a study had to meet a majority of the high- and some lower weighted criteria and so on.

View this table:

• View inline
• View popup

Table 1:

Data extracted from studies meeting primary inclusion criteria

An additional score (0, 0.5, or 1) was assigned for “overall impression” based on how many of the following standards were judged to have been met: adequate reference list, objective discussion, discussion of limitations of the study, logical clear presentation of data, providing numerators/denominators as well as percentages, placing the study in context, and impact factor of the journal >1.8. The possible appraisal score was 0–10, and a final percentage score was determined on the basis of the mean of the 2 quality scores awarded by the reviewers. A score of >40% was required for inclusion. When the reviewers’ disagreed on inclusion/exclusion, this disagreement was resolved by a consensus review of the article and the data-extraction forms. We did not set the bar for inclusion very high at all, and most studies following the usual scientific conventions of data description and presentation should have met these quality criteria.

Results

Results of the computerized search of EMBASE and MEDLINE are given in Table 2. Two hundred twenty-eight references for “coated” coil used in cerebral aneurysm treatment were identified by the search strategy. Removing duplicates and irrelevant references, such as letters or editorials or other reports that did not meet the primary inclusion criteria, left 32 studies to be critically reviewed. The sensitivity of the electronic search strategy was 97% (it identified 31/32 studies).

Of these studies, 15 were on Matrix (PGLA-coated coils from Boston Scientific, Natick, Mass),^12-26 11 were on hydrogel-coated coils (HydroCoil embolic system; MicroVention, Aliso Viejo, Calif),^27-37 1 was on both hydrogel and Matrix,³⁸ 2 were on Sapphire fibered coils (ev3, Irvine, Calif),^39,40 and 3 were on Cerecyte (PGA-coated coils; Micrus, Sunnyvale, Calif).^41-43 The study on PGLA- and hydrogel-coated coils was excluded because it was impossible to separate information on the 2 coil types from bare platinum data, let alone from each other; thus in terms of results by coated coil, it was impossible to assess. One study on PGLA scored 30% and was excluded, 1 short report on PGA-coated coils scored 30% and was excluded, and 1 early technical report on hydrogel-coated coils scored 20% and was excluded. One fibered coil article was a similar series from the same unit, so the report focused on fibered coils was the 1 formally reviewed. In just 1 case (3%), the reviewers disagreed on inclusion/exclusion, and this disagreement was resolved by a consensus review with an agreement not to include it. The other 26 studies were included. Median quality assessment scores for included studies were 60% (range, 45%–90%) with no difference in median scores between studies on different coil types.

Discussion

First, we need to issue a strong “health warning”: These data were extracted from uncontrolled mostly self-assessed case series and are, at best, exploratory. Far from guiding clinical actions, they can be used only to provide hypotheses for future trials. There were extreme differences in patient and aneurysm characteristics, in study methodologies (generally poor), and quantities of data between studies and cohorts, so these results must be viewed with considerable caution. In particular, the amalgamated coil series were vastly different for many risk factors that are believed to have impact on procedural and periprocedural morbidity and mortality and on initial and follow-up angiographic results, including the proportion of ruptured/unruptured cases, the number of aneurysms in various size categories, and the proportion of patients followed up.⁴⁴ Very incomplete data were available on assist-device usage. Furthermore, there was considerable inconsistency in the data reported (demonstrated by the varying numbers of studies reporting the different outcomes assessed and thus the varying denominator values in Tables 3–5). One particularly disappointing feature was the overall low level of clinical and angiographic follow-up reported. It is clearly unacceptable that the overall angiographic follow-up rate reported on these new unproven technologies was so low at 56% and the duration of follow-up, so short at 7.3 months.

None of the data published on coated coils to the end of 2007 were from randomized controlled trials. This in itself is an indictment of the state of scientific evidence in the neurointerventional field, despite the good example set early on in the coiling experience by the International Subarachnoid Aneurysm Trial (ISAT),^4,5 albeit things are now improving. The absence of control groups and the lack of randomization are not the sole deficiencies identified. By the Cochrane review methodology, all studies reviewed were classified as level C with a high risk of bias.⁹ Only 3 studies (9%) had any independent assessment of outcomes (2 studies on hydrogel-coated coils and 1 on PGLA coils had an independent core lab). Only 6/26 studies had a quality score above 60%, and 9/26 had a score of ≤50%. Only 1/26 studies had a domain score of 3 (good) in >1 data category, despite a very generous review/scoring policy.

Despite the avowed aims of assessing “safety and efficacy,” none of the series attempted to prespecify 1 or more research questions that would 1) allow an estimate of the sample size necessary to answer these questions, 2) control the error that would come with the data, and 3) set a limit to the claims of the conclusions. Claims of efficacy and safety are unsubstantiated and based on series too small to conclude anything with any confidence. More disturbing, in the light of current policies regarding clinical research involving human subjects, is the fact that though most authors would acknowledge the lack of evidence regarding the safety and efficacy of the new material in the introduction of their report, they would still proceed with an uncontrolled study (which could be called an “experiment”) on their patients. In most cases, this was done without any statement regarding a submission to ethics review and without, presumably therefore, obtaining a specific research-related informed consent from participants, which would acknowledge that the authors were trying a new material with unknown risks. This may perhaps be explained by confusion between using devices that have passed arbitrary (and often deficient) regulatory requirements and what it is ethical to do in clinical practice and research. The systematic review only represents a very small proportion of patients who have been submitted to the new materials over a 5-year period, without any clear evidence of benefit. This leads to a feeling of disquiet, to say the least, once the reader realizes that dozens of institutions and physicians were simultaneously performing poorly planned “experiments,” without any possibility of providing a convincing answer to important clinical questions and mostly without consulting and collaborating with each other. One must come to the conclusion that neurointerventionists, as a group of clinicians respectful of their patients, must find a way to do things better.^1-3,45

So if it is a case of generally poor data in, is it necessarily poor data out and thus a waste of time doing a systematic review? Is it possible to turn the flock of ugly ducklings into swans?

Because the structured quality assessment process, by using a specially designed data-extraction form, results in an objective measure of study quality, the method itself provides a useful analysis of existing literature.⁴⁶ There are moderately large numbers of patients/aneurysms to draw on (2390/2532, respectively) from nearly 30 peer-reviewed studies. Despite many limitations, a systematic review methodology may allow a useful overview of new-device use and some comparisons to be made, though one must be disciplined to refrain from hasty conclusions. So, if anything can be said about this literature, how do the coated-coil technologies compare? Due to the limitations of the raw data, we are here condemned to proceed with questionable comparisons and extrapolations, attempting to adjust for various risk factors purported to have an impact on complications and recurrences.

Overall procedural morbidity/mortality rates are comparable at 2.9%–5.4% and are reassuringly in line with those from the bare platinum literature and early randomized controlled trial evidence for coated coils.^4,6,44,47-50 The fluctuations between coated-coil types in perforation and thromboembolic complication rates are modest and likely to represent statistical noise (because numbers are rather small), the presence of heterogeneity, or bias. One must remember that very large controlled trials are necessary to exclude a clinically significant difference in events with such a low frequency. Hydrocephalus was reported for hydrogel-coated coils in only 4/10 studies (at a 3.5% rate overall) but was just not mentioned in any studies on the other coated coil types. Therefore, the systematic review can add no useful information on the relationship between coated coils and hydrocephalus rates. Given the discrepancies between case series and the fact that divergences for various risk factors can pull in different directions, trying to adjust for heterogeneity becomes a highly speculative endeavor. We have attempted to provide comparisons between coils by using “virtual control” platinum groups adjusted for only 1 factor, the size categories. Most studies of angiographic remnants/recurrences in bare platinum coils indicate rates of 15%–19% by 6 months and approximately 20% by 18 months with far higher rates in large than in small aneurysms.^44,47-54

Thus Fig 2 is a comparison between the observed results for coated-coil cohorts and the expected results for bare platinum coils for similar aneurysm-size cohorts—the “virtual control” platinum groups, with the assumption made for bare platinum coils of a 15% recurrence rate for small/medium aneurysms, a 30% recurrence rate for large aneurysms, and a 50% rate for giant aneurysms. Again one must remember that this is not hard evidence, but extrapolation. An increased prevalence of large/giant aneurysms would be expected to result in higher recurrence and retreatment rates because it has been demonstrated that lesions prone to recurrence such as these have a 34%–50% recurrence rate.⁴⁴ Conversely, series of smaller aneurysms were also prone to include a larger proportion of ruptured aneurysms, another risk factor for recurrences.^47,48 Despite the very different aneurysm size profiles, the angiographic follow-up results for hydrogel- and PGA-coated coils compared with bare platinum are similar (Fig 2). In comparison, PGLA coils, with an in-between aneurysm size profile, had significantly lower complete/near-complete occlusion rates. However, the marked disparities between the aneurysm cohorts may account for most of the differences despite the calculated P values; the cohorts are particularly discrepant between PGA-coated coils and the others.

• Download figure
• Open in new tab
• Download powerpoint

Fig 2.

Incomplete occlusion rate observed with coated coils by comparison with the incomplete occlusion rate expected for a “virtual series” of bare platinum coils in a size-matched aneurysm cohort.

The retreatment rate is a relatively hard end point. An aneurysm either has been retreated or it has not. However, retreatment policies do vary substantially between units.^55,56 Due to the heterogeneous mix of aneurysms and the wider use of assist devices in the systematic review, it is not possible to draw any meaningful comparisons with the ISAT data on retreatment rates (where retreatment in the endovascular arm was 11% at 1 year but in a very different aneurysm population).^4,5,55 The retreatment rate in aneurysms followed up was 13% for PGLA, 5% for hydrogel, and 2% for PGA. The problem of not comparing like with like in uncontrolled series means that the significantly lower retreatment rate for hydrogel- and PGA-coated coils compared with PGLA-coated coils needs to be viewed with skepticism. On the flawed evidence to date, overall the retreatment rate with coated coils seems to be little/no improvement over bare platinum despite advances in coil technologies (complex shapes, coils of varying softness) and access/assist devices all aimed at reducing recurrence and retreatment rates. Explanations for this might include the facts that early PGLA studies were on the Matrix Version 1, which was undoubtedly more difficult to use and pack with than the revised Version 2, and that early hydrogel studies used the much more limited range of hydrogel-coated coils then available, which were stiffer and harder to obtain gel-neck coverage—a factor linked to outcome in 1 independently assessed registry study.²⁷

Reassuringly, reported rebleed rates in the patients with SAH were low, which is in line with the bare platinum coiling literature,^2,55,56 albeit follow-up in the coated-coil data is limited to the first year or less. There was no appreciable difference among coil types for which data were available, but only 19/26 studies documented rebleeds.

Conclusions

The published literature to the end of 2007 provides no high-quality (grade A or B) evidence for the safety or efficacy of bioactive/coated coils. Conversely, there is little indication that coated coil technologies are associated with increased procedural complications but also little evidence for any appreciable improvement in angiographic outcomes. This solid randomized controlled trial evidence with independently assessed outcomes is urgently needed. Until available, it is the authors’ opinion that these coils should be used sparingly, if at all, outside an ethically and scientifically approved randomized trial (or in exceptional circumstances in other ethically approved studies with independent outcome assessment and informed patient consent). This review clearly demonstrates the past failure of the device industry, the regulatory authorities, and the neurointerventional community combined, to provide a prudent approach to the introduction of new technologies and to obtain reliable knowledge on new devices, and it provides a good illustration of the hype effect.⁵⁷ As a community of physicians and allied professionals, we must find ways to coordinate our efforts to progress in a more scientific and ethical manner for the ultimate benefit of our patients.