You are here

Article Text

Article menu

Download PDFPDF

Basic science
Original research
Flow diverters: inter and intra-rater reliability of porosity and pore density measurements
  1. B Farzin1,
  2. L Brosseau2,
  3. S Jamali1,
  4. I Salazkin1,
  5. A Jack3,
  6. T E Darsaut3,
  7. J Raymond1,2
  1. 1Laboratory of Interventional Neuroradiology, Centre Hospitalier de l'Université de Montréal Research Centre (CRCHUM), Montreal, Quebec, Canada
  2. 2Department of Radiology, Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal, Quebec, Canada
  3. 3Division of Neurosurgery, Department of Surgery, University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Alberta, Canada
  1. Correspondence to Dr J Raymond, CHUM—Notre-Dame Hospital, Interventional Neuroradiology (NRI), 1560 Sherbrooke East, Pavilion Simard, Room Z12909, Montreal, Quebec, Canada H2L 4M1; jean.raymond{at}umontreal.ca

Abstract

Background and purpose Porosity and pore density (PD) are important characteristics of flow diverters (FDs), because they may influence device efficacy and safety. Reliable measurement of these parameters would seem to be required for comparisons between devices, device selection at the time of clinical usage, as well as for research purposes. Because there is no standard method of measurement, our aim was to assess the intra-rater and inter-rater reliability of PD measurements and of three different ways of measuring porosity.

Methods Six microphotographs of two fully deployed FDs were taken overlying two different millimetric reference grids: one flat and the other corrected to match the cylindrical stent. Standardized protocols for independently measuring PD and porosity according to three different methods were used by three trained observers and by the same observer twice. Bland–Altman plots and intra-class correlation coefficients (ICC) were used to study the reliability of the measurements.

Results For porosity, satisfactory agreement occurred only when the same method of measurement was performed by the same observer. Intra-observer and inter-observer agreement were poor for measures of porosity when different methods were used (with differences in the range of 5–10%, ICC <0.6 for all methods). Measurement of PD was more reliable within (ICC 0.991 (0.946 to 0.999)) and between (ICC 0.945 (0.781 to 0.991)) observers.

Conclusions Without standardization, the porosity of different devices cannot reliably be compared because use of different methods or different observers substantially changes results. Pore density seems to be more reliably measured than porosity.

  • Aneurysm
  • Stent
  • Flow Diverter

https://doi.org/10.1136/neurintsurg-2014-011240

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Flow diverters (FDs) have successfully been used in various clinical contexts. They may play an important role in the management of difficult to treat aneurysms.1–5 FDs are stent-like structures, but they are much less ‘porous’ than stents used for coiling assistance or to treat stenotic lesions. The size, number, and distribution of struts and pores (or openings in the device), and the porosity at which a stent is considered an FD, have not been defined. At least four different devices are currently in clinical use or are under investigation in various countries, but all currently available FDs are constructions made of braided metallic filaments.5–8 A standardized way to characterize and measure, under comparable conditions, the structural properties of various devices, which are potentially related to their capacity to normalize flows and occlude aneurysms, is required.9 Reliable measurement of these architectural properties would seem necessary to compare devices, to aid clinicians in the proper selection of devices, or to predict device efficacy and safety in different clinical circumstances.

    Two commonly used terms to describe the generic characteristics of FDs are porosity and pore density (PD), but standard conditions to measure these quantities and the inter-rater reliability of the measurement of porosity and PD have not previously been reported.10Embedded Image

    The concept of device porosity (the proportion of open space at the surface of the cylindrical device)11 makes intuitive sense, but by itself porosity does not precisely and completely describe the ‘mesh’ of the FD as it does not take into account the caliber of the individual struts making up the mesh. PD (number of metal enclosed pores/mm2)10 was thus introduced to better capture this additional important characteristic of endoluminal devices.10 According to Augsburger, porosity is the main parameter influencing flow reduction, at least using in vitro models.12 Other authors claim that devices of equivalent or greater porosity, but with a higher PD, are more efficient in diverting blood flow and occluding aneurysms, at least in animal models.13 ,14

    Porosity and PD will vary, for the same device, according to the way the device is constrained to adapt to particular vascular geometries.15 Yet even without this additional complexity, we still require estimates of porosity and PD under normal or standard conditions, in order to be able to compare the generic properties of various devices. Estimates of porosity and PD of a particular device in normalized conditions are important for research purposes, in order to define and correlate the device variables that will influence experimental results,9 ,16 in the quest to determine the optimal FD characteristics for effective aneurysm occlusion. Proper estimates of porosity and PD may eventually also be important for clinicians to consider when choosing which device to deploy in a particular patient; an FD with a very low porosity and high PD may have a greater propensity for aneurysm occlusion but perhaps at an increased risk of branch occlusion.9 Some preclinical evidence suggests that complex vascular geometries, such as more acute arterial curves and wider aneurysm necks, may require devices of lower porosity (and higher PD) for successful occlusion.9 ,17 ,18 To date, manufacturers have not included these important measures on their product inserts, perhaps in part due to the absence of standardization and to the difficulties inherent in measuring these variables. In our early preclinical experiences, we recognized that the precise and reliable measurement of porosity and PD is not straightforward. We initially attempted, unsuccessfully, to use Image J automated analyses to quantitate FD porosity, as has been described,19 our main difficulties lying in setting the threshold to differentiate strut from pore, as many struts were affected by artifacts from reflected light. We thus sought to measure porosity and PD with manual methods, and recognized that there were several methods to measure these variables, each of which potentially yielded different results.

    The objective of this paper became to study the reliability of various simple methods of measurement of porosity and PD obtained by various observers, and to determine which methods were most reliable when comparing device characteristics.

    Materials and methods

    Two different FDs, the Silk device (4×20 mm; Balt, Montmorency, France) and a 3.75×28 mm prototype FD similar to the Pipeline (Covidien, Irvine, California, USA) constructed of 48 braided wires 35.6 μm in caliber (MV48) (Microvention Inc, Tustin, California, USA) were photographed unconstrained (at rest) on a millimetric paper reference grid. To correct for the curvature of the device, measurements were also taken with the paper reference grid inserted inside the device and conforming to the cylindrical curvature (figure 1A, B).

    Figure 1
    Figure 1

    Methods. The Silk flow diverter (A) or the MV48 flow diverter (B) was measured three times with a flat (A) or a cylindrical reference grid (B). Three methods were used to calculate porosity: method M1, where the total pore area was measured (sum of all yellowed areas between the struts in C*); method M2, where the total strut area was measured (sum of all yellowed areas of the struts in D*) (porosity obtained with M2=1−total strut area); and by method M3 (E*), obtained by measuring the total length and width of all of the struts within the reference square to obtain the total strut area, and subtracting an area of (strut width×strut width) for each incidence of strut overlap. Porosity obtained using method M3=1−total strut area. *Images in (C–E) are enlarged to promote clarity of explanation.

    Microphotographs of three areas of interests per device, per reference grid, were captured to minimize sampling variations between observers. Three observers (BF, LB, SJ; all post-doc students or research associates) were provided with the microphotographs and a standardized protocol, a manual of procedures detailing how to select regions of interest (ROIs) using Image J software (US National Institutes of Health), and instructed to independently measure porosity according to three different methods, using the following definitions: (i) ROI: an area to be selected and independently traced by each observer; (ii) reference square: an area of dimension 1 mm2 within which the ROIs would be selected, traced, and summed; (iii) pore: open space bounded by the device struts; (iv) total pore area: the summed area of individual pores within the reference square; (v) total strut area: the summed area of individual struts within the reference square; (vi) strut length: length of device strut from one boundary of the reference square to another (figure 1E); and (vii) strut overlap: where one device strut crosses (and covers) another device strut (also shown on figure 1E).

    The first method, method 1 (M1), detailed measurement of the pores (open spaces) between the photographed metallic struts within a square millimeter reference square. Each observer used the ROI manager feature of the Image J software to trace all of the areas of open space between device struts, which were then summed to obtain the total pore area (figure 1C). When pores were transected by the boundaries of the reference square, the pore area outside the boundary of the reference square was not included in the final sum. Method 2 (M2) detailed the measurement of the surface area of the metallic struts within the same square millimeter reference square (M2=1−metallic density) (figure 1D). In this manner, method M2 was similar to M1, but it was the surface area of the struts that each observer traced with the Image J software, rather than the open surface area of the pores. Method 3 (M3) required observers to measure the total length and width of device struts from one boundary of the reference square to the other (figure 1E). The measured width of each strut was multiplied by the measured total length of all struts present within the reference square to obtain the total strut area. Each observer thus measured the total length of the struts present within the reference square, multiplied by the width of the struts. However, in multiple places, the struts crossed each other, which would erroneously increase the total strut surface area. Thus to correct for the area where struts overlapped, an equilateral parallelogram of surface area (strut width×strut width) multiplied by the number of counted incidences of strut overlaps within the reference square was subtracted from the strut surface area obtained.

    PD, defined as the number of pores per millimeter square (complete or partial), was also measured using the same microphotographs. Complete pores were defined as those pores which fell entirely within the reference square used for measurement, while a partially enclosed pore was defined as a pore which was artificially divided by the arbitrary boundaries of the reference square. The values of PD presented in table 1 were obtained by counting all of the pores (complete and partial) within a reference square, recognizing that the choice to include the partial pores (they were counted just as complete pores) in the value of PD would artificially increase the estimate of PD. To offset this overestimate, a correction factor for the calculation of PD for each FD was determined as follows: Embedded Image where *=within the reference square; **=sum of surface area of all full pores/number of full pores; and ***=obtained using method M1 to measure porosity.

    View this table:
    Table 1

    Measures of porosity and pore density obtained with curved (C) and flat (F) millimetric reference grids

    The correction factors for the MV48 and Silk FDs were 0.641 and 0.667 respectively, which signifies that the counted pores, which were contributing to the PD estimate, were artificially increased by a factor of 1.56 and 1.50, for the MV48 and Silk FDs, respectively.

    All observers used calibrated windows over the best quality portion of any given image. All measurements were documented and their quality reviewed by one investigator (BF). To estimate intra-observer agreement, the same observer (BF) performed the same measurements twice independently in two sessions 1 month apart.

    From preliminary studies, we estimated that a total sample of 36 measurements obtained from three observers (for inter-rater reliability), and studying a sample of 12 photographs analyzed using three different methods, would suffice to find a difference of 5% or more between porosity measurements.20 Means and mean differences were calculated between observers and between methods. The agreement between methods and between observers was estimated by visual inspection of Bland–Altman plots and by calculating intra-class correlation coefficients (ICC) with 95% CIs for absolute agreement between measurements, using a two way model (same raters for all subjects), with MedCalc software V.12.7.5. ICCs >0.6 were considered ‘satisfactory’ for this study. The manuscript was prepared in accordance with the Guidelines for Reporting Reliability and Agreement Studies.21

    Results

    The porosity and PD estimates of the photographed FDs obtained by three observers, using various methods, for the two devices, are summarized in table 1.

    Intra-observer agreement

    The intra-observer repeatability of measurements of porosity by the various methods was satisfactory for each method considered separately: ICCs were estimated at 0.912 (0.690 to 0.974) for M1; 0.870 (0.501 to 0.965) for M2; 0.928 (0.752 to 0.979) for M3. There was almost perfect agreement between the two measures of PD rated by the same observer in two different sessions (ICC 0.991 (0.946 to 0.999)).

    Large differences in the measurements of porosity were found when different methods were employed by the same rater. The porosity estimates obtained from measuring open spaces (M1) were lower than those obtained from measuring strut surfaces (M2) or from calculating strut lengths (M3) by a mean of 8.2% (15.0% to 1.3%) (figure 2A). The agreement between the porosities obtained with methods M2 and M3 was satisfactory (ICC 0.904 (0.677 to 0.972)) but poor when comparing methods M1 and M2 (ICC 0.0878 (0 to 0.451)).

    Figure 2
    Figure 2

    Bland–Altman plots show intra-observer (A, B) and inter-observer agreement (C, D) between method 1 (M1) and method 2 (M2) (A), between M1 measured with a flat (M1a) or cylindrical (M1b) grid (B), between M1 measured by the two most discrepant raters (C), and between M1 and M2 for all observers (D).

    The agreement between porosity measurements obtained with or without correcting for the cylindrical curvature, using the same observer, was satisfactory (0.960 (0.897 to 0.987)), with minimal to moderate differences between measurements (mean −0.7%; −5.4% to 4.1%) (figure 2B).

    Inter-observer agreement

    The inter-observer agreement of measuring porosity was poor when using M1 and M2: ICCs were estimated at 0.131 (−0.0565 to 0.618) for M1 and −0.007 (−5.462 to 0.860) for M2. Inter-observer repeatability was somewhat better using M3 (0.649), although there was a wide CI (−0.035 to 0.939).

    The porosity estimates obtained using method M1 differed by up to 9.0% (5.2% to 12.8%) between the two most discrepant readers (figure 2C).

    Major differences in the measurement of porosity were found when different methods were used, for all observers. Porosity estimates obtained from open spaces (M1) were constantly lower than those obtained from strut surfaces (M2) by a mean of 6.1% (0.3% to 11.9%) (figure 2D). For all observers, the estimates of porosity differed significantly between M1 and M2, with poor agreement between methods (ICC 0.478 (0.184 to 0.812)).

    The agreement between porosity measurements taken with or without correction for the cylindrical curvature of devices for all observers was satisfactory (0.961 (0.897 to 0.987)).

    The agreement between PD measurements between observers was satisfactory (ICC 0.945 (0.781 to 0.991)). The mean difference between observers 1 and 2 was 0.2 pores/mm2 (−3.3 to 3.0), and between observers 2 and 3, 2.8 pores/mm2 (−6.0 to 11.9).

    Discussion

    The most important finding of this study is that different methods of measuring FD porosity will yield substantially different results, both when the same observer uses different methods and when different observers use the same method. Maximal discrepancies between observers were found with direct measurement of open spaces within a sample area. For all observers, measuring metallic density instead of ‘open spaces’ porosity led to overestimates of device porosities, in the range of 6–12%, compared with direct porosity methods, a discrepancy likely to be clinically significant. To provide a reference for the importance of this discrepancy, animal studies have shown that 3% difference in porosity can differentiate devices that occlude aneurysms from those that do not.9 PD measurements were more repeatable, within and between observers.

    The difficulties with measuring porosity and PD can be understood by analyzing the different components of the problem. The first problematic aspect is the use of a two-dimensional measure (surface area) to calculate porosity while, in reality, a stent or FD is a cylindrical or curved tubular structure that occupies three-dimensional space. One means to overcome this problem is to render the tubular device two-dimensional (to assume the sampling area is flat) when taking measurements. This simple work around is thought to contribute little error when measuring the porosity for any single device; in the present work, falsely assuming the surface was flat underestimated porosity by approximately 1% (−5.4% to 4.1%). We have not examined more complex constructions, but it is safe to assume that when multiple overlapping devices are treated with this assumption, or when complex devices composed of stents in stents are used, the validity of a method that treats a multilayered porous tubular structure as if it were a two-dimensional surface area is increasingly questionable.9

    The second readily apparent problem is that of sampling error, both in terms of size and location of the reference square used to calculate porosity and/or the number of pores/area unit of measure. This reference square can introduce variability depending on where it is placed over the FD mesh, which is not necessarily homogeneous.

    The concept of PD is not as intuitive as the concept of porosity but it seems to be simpler to measure repeatedly by the same and different observers. The placement of a reference square over a two-dimensional photograph of a stent nonetheless created a methodological difficulty with the measurement of PD; some pores were artificially divided by the reference square, creating the problem of whether what is inside the reference square is a ‘true, metal enclosed pore’ or a ‘partially metal enclosed pore’. We chose to measure all pores, complete or partial, but provided a correction factor in order to facilitate comparisons with other studies.

    The problem of complete versus partial pores is compounded when multiple devices are telescoped, when metallic struts cross but at different depths in the sampling area.

    Although the concept of porosity makes intuitive sense, direct methods of measuring porosities are fastidious and time consuming. From our study of inter-rater variability, it would seem that the more fastidious the method, the more likely for results to differ between observers. Other more expedient analytic methods based on device construction exist, such as calculations based on a measurement of the angle between struts along the long axis of the stent. Computer assisted pixel quantification of digitized images is also possible, and if the same software is used with the same manual of procedures, this approach could minimize inter-observer variations.19 ,22–25

    Although these methods may permit comparison of different dispositions of the same device deployed in arteries of various diameters, the general formula for such estimates may not universally apply to different devices without modifications, and cannot be used to compare devices of more complex architectures or for multiple telescoping devices. Perhaps some of these shortcomings can be overcome using micro-CT technology, as has been previously suggested.26–28

    Other ways to characterize FDs could be conceived to improve reliability of measurements, such as the densitometry of the radiographs of the devices. Another potential solution would involve, for each new device, comparisons with a ‘standard FD’ (akin to the ‘standard meter’), made by measuring porosity and PD by the same observer using the same methodology. Perhaps porosity and PD, which are in actuality surrogate characteristics for the flow diverting capacity of a particular device, should be replaced by direct in vitro testing using a standardized model.13

    Limitations

    This study has several limitations. Repeated measurements of microphotographs is a very fastidious task, and for pragmatic reasons we only employed a small number of observers. The inter-rater variations we observed combined both systematic and random errors, and it may be impossible to reliably disentangle the two. One systematic type of error we attempted to isolate was the one related to measurements with the flat reference grid, which in this case was in the range of 1–2%. Only two devices were studied. If confirmation of the reliability of measurements would have required a larger number of observers, measuring an even greater variety of devices, this small study was nonetheless sufficient to show an alarming divergence in the estimates obtained by trained observers. It is important to emphasize that the wide variability of porosities measured by multiple observers occurred in the simplest setting possible. The fact that we minimized sampling variability by providing observers with the same photographs of the same devices could only lead to underestimates of variability. In addition, because FDs are self-expanding devices, they exhibit different zones of diverse metallic density, according to the size and shape of the vessel into which they are deployed.15 ,25

    Conclusion

    Comparisons of published values of porosity for devices measured by different observers or companies using various methods are unlikely to be valid. PD seems to be more reliably measured. Standardized methods for providing measures of porosity and PD for each device when deployed at nominal size are needed if we want to communicate research findings, clinical or preclinical, using common terms and language.

    References

    1. ↵
      1. Becske T,
      2. Kallmes DF,
      3. Saatci I,
      4. et al
      . Pipeline for uncoilable or failed aneurysms: results from a multicenter clinical trial. Radiology 2013;267:858–68. doi:10.1148/radiol.13120099
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Byrne JV,
      2. Beltechi R,
      3. Yarnold JA,
      4. et al
      . Early experience in the treatment of intra-cranial aneurysms by endovascular flow diversion: a multicentre prospective study. PLoS ONE 2010;5:pii: e12492. doi:10.1371/journal.pone.0012492
      OpenUrlCrossRef
    3. ↵
      1. Nelson PK,
      2. Lylyk P,
      3. Szikora I,
      4. et al
      . The pipeline embolization device for the intracranial treatment of aneurysms trial. AJNR Am J Neuroradiol 2011;32:34–40. doi:10.3174/ajnr.A2335
      OpenUrlAbstract/FREE Full Text
    4. ↵
      1. Pistocchi S,
      2. Blanc R,
      3. Bartolini B,
      4. et al
      . Flow diverters at and beyond the level of the circle of Willis for the treatment of intracranial aneurysms. Stroke 2012;43:1032–8. doi:10.1161/STROKEAHA.111.636019
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Szikora I,
      2. Berentei Z,
      3. Kulcsar Z,
      4. et al
      . Treatment of intracranial aneurysms by functional reconstruction of the parent artery: the Budapest experience with the pipeline embolization device. AJNR Am J Neuroradiol 2010;31:1139–47. doi:10.3174/ajnr.A2023
      OpenUrlAbstract/FREE Full Text
    6. ↵
      1. Byrne JV,
      2. Szikora I
      . Flow diverters in the management of intracranial aneurysms: a review. EJMINT Original Article 2012:1225000057. (22nd June 2012)
    7. ↵
      1. De Vries J,
      2. Boogaarts J,
      3. Van Norden A,
      4. et al
      . New generation of flow diverter (Surpass) for unruptured intracranial aneurysms: a prospective single-center study in 37 patients. Stroke 2013;44:1567–77. doi:10.1161/STROKEAHA.111.000434
      OpenUrlAbstract/FREE Full Text
    8. ↵
      1. Diaz O,
      2. Gist TL,
      3. Manjarez G,
      4. et al
      . Treatment of 14 intracranial aneurysms with the FRED system. J Neurointerv Surg 2014;6:6147.
      OpenUrlAbstract/FREE Full Text
    9. ↵
      1. Darsaut TE,
      2. Bing F,
      3. Salazkin I,
      4. et al
      . Flow diverters can occlude aneurysms and preserve arterial branches: a new experimental model. AJNR Am J Neuroradiol 2012;33:2004–9. doi:10.3174/ajnr.A3075
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Lieber BB,
      2. Sadasivan C
      . Endoluminal scaffolds for vascular reconstruction and exclusion of aneurysms from the cerebral circulation. Stroke 2010;41(10 Suppl):S21–5. doi:10.1161/STROKEAHA.110.595066
      OpenUrlAbstract/FREE Full Text
    11. ↵
      1. Lieber BB,
      2. Stancampiano AP,
      3. Wakhloo AK
      . Alteration of hemodynamics in aneurysm models by stenting: influence of stent porosity. Ann Biomed Eng 1997;25:460–9. doi:10.1007/BF02684187
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      1. Augsburger L,
      2. Farhat M,
      3. Reymond P,
      4. et al
      . Effect of flow diverter porosity on intraaneurysmal blood flow. Klin Neuroradiol 2009;19:204–14. doi:10.1007/s00062-009-9005-0
      OpenUrlCrossRefPubMed
    13. ↵
      1. Rayepalli S,
      2. Gupta R,
      3. Lum C,
      4. et al
      . The impact of stent strut porosity on reducing flow in cerebral aneurysms. J Neuroimaging 2013;23:495–501. doi:10.1111/jon.12044
      OpenUrlPubMedWeb of Science
    14. ↵
      1. Sadasivan C,
      2. Cesar L,
      3. Seong J,
      4. et al
      . An original flow diversion device for the treatment of intracranial aneurysms: evaluation in the rabbit elastase-induced model. Stroke 2009;40:952–8. doi:10.1161/STROKEAHA.108.533760
      OpenUrlAbstract/FREE Full Text
    15. ↵
      1. Makoyeva A,
      2. Bing F,
      3. Darsaut TE,
      4. et al
      . The varying porosity of braided self-expanding stents and flow diverters: an experimental study. AJNR Am J Neuroradiol 2013;34:596–602. doi:10.3174/ajnr.A3234
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Darsaut TE,
      2. Bing F,
      3. Makoyeva A,
      4. et al
      . Flow diversion of giant curved sidewall and bifurcation experimental aneurysms with very low porosity devices. World Neurosurg 2013: Sep 23. pii: S1878-8750(13)01224-2. doi: 10.1016/j.wneu.2013.09.036. [Epub ahead of print] Review.
    17. ↵
      1. Gentric JC,
      2. Darsaut TE,
      3. Makoyeva A,
      4. et al
      . The success of flow diversion of large and giant sidewall aneurysms may depend on the size of the defect in the parent artery. AJNR Am J Neuroradiol 2014. Jun 26. [Epub ahead of print].
    18. ↵
      1. Darsaut TE,
      2. Bing F,
      3. Salazkin I,
      4. et al
      . Flow diverters failing to occlude experimental bifurcation or curved sidewall aneurysms: an in vitro study in canines. J Neurosurg 2012;117:37–44. doi:10.3171/2012.4.JNS111916
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Radaelli AG,
      2. Augsburger L,
      3. Cebral JR,
      4. et al
      . Reproducibility of haemodynamical simulations in a subject-specific stented aneurysm model—a report on the Virtual Intracranial Stenting Challenge 2007. J Biomech 2008;41:2069–81. doi:10.1016/j.jbiomech.2008.04.035
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      1. Obuchowski NA
      . How many observers are needed in clinical studies of medical imaging? AJR Am J Roentgenol 2004;182:867–9. doi:10.2214/ajr.182.4.1820867
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Kottner J,
      2. Audigé L,
      3. Brorson S,
      4. et al
      . Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol 2011;64:96–106. doi:10.1016/j.jclinepi.2010.03.002
      OpenUrlCrossRefPubMed
    22. ↵
      1. Yodfat O,
      2. Lieber B,
      3. Grad Y,
      4. et al
      . Implantable intraluminal device and method of using same in treating aneurysms. EP1455679B1, filed 6 November 2002. http://www.google.com.ar/patents/EP1455679B1?cl=en
    23. ↵
      1. Wang K,
      2. Huang Q,
      3. Hong B,
      4. et al
      . Correlation of aneurysm occlusion with actual metal coverage at neck after implantation of flow-diverting stent in rabbit models. Neuroradiology 2012;54:607–13. doi:10.1007/s00234-011-0922-7
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      1. Wang K,
      2. Yuan S
      . Actual metal coverage at the neck is critical for flow-diverting stents in treating intracranial aneurysms. AJNR Am J Neuroradiol 2013;34:E31–2. doi:10.3174/ajnr.A3451
      OpenUrlFREE Full Text
    25. ↵
      1. Shapiro M,
      2. Raz E,
      3. Becske T,
      4. et al
      . Variable porosity of the pipeline embolization device in straight and curved vessels: a guide for optimal deployment strategy. AJNR Am J Neuroradiol 2014. Apr;35(4):727–33. doi: 10.3174/ajnr.A3742. Epub 2013 Sep 26.
      OpenUrlAbstract/FREE Full Text
    26. ↵
      1. Mortier P,
      2. De Beule M,
      3. Van Loo D,
      4. et al
      . Automated generation of a finite element stent model. Med Biol Eng Comput 2008;46:1169–73. doi:10.1007/s11517-008-0410-3
      OpenUrlPubMed
    27. ↵
      1. De Beule M,
      2. Van Cauter S,
      3. Mortier P,
      4. et al
      . Virtual optimization of self-expandable braided wire stents. Med Eng Phys 2009;31:448–53. doi:10.1016/j.medengphy.2008.11.008
      OpenUrlCrossRefPubMed
    28. ↵
      1. Huang Q,
      2. Xu J,
      3. Cheng J,
      4. et al
      . Hemodynamic changes by flow diverters in rabbit aneurysm models: a computational fluid dynamic study based on micro-computed tomography reconstruction. Stroke 2013;44:1936–41. doi:10.1161/STROKEAHA.113.001202
      OpenUrlAbstract/FREE Full Text

    Footnotes

    • Contributors Conception, design, interpretation, and drafting of the manuscript: BF, TED, IS, and JR. Acquisition and interpretation of the data: SJ, LB, and BF. Acquisition of the data and literature search: LB, SJ, and AJ. Final approval of the manuscript submitted: BF, JR, and TED.

    • Competing interests None.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement Additional raw data from each experiment are available from the corresponding author on request.