Quality-Evaluation Scheme for Cerebral Time-Resolved 3D Contrast-Enhanced MR Angiography Techniques

BACKGROUND AND PURPOSE: No practical tool has been reported in the literature to evaluate the quality of cerebral TR-3D-CE-MRA techniques. Our study assessed a large list of parameters used to propose a quality-evaluation scheme for TR-3D-CE-MRA. MATERIALS AND METHODS: A large list of visual and quantitative parameters used to study the quality of images was collected from the literature and evaluated in 19 healthy patients and 11 patients with arteriovenous shunts who had undergone both CENTRA keyhole TR-3D-CE-MRA at 3T and CCA. Several observers evaluated the visual parameters, such as the diagnostic confidence index, artifacts, maximum vascular signal intensity, arterial-to-venous separation, and visibility of 17 arteries and 7 veins; and quantitative parameters, such as maximum arterial SI, arteriovenous transit time, arteriovenous contrast curve, and ADW. A statistical analysis was used to determine interobserver reproducibility of the visual parameters, to calculate the sensitivity of TR-3D-CE-MRA for detecting each vessel (with CCA as standard of reference), and to compare the results of the visual and quantitative evaluations. RESULTS: Diagnostic confidence index, artifacts, arterial-to-venous separation, and 4 vessels—the PICA, ophthalmic and occipital arteries, and the ISS—demonstrated high reproducibility and sensitivity. The ADW was the most reliable dynamic quantitative parameter and was correlated with arterial-to-venous separation. CONCLUSIONS: The image quality of TR-3D-CE-MRA can be effectively evaluated with a scheme of 1 quantitative and 7 visual parameters.

T R-3D-CE-MRA, a multiphase acquisition technique, enables a rapid sequential analysis of arteries, capillaries and veins and facilitates the identification of early venous filling or retrograde flow investigated in a volumetric study. Parallel imaging associated with k-space sampling strategies at a high field strength has recently improved the compromise between spatial and temporal resolution. So, TR-3D-CE-MRA techniques have rapidly increased in popularity, and a recent study 1 showed that TR-3D-CE-MRA detected additional relevant clinical findings that were not seen on the routine contrast-enhanced protocol in 48% of 126 patients. TR-3D-CE-MRA has already proved helpful and effective for the etiologic diagnosis of intracranial hematoma, 2 for the classification and follow-up of AVMs [3][4][5] and dural fistulas, 6 and for the characterization of hypervascular tumors. 7 The use and evaluation of cerebral TR-3D-CE-MRA are clearly reflected in the literature, with approximately 50 articles on the technique since it was first described in 1996, 8 approximately half of which were published during the past 2 years. In most publications evaluating TR-3D-CE-MRA and often comparing it directly with CCA, the extensive anatomic and hemodynamic information provided by this technique is assessed and rated according to subjective and constantly changing parameters. Consequently, evaluation of the image quality of TR-3D-CE-MRA techniques is generally incomplete and differs from 1 article to another. To our knowledge, no robust, easy-to-use, practical, and explicated tool for the qualitative evaluation and comparison of TR-3D-CE-MRA sequences, based on a few reproducible parameters, has yet been proposed. The aim of this study was to evaluate the image quality of TR-3D-CE-MRA by using CENTRA keyhole at 3T from a large list of parameters collected from the literature in healthy patients and in patients with arteriovenous shunt and to propose a robust quality-evaluation scheme usable in practical routine.

Materials and Methods
The methodology of this study included 2 steps: 1) a systematic review of the recent medical literature to collect a large list of visual and quantitative parameters to evaluate TR-3D-CE-MRA images, and 2) an evaluation of CENTRA keyhole TR-3D-CE-MRA by several observers, in both healthy patients and those with pathology who had also undergone CCA, to determine a robust quality-evaluation scheme with the most reproducible and sensitive parameters.

Systematic Review of the Medical Literature and Listing of Parameters
The first step in our methodology, which took place in January 2009, was to search MEDLINE databases by using terms such as "cerebral MRA" with or without "contrast agent," "time-resolved imaging," "parallel imaging," and "dynamic or digital subtraction angiography." From approximately 100 potentially relevant articles, we reviewed the 32 publications 2-6,9-35 that proposed or described a qualitative evaluation of images. We then listed all the reported parameters used to evaluate image quality in terms of morphologic and hemodynamic information. The parameters were classified into 2 groups: visual evaluation and quantitative evaluation.

Visual Evaluation
The first part evaluated the overall image quality with 4 parameters: 1) Diagnostic confidence index. 4,11,12,14,17,21,31,33,35 Grading was as follows: 0, no vascular study possible; 1, vascular study possible but with low diagnostic confidence; 2, vascular study possible with adequate diagnostic confidence; and 3, vascular study possible with high diagnostic confidence.
3) Maximum arterial and venous SI. 20,29,31 Grading was as follows: 0, poor, only slightly higher than the background SI; 1, average, clearly higher than the background SI; and 2, major, optimal for anatomic vascular study. 4) Arterial-to-venous separation during the arterial phase. 5,12,21,30,34,35 Grading was as follows: 0, poor, Յ2 phases without venous contamination; 1, good, 3 phases without venous contamination; and 2, excellent, Ն4 phases without venous contamination. This parameter reflected the temporal resolution and determined the ability of the TR-3D-CE-MRA to identify the rapid arteriovenous shunt involved in most vascular malformations. A recent study 30 justified the threshold of 3 pure arterial phases because of evaluated optimal acceleration (ie, not affecting vessel visibility) at 2.6 arterial phases.
The second part was an anatomic vascular study, which evaluated the vascular visibility of vessels of different diameters, the visibility of the smallest reflecting spatial resolution, and from different territories. Most vessels were listed from the literature, 9,10,13,17,22,26,30,34 but some were added to complete the list. The visibility of venous drainage and of small-diameter arteries has important clinical implications because it can help to determine the risk of neurologic complications and the therapeutic strategy for vascular malformations. 4,6,19,29 The arteries were divided into 2 classes according to their diameter: 1) Large: proximal segments of the anterior cerebral artery (A1), MCA (M1), and posterior cerebral artery (P1).
The veins were divided into 2 classes according to their location: superficial venous drainage (SSS and TS) or deep venous drainage (ISS, internal cerebral vein, basal vein, great cerebral vein, and SR). The ISS and internal cerebral and basal veins were considered to be the smallest venous segments.
The mean visibility scores of 17 arteries and 7 veins were rated on the following 3-point scale according to both length and intensity of enhancement: 0, no visibility; 1, partial visibility (incomplete or complete but with low intensity); and 2, good visibility (complete and with high intensity).

Quantitative Evaluation
Dynamic enhancement curves were generated from subtracted images with regions of interest placed on the supraclinoidal portion of the ICA and the middle third of the SSS, in the sagittal plane. The region of interest was chosen to cover a large part of the vascular lumen with careful exclusion of voxels containing partial volume and to provide an easily reproducible location. 3,24 We defined arterial and venous SI and arteriovenous contrast with methods excluding noise estimation 3,11,12,24 because of the inhomogeneous distribution of noise throughout the image due to parallel imaging. 36 Arterial and venous SI curves were obtained, and 4 quantitative parameters were plotted (Fig 1) 23 as follows: ADW ϭ AUC a50 -100 Ϫ (AUC v100 -AUC v50), meaning the AUC of 50%-100% of the maximum arterial SI minus Ͼ50% of the venous SI. The ADW included the previous 3 parameters and theoretically represented good arterial filling with no significant venous contamination. 23 The number of phases and the mean AVC in the ADW were noted.

Quality-Evaluation Scheme of CENTRA Keyhole TR-3D-CE-MRA
This large list of parameters was applied to evaluate the TR-3D-CE-MRA images from a retrospective selection of 2 cohorts of patients, 1 without and 1 with vascular pathology. Both cohorts were included to assess the robustness of the evaluation scheme, in comparison with CCA (considered as the standard of reference for the anatomic vascular study).
Patients. Retrospectively, we selected 2 cohorts of consecutive patients studied with both cerebral TR-3D-CE-MRA and CCA between December 2007 and May 2008 in the neuroradiology department. The first cohort included 19 consecutive patients (7 females and 12 males; median age, 36 years; range, 8 -68 years), mostly investigated because of cerebral hematoma, whose CCA images showed no significant abnormalities and who were therefore considered healthy. The interval between TR-3D-CE-MRA and CCA was always Ͻ7 days. The second cohort included 11 consecutive patients (6 women and 5 men; median age, 47 years; range, 22-69 years) with vascular diseases involving arteriovenous shunt: 6 untreated AVMs, 2 recurrent or residual AVMs treated with both embolization and radiosurgery, and 3 untreated arteriovenous dural fistulas. Patients with arteriovenous shunt were included because this pathologic context is a frequent indication for performing TR-3D-CE-MRA and CCA. [3][4][5][6]14,16,19,29 The interval between TR-3D-CE-MRA and CCA was always Ͻ14 days.
The view order acquisition was as follows: the central part of k-space was sampled 25 times with a temporal resolution of 1.2 seconds, followed by a 6.2-second reference dataset with full k-space sampling. Finally, a total of 25 dynamic high-resolution 3D datasets were reconstructed by using missed data provided by the last reference dataset. 37 Total imaging time was 36.2 seconds. Parameters were optimized for high isotropic spatial resolution while maintaining satisfactory temporal resolution. TR-3D-CE-MRA was initiated simultaneously with a 20-mL gadoterate meglumine (Dotarem; Guerbet, Aulnay-Sous-Bois, France) bolus, administered intravenously at a rate of 3 mL/s by using a Spectris power injector (MedRad, Indianola, Pennsylvania), followed by a 30-mL saline flush at the same rate.
After the TR-3D-CE-MRA sequence, the data were transferred to a workstation (View Forum, Philips Healthcare). Image processing included the following techniques: The first dynamic volume (mask) was subtracted from the subsequent 3D volumes to remove any background intensity (magnitude subtraction). Automated reconstruc-tions of the MIPs were then generated in the coronal, sagittal, and axial planes.
CCA. CCA was performed with a 4F catheter via a femoral artery approach (Advantx LCA/LCVϩ; GE Healthcare, Milwaukee, Wisconsin) with a filming rate of 2-5 images per second. CCA included a selective injection of the ICA, ECA, or vertebral arteries in anteroposterior and lateral views, supplemented by additional views when necessary. Images had a matrix of 1024 ϫ 1024 and an FOV of 17 cm. For each projection, an 8-or 9-mL bolus of iodinated contrast material (iobitridol, Xenetix; Guerbet) was injected at a rate of 4 -6 mL/s. Image Analysis. Analysis of the visual parameters was performed on the TR-3D-CE-MRA images by 2 observers (with Ͼ10 years' experience): a neuroanatomist and neurosurgeon (X.M.) and a neuroradiologist (J.-Y.G.). They were both blinded to the CCA findings and to the results of both studies and independently reviewed the TR-3D-CE-MRA data from 3 MIP-subtracted orthogonal images (sagittal, coronal, and axial). Unsubtracted images were not analyzed because they are not routinely available on postprocessing stations. 30 Disagreements between observers were reviewed by both readers to reach a consensus. For the evaluation of overall image quality in the patients with pathology, the observers focused on the vessels that were not involved in the vascular malformations. For the anatomic vascular study, the CCA images, considered as the standard of reference, were reviewed by a third experienced neuroradiologist (J.-C.F.) who was unaware of the TR-3D-CE-MRA findings. The visibility of 17 arteries and 7 veins was rated on a 3-point scale similar to that in the evaluation of the TR-3D-CE-MRA. The quantitative evaluation was performed by a fourth radiologist (H.R.), by using subtracted images, once only, because the automatic computation was an objective and reproducible method. 11,12,15,23,24,30
The sensitivity of TR-3D-CE-MRA (with CCA used as the standard of reference) for detecting each vessel was calculated: A partially or completely visible vessel was considered as detected and a nonvisible vessel, as undetected. 13 A sensitivity of Ͼ70% was considered significant.
Last, a comparative analysis was performed to detect whether there were significant correlations between the results for the different visual parameters, between the results for the 4 quantitative parameters, or between the results for the different visual and quantitative parameters. The Mann-Whitney U test or the Spearman correlation coefficient was used as appropriate.
The statistical analysis was performed by using MedCalc for Windows (MedCalc Software, Version 9.5.2.0; Mariakerke, Belgium) and the Statistical Package for the Social Sciences, Version 16.0 (SPSS, Chicago, Illinois) software.

Visual Evaluation
Healthy Patients. Successful evaluation was achieved for TR-3D-CE-MRA and CCA images in all healthy patients (Fig 2). The overall image-quality evaluation showed that the diagnostic confidence index for images was high in 8 patients (42%), adequate in 10 patients (53%), and low in 1 patient (5%). We noted 2 cases of moderate artifacts: 1 aliasing artifact and 1 vascular edge artifact during the early arterial phase. Maximum arterial SI was graded as major in 13 (68%) and average in 6 (32%) patients. Maximum venous SI was graded as major in all 19 patients (100%). Arterial-to-venous separation quality was graded as excellent in 11 patients (58%) and good in 8 patients (42%). Interobserver agreement was good for the diagnostic confidence index, artifacts, and arterial-tovenous separation grading ( ϭ 0.68, 0.63, and 0.60, respectively); fair for maximum arterial SI grading ( ϭ 0.30); and not assessable for maximum venous SI.
The anatomic vascular study (Tables 1 and 2) involved a total of 322 arteries and 133 veins on the TR-3D-CE-MRA images, among which only 288 arteries and 121 veins were compared with the CCA images, due to the localized study on the CCA. On the CCA images, 95% of vessels were detected and 5% were undetected. Three arteries (the PICA and occipital and ophthalmic arteries) and all veins achieved very good interobserver agreement and a sensitivity of Ͼ70% compared with the CCA.
The comparative analysis showed 2 significant results: First, the diagnostic confidence index was strongly correlated with the presence of artifacts (Spearman coefficient: r ϭ 0.58, P 0.01). Second, the only case of the low diagnostic confidence index corresponding to the single aliasing artifact case was moderately correlated with the visibility of the PICA (r ϭ 0.60, P ϭ .06).
Patients with Pathology. Successful evaluation was achieved for TR-3D-CE-MRA and CCA images in all patients with pathology (Figs 3 and 4). The overall image-quality evaluation showed that the diagnostic confidence index for images was high in 6 patients (55%) and adequate in 5 patients (45%). We noted 2 cases of moderate artifacts: 1 aliasing artifact and 1 vascular edge artifact. Maximum arterial SI was rated as major in 8 (73%) and average in 3 (27%) patients. Maximum venous SI was graded as major in all 11 patients (100%). Arterial-to-venous separation quality was graded as excellent in 7 patients (64%), good in 3 patients (27%), and fair in 1 patient (9%). As in the healthy patients, interobserver agreement was good for the diagnostic confidence index, artifacts, and arterial-to-venous separation grading ( ϭ 0.64, 0.74, and 0.69, respectively); fair for maximum arterial SI grading ( ϭ 0.23); and not assessable for maximum venous SI.
The anatomic vascular study (Tables 3 and 4) involved a total of 183 arteries and 77 veins on the TR-3D-CE-MRA images, among which only 174 arteries but all veins were compared with the CCA images. On the CCA images, 97% of vessels were detected and 3% were undetected. Five arteries (the PICA and ophthalmic, calcarine, occipital, and maxillary arteries) achieved very good interobserver agreement and a sensitivity of Ͼ70% compared with evaluation with CCA. Only 3 were the same as those in the healthy patients: the PICA and ophthalmic and occipital arteries. Among veins, unlike those in healthy patients, all segments were reproducible except the internal cerebral and basal veins.
The comparative analysis showed 2 significant results: The diagnostic confidence index was strongly correlated with the presence of artifacts (r ϭ 0.81, P ϭ .002) and highly correlated with the visibility of the PICA (r ϭ 0.68, P ϭ .02).

Comparison of Visual and Quantitative Evaluations
In both the healthy patients and those with pathology, the comparative analysis between the visual and quantitative parameters demonstrated some significant results: The ADW as the number of phases and the mean AVC during the ADW were significantly higher when the arterial-to-venous separation was rated excellent (Mann-Whitney U test, P Ͻ .006 and Ͻ .008).

Discussion
In the literature, several subjective and changing parameters but no practical tool exists to evaluate the quality of cerebral TR-3D-CE-MRA techniques. Our study shows that a robust quality-evaluation scheme with only 7 highly reproducible and sensitive visual parameters and 1 reliable quantitative parameter can be effectively used to assess TR-3D-CE-MRA images in practical routine. Because the scheme evaluates overall quality and spatial and temporal resolution, it can be considered as complete.
For overall image-quality evaluation, 3 parameters, the diagnostic confidence index, artifacts, and arterial-to-venous separation, showed high reproducibility in both healthy patients and those with arteriovenous shunt. Diagnostic confidence index grading of images was recommended by most authors 4,11,12,17,21,31,33,35 because an initial appraisal is necessary before a more specific evaluation. The grading provides an immediate estimate of the level of diagnostic accuracy of the images to characterize possible vascular disease. The artifacts were correlated with the diagnostic confidence index in our study, and this confirmed the literature data suggesting that artifacts may degrade image quality and lead to misinterpretation. 11,14,22,28,30,31,33 The type of artifacts, such as aliasing, ringing, movement, or vascular edge, must be specified because it is influenced by the TR-3D-CE-MRA strategy used. 11,35,[37][38][39] Arterial-to-venous separation, assessed with a semiquantitative scale, 5,11,12,30,34 provides a quick evaluation of the practical temporal resolution (ie, the ability of TR-3D-CE-MRA to  detect rapid arteriovenous shunt involved in certain vascular malformations). However, our study shows no obvious interest in visually grading maximum arterial or venous SI. This is because visual grading of maximum arterial SI is not sufficiently reproducible and it is not correlated with the quantitative value. Also, the maximum venous SI is not discriminative enough because it is graded as optimal in all healthy patients and in those with pathology, which is in accordance with the literature data. 9 For practical spatial resolution evaluation, in both healthy patients and those with arteriovenous shunt, the following 4 vessels showed very good reproducibility and a high sensitivity of detection in comparison with CCA along with high spatial discrimination: 3 arteries, the PICA and the ophthalmic and occipital arteries, and 1 vein, the ISS, considered to be the smallest venous segment (noting that the internal cerebral and basal veins are not highly reproducible in patients with pathology). These 4 vessels provide high spatial discrimination with a diameter in most cases Ͻ1 mm. 3,13 Moreover, the occipital artery is often involved in arteriovenous dural fistulas. 6,19 These 4 selected vessels provide an efficient evaluation that is sufficiently short to be used in clinical routine. However, despite being longer, the evaluation of additional vessels might be advantageous in specific pathologic situations (eg, the evaluation of the maxillary artery if involved in a dural fistula) and thus might help to determine if images are of reasonable quality.
Only 1 quantitative parameter appears relevant: the ADW, the dynamic parameter representing high and lasting arteriovenous contrast with a high arterial SI. 23 In our study, the ADW was correlated with visual arterial-to-venous separation, and it provided higher and more objective accuracy than the visual parameters. The ADW could, therefore, be calculated to compare different types of TR-3D-CE-MRA strategy. 23 Another benefit of the ADW is that it includes the notion of arteriovenous transit time, which may be very important for arteriovenous shunt evaluation. However, it involves a more complicated process than visual evaluation.
It is difficult to provide an image-quality analysis system that is sufficiently general and practical and that addresses the particular features of all the numerous techniques already described in the literature. First, if the types of artifacts we have listed are adapted to current TR-3D-CE-MRA sequences, they might need to be updated along with the technologic innovations. The artifacts we observed in our study were related to the sequence that combined CENTRA keyhole and sensitivity encoding. This sequence was chosen because it delivers high performance, as demonstrated in recent studies. 4,11 The specificity of CENTRA keyhole is to place the emphasis on contrast data (central k-space) rather than on spatial resolution data (peripheral k-space). The observed high contrast values enable an accurate investigation of the vascular anatomy and the vascular malformations but limit the visibility of small-diameter arteries. Other undersampling strategies may provide better spatial resolution but at the expense of contrast. Moreover, CENTRA randomly acquires the central k-space with assumed smoother transition of central and peripheral data 37 and so reduces some artifacts, such as ringing and vascular edge artifacts, 11,37,38 which may occur in echo-sharing MRA techniques with peripheral k-space segmentation, such as timeresolved imaging of contrast kinetics or a time-resolved echoshared angiography technique. 39,40 However, the specific temporal footprint 35 of keyhole, with sampling of peripheral k-space temporally following sampling of central k-space, is also probably responsible for the anticipation phenomenon we observed with a faint venous signal intensity on our initial arterial images. The duration of and order of views within the temporal footprint are specific to the view-sharing technique used and may influence image quality. 35 Therefore, these parameters will have to be considered in the methodology of any future evaluation study of TR-3D-CE-MRA. The proposed scheme evaluates their impact on image quality, especially due to the parameters of diagnostic confidence index, arterial-to-venous separation, and artifacts. 35 Second, our results must be interpreted knowing the injection protocol used (20-mL gadolinium bolus at 3 mL/s). In fact, the TR-3D-CE-MRA sequences can achieve diagnostic accuracy with a lower dose of contrast agent and faster injection, particularly with new high-relaxivity contrast agents. Third, the small size of our study population limited the statistical power and parametric tests. Last, the quality of individual TR-3D-CE-MRA depends not only on the methodologic approach but also on the individual arterial conditions (eg, age-dependent blood flow in cerebral vessels). For this reason, to cover a maximum of situations encountered in clinical practice, we have studied 2 different groups of patients. The pathologic group did not include all cerebrovascular diseases but focused on cerebral arteriovenous shunt because of the frequency of this indication.

Conclusions
The quality of TR-3D-CE-MRA can be effectively evaluated in clinical routine with a scheme of 1 quantitative and 7 visual parameters. In the future, this scheme should prove helpful for   evaluating current TR-3D-CE-MRA techniques and comparing technologic advances improving spatial or temporal resolution with an optimal trade-off, such as high-field highly accelerated MRA with Ն32 coil elements and also new contrast agents.