Abstract
BACKGROUND AND PURPOSE: 4D-DSA is a time-resolved technique that allows viewing of a contrast bolus at any time and from any desired viewing angle. Our hypothesis was that the information content in a 4D-DSA reconstruction was essentially equivalent to that in a combination of 2D acquisitions and a 3D-DSA reconstruction.
MATERIALS AND METHODS: Twenty-six consecutive patients who had both 2D- and 3D-DSA acquisitions were included in the study. The angiography report was used to obtain diagnoses and characteristics of abnormalities. Diagnoses included AVM/AVFs, aneurysms, stenosis, and healthy individuals. 4D-DSA reconstructions were independently reviewed by 3 experienced observers who had no part in the clinical care. Using an electronic evaluation form, these observers recorded their assessments based only on the 4D reconstructions. The clinical evaluations were then compared with the 4D evaluations for diagnosis and lesion characteristics.
RESULTS: Results showed both interrater and interclass agreements (κ = 0.813 and 0.858). Comparing the 4D diagnosis with the clinical diagnosis for the 3 observers yielded κ values of 0.906, 0.912, and 0.906. The κ values for agreement among the 3 observers for the type of abnormality were 0.949, 0.845, and 0.895. There was complete agreement on the presence of an abnormality between the clinical and 4D-DSA in 23/26 cases. In 2 cases, there were conflicting opinions.
CONCLUSIONS: In this study, the information content of 4D-DSA reconstructions was largely equivalent to that of the combined 2D/3D studies. The availability of 4D-DSA should reduce the requirement for 2D-DSA acquisitions.
DSA is the criterion standard for the angiographic evaluation of cerebrovascular diseases. Current clinical use of DSA is based on a combination of multiple 2D projections and 3D-DSA volume reconstructions. 3D-DSA is not time-resolved. Often, several 2D acquisitions are necessary because of vascular overlap or due to the inability to visualize a particular attribute or structure due to rapid vascular filling. 4D-DSA is a reconstruction technique that provides time-resolved 3D reconstructions (ie, 4D-DSA).1 Initial studies indicated that the ability to view the contrast bolus at any time from any angle provided by 4D-DSA eliminates the problems inherent in the use of 2D- and 3D-DSA (eg, vascular overlap).2,3
To our knowledge, no formal comparison has been made of the utility of 4D-DSA compared with conventional 2D or 3D images. The purpose of our pilot study was to assess the hypothesis that the information content of a 4D-DSA reconstruction was equivalent to that of the combination of 2D- and 3D-DSA. If this hypothesis is correct, then it should be possible to reduce the number of 2D-DSAs required in diagnostic and therapeutic procedures. This then would translate into a saving in radiation exposure, contrast medium, and procedural time.
Materials and Methods
Under an institutionally approved protocol, we retrospectively evaluated 26 patients who had undergone diagnostic angiograms as part of their routine evaluations from August 2013 through March 2015. We wanted our cohort to consist of subjects with the most common conditions subjected to conventional angiography at our institutions and attempted to do this by selecting consecutive patients who fell into the diagnostic categories of healthy, aneurysm, AVM, or AVF and also had ≥1 3D-DSA reconstruction. Our subjects included 9 with aneurysms, 8 healthy individuals, 6 with AVMs/AVFs, and 3 with vascular stenosis/occlusions. The 3D-DSAs from these subjects were then retrospectively reconstructed as 4D-DSAs (at the time of data collection, 4D-DSA was not approved for clinical use and thus was not available real-time in the angiography suite).
The final operative/radiology report was used to obtain the diagnosis and measurements that were made with the 2D + 3D combination. Once cases were collected and properly de-identified per existing institutional review board guidelines at the University of Wisconsin - Madison, the rotational projections from each 3D acquisition were transferred to a research workstation running both the commercial software (syngo X Workplace VB21; Siemens, Erlangen, Germany) and the 4D-DSA prototype software. These were then reconstructed as 4D-DSA with the prototype software. These 4D-DSA volumes constituted the test images that were evaluated in the study.
The cases were maintained on a research workstation and were reviewed independently by 3 fellowship-trained neurointerventionalists (2 neuroradiologists and 1 neurosurgeon) who had not been involved in any aspect of the patient's clinical care. No clinical information was provided to these reviewers, and the cases were presented in random order. As in a clinical environment, the evaluators had the opportunity to customize and use all the functional features of the 4D-DSA reconstructions in the workstation (ie, window, level, volume clipping, and so forth) to study each case and complete an evaluation form.
An electronic evaluation form was applied by using a “drill-down” methodology in which specific aspects of the vascular pathology (if any) were answered on the basis of the evaluator's previous selections. For example, for the vascular malformation cases, the form took the evaluator through the pertinent questions starting with the presence or absence of an abnormality and, if present, which type of abnormality and then a series of morphologic and measurement questions identical to those included in the clinical report. A sample of the questionnaire is shown in Fig 1.
Screenshot of the evaluation form used in the evaluation of the 4D-DSA reconstructions.
Study data were collected and managed by using Research Electronic Data Capture (REDCap) electronic data capture tools (https://catalyst.harvard.edu/services/redcap/) hosted at the University of Wisconsin Institute for Clinical and Translational Research with grant support (Clinical and Translational Science Award program, through the National Institutes of Health National Center for Advancing Translational Sciences, grant UL1TR000427). REDCap4 is a secure, Web-based application designed to support data capture for research studies, providing an intuitive interface for validated data entry, audit trails for tracking data manipulation, and export procedures for seamless data downloads to common statistical packages.
Diagnosis, diagnostic confidence (ie, abnormality present: “definitely not,” “probably not,” “unsure,” “probably yes,” and “definitely yes”), and the abnormality characteristics derived from the evaluation forms were compared within evaluators and with the clinical reports.
Statistical Analysis
Statistical analysis included using the Fleiss κ for assessing interrater agreement for multiple evaluators for the likelihood of an abnormality (5 category scales and 3 category scales) and type of abnormality.5 The likelihood of an abnormality was also treated as a numeric outcome, and interrater agreement was assessed with intraclass correlation coefficient (2,1) as defined by Shrout and Fleiss.6,7 The evaluator responses for the likelihood of an abnormality were condensed to 2 categories, to assess agreement of the likelihood of an abnormality on 4D with the 2D + 3D criterion standard diagnosis of abnormality. “Definitely yes” and “probably yes” were considered “yes” for presence of abnormality, and “definitely no” and “probably no” were considered “No.” The 1 response of “unsure” was treated as an incorrect assessment and was, therefore, defined as “yes” when the 2D + 3D diagnosis was “no” abnormality. Agreement in the abnormality diagnosis between 4D and 2D + 3D was assessed with the individual Cohen κ for each evaluator and for a consensus of the 3 evaluators. Sensitivity, specificity, positive/negative predictive values, and area under the ROC curve were calculated individually for each evaluator and for the consensus measure. Agreement about the abnormality type between 4D and 2D + 3D was assessed individually for each evaluator and for the consensus with the Fleiss κ. “Consensus” refers to the agreement between the 4D evaluators' assessments and the clinical diagnosis.
Statistical analysis was conducted by using R statistical and computing software, Version 3.0 (http://www.r-project.org/). An additional R package other than the base package was interrater reliability (irr).6⇓–8
Results
To determine whether 4D images alone were sufficient to answer the 2 main diagnostic questions, whether there is an abnormality in the study and, if so, the type of abnormality and its characterization, we analyzed the 26 datasets according to the methods just described.
Is There an Abnormality in the Study?
Tables 1 and 2 show the results for each evaluator. A correct 4D evaluation was defined as the evaluator's response equal to “definitely yes” or “probably yes” when the results of the standard method were “definitely yes,” or the evaluator's response being equal to “definitely no” or “probably no” when the results of the standard method were “definitely no.” The Fleiss κ for interrater agreement was 0.559, and the intraclass correlation coefficient (2,1) for interrater reliability of ordinal categoric data was 0.882. Consolidating the 4D responses to “yes” = “definitely yes” or “probably yes” and “no” = “definitely no” or “probably no” yielded a Fleiss κ for interrater agreement of 0.813. Intraclass correlation coefficient (2,1) for interrater reliability of ordinal categoric data was similar at 0.858.
Results of the 3 evaluators decisions regarding the presence of an abnormality on the 4D studies
Results of the 3 evaluator's decisions when results were consolidated into responses of yes, unsure, and no
After comparing interrater reliability, we then proceeded to analyze the agreement between the clinical diagnosis of 2D + 3D and the evaluators' conclusions based on 4D. To assess this agreement, we assumed that categorizing the 4D image as “definitely yes” or “probably yes” matched a true diagnosis of “definitely yes” and that categorizing the 4D image as “definitely no” or “probably no” matched a true diagnosis of “definitely no.” A consensus among the 3 evaluators was defined as a classification on 4D imaging that was the same for at least 2 of the 3 evaluators. Table 3 summarizes the data. κ values to assess the diagnostic agreement of evaluators 1, 2, and 3 and consensus against the standard are 0.906, 0.912, 0.906, and 1, respectively.
Summary of correctly identifying abnormality by evaluator and consensusa
Sensitivity, specificity, positive predictive value, negative predictive value, and area under the ROC curve for each evaluator and consensus 4D imaging results are provided in Table 4. Unsure responses are considered incorrect classifications.
Evaluator's sensitivity, specificity, positive predictive value, negative predictive value, area under the curve, and consensus results
Which Type of Abnormality Was Present in the Study?
Table 5 summarizes the results for each evaluator and the correct responses compared with the standard. The Fleiss κ for interrater agreement was 0.884.
Summary of each evaluator's resulta
Regarding agreement of the type of abnormality (when one was found) between the 4D evaluation and the actual clinical diagnosis based on the 2D + 3D studies, Table 6 shows a summary of these data. The Fleiss κ for diagnostic agreement of true abnormality type for evaluators 1, 2, and 3 and consensus was 0.949, 0.845, 0.842, and 0.895, respectively.
Summary of agreement between 4D and 2D/3D evaluations regarding type of abnormality (when there was one)a
In summary, there was complete agreement on the existence of an abnormality between the 4D and the clinical evaluations in 23/26 cases (88.5%). In 2 cases, there were conflicting opinions. There were 2 false-positive diagnoses, 1 because of mistakenly identifying occlusive stenosis disease and the other because of classifying the origin of a lenticulostriate artery as a small MCA aneurysm. In the third case of disagreement, one of the evaluators identified a small area of stenosis in an MCA branch, which indeed was present, and, on retrospective evaluation, was, because of overlap, not initially recognized on the 2D and 3D images. This was seen on the 4D image as a result of delayed filling of an opercular branch of the MCA. Of the 18 cases with a vascular abnormality present, agreement on the exact abnormality was diagnosed in 16 (88.8%). The other 2 cases were ones in which only one of the evaluators' assessments agreed with the clinical diagnosis. In one of these, 1 evaluator correctly identified an abnormality as an AVF, while the other 2 considered it an AVM. The other case was an internal carotid dissection with an arterial stenosis and a pseudoaneurysm; only one of the evaluators graded and scored the stenosis associated with the dissection (the patient was evaluated because of ischemic symptoms), while the other 2 based their assessments on the pseudoaneurysm (which was also present).
Characteristics, such as aneurysm dimensions, nidus size, feeding and draining pedicles, and the degree of stenosis were similar to those in clinical reporting, and there were no disagreements at a level that would have impacted clinical management. The sample size was too small for any meaningful statistical analysis of these subgroups.
Discussion
DSA remains the criterion standard for the evaluation of cerebrovascular pathology. Current practice usually involves the use of a combination of multiple conventional 2D projections and 3D-DSA volume reconstructions that are not time-resolved. In our small pilot study, 4D-DSA reconstructions contained, in most cases, information content that was equivalent to that of the combination of the 2D and 3D acquisitions that were used for the clinical evaluations.
The benefits of 3D-DSA compared with 2D-DSA are well-recognized.9 One of these is the potential for a reduction in the number of 2D acquisitions required for a particular study. While the 3D reconstructions provide the ability to view an abnormality from any desired angle, they do not provide a remedy for the overlap of vascular structures that are either situated in close proximity to an abnormality (eg, a complex MCA trifurcation aneurysm) or that fill extremely quickly, thereby preventing visualization of important angioarchitectural features (eg, intranidal aneurysms or venous obstructions in an AVM nidus).
By providing fully time-resolved 3D volumetric images of the vasculature, 4D-DSA allows a viewer to follow a contrast bolus through the vasculature. First described in 2012, 4D-DSA is now available on commercial angiographic systems. While experience with this application is still limited, early evaluations have shown some advantages over conventional 3D-DSA in the evaluation of complex AVMs and AVFs.1,10⇓–12 In our experience and in previous reports, the 4D studies have been particularly helpful in the assessment of the angioarchitecture of the nidus of AVMs and the location of the fistula connections in AVFs.10,11
One motivation for the development of 4D-DSA was to provide a method that would allow a viewer to see an abnormality not just from any desired viewing angle but also at any time during the passage of a contrast bolus through the vasculature (Figs 2 and 3).
A comparison of conventional 3D-DSA (upper row) with different timeframes from a 4D-DSA reconstruction (lower row). The projections of the 2 image types are identical. The 3D-DSA images allow viewing from any desired angle; however, in this case because of the vascular overlap, it is impossible to clearly see the relationship of the small aneurysm (yellow circle) to its parent artery. Because the 4D images allow viewing not only from any desired angle but also at any time during the passage of a contrast bolus through the vasculature, the early timeframes of the 4D images allow clear visualization of these relationships.
A, 3D-DSA of an AVM supplied by 2 branches of the right MCA. Despite the excellent image quality, the image is a composite of all of the 2D projections used in the reconstruction. The overlap of vascular components in the nidus precludes identification of intranidal aneurysms, the direct AVF, or venous outflow stenosis. B, Two views of early timeframes from a 4D-DSA reconstruction of the AVM shown in part A. The projections for this reconstruction are obtained at 30 frames per second. The number of each timeframe is shown beside each image. The angioarchitecture of the nidus can be clearly seen. The 4D-DSA images may be viewed from any desired angle at any time in the bolus passage.
In our opinion, the availability of the 4D-DSA should further decrease the need for multiple 2D acquisitions. The benefit derived would be a reduction of patient and physician radiation exposure, contrast dose, and even the length of the procedure.
Limitations of our study include the small sample size and a lack of information regarding the prevalence of patients in whom it will be difficult or impossible to obtain a satisfactory rotational acquisition.
Conclusions
In this small study, the information obtained from 4D-DSAs alone was largely equivalent to that of the combined 2D- and 3D-DSA studies. Use of 4D-DSA should reduce the requirement for 2D-DSA acquisitions.
Footnotes
Disclosures: Carolina Sandoval-Garcia—RELATED: Grant: AANS/CNS cerebrovascular section, Comments: AANS CV section/CNS Robert J. Dempsey, MD, Cerebrovascular Research Award, $15,000 awarded for resident research.* Sebastian Schafer—UNRELATED: Employment: Siemens Healthineers, Comments: full-time employee of Siemens Healthineers as identified in the affiliations. Azam Ahmed—UNRELATED: Grant/Grants Pending: National Institutes of Health.* Charles Strother—RELATED: Other: Siemens, Comments: contract for research support.* *Money paid to the institution.
Funding for this work was provided by RO1HL116567. Partial funding was provided by Siemens Healthineers AX. Erlangen, Germany.
Indicates open access to non-subscribers at www.ajnr.org
REFERENCES
- Received September 26, 2016.
- Accepted after revision November 15, 2016.
- © 2017 by American Journal of Neuroradiology
Jump to section
Related Articles
Cited By...
- Endovascular treatment in the multimodality management of brain arteriovenous malformations: report of the Society of NeuroInterventional Surgery Standards and Guidelines Committee
- 4D-DSA: Development and Current Neurovascular Applications
- Four-dimensional digital subtraction angiography for exploration of spinal cord vascular malformations: preliminary experience
- Quantitative and Qualitative Comparison of 4D-DSA with 3D-DSA Using Computational Fluid Dynamics Simulations in Cerebral Aneurysms