Performance and Predictive Value of a User-Independent Platform for CT Perfusion Analysis: Threshold-Derived Automated Systems Outperform Examiner-Driven Approaches in Outcome Prediction of Acute Ischemic Stroke

BACKGROUND AND PURPOSE: Treatment strategies in acute ischemic stroke aim to curtail ischemic progression. Emerging paradigms propose patient subselection using imaging biomarkers derived from CT, CTA, and CT perfusion. We evaluated the performance of a fully-automated computational tool, hypothesizing enhancements compared with qualitative approaches. The correlation between imaging variables and clinical outcomes in a cohort of patients with acute ischemic stroke is reported. MATERIALS AND METHODS: Sixty-two patients with acute ischemic stroke and MCA or ICA occlusion undergoing multidetector CT, CTA, and CTP were retrospectively evaluated. CTP was processed on a fully operator-independent platform (RApid processing of PerfusIon and Diffusion [RAPID]) computing automated core estimates based on relative cerebral blood flow and relative cerebral blood volume and hypoperfused tissue volumes at varying thresholds of time-to-maximum. Qualitative analysis was assigned by 2 independent reviewers for each variable, including CT-ASPECTS, CBV-ASPECTS, CBF-ASPECTS, CTA collateral score, and CTA clot burden score. Performance as predictors of favorable clinical outcome and final infarct volume was established for each variable. RESULTS: Both RAPID core estimates, CT-ASPECTS, CBV-ASPECTS, and clot burden score correlated with favorable clinical outcome (P < .05); CBF-ASPECTS and collateral score were not significantly associated with favorable outcome, while hypoperfusion estimates were variably associated, depending on the selected time-to-maximum thresholds. Receiver operating characteristic analysis demonstrated disparities among tested variables, with RAPID core and hypoperfusion estimates outperforming all qualitative approaches (area under the curve, relative CBV = 0.86, relative CBF = 0.81; P < .001). CONCLUSIONS: Qualitative approaches to acute ischemic stroke imaging are subject to limitations due to their subjective nature and lack of physiologic information. These findings support the benefits of high-speed automated analysis, outperforming conventional methodologies while limiting delays in clinical management.

P rimary goals in the management of acute ischemic stroke (AIS) include timely pharmacologic or mechanical intervention, while avoiding untoward risks related to complications of treatment. Appropriate patient selection may thus prove paramount, and identification of a subpopulation most likely to benefit has been the subject of extensive inquiry. 1,2 While the precise profile of this target population remains to be conclusively defined, recent work has highlighted the potential strengths of stratification by using the ischemic penumbra formalism defining an irreversibly injured infarct core and putative penumbra of at-risk tissues. 1,3 Despite promising results, penumbral imaging has met with skepticism and inconsistent outcomes, particularly because the broad array of imaging and computational approaches and interpretive parameters has precluded formulation of generalizable conclusions. [4][5][6][7] With studies further complicated by the time, materials, and expertise requisite to successfully undertake perfusion imaging, some investigators have focused on triage algorithms examining more readily attainable biomarkers derived from noncontrast CT (eg, Alberta Stroke Program Early CT Score) or CT angiography (eg, collateral score [CS], clot burden score [CBS]) common to stroke protocols. [8][9][10][11] While quickly attainable, the performance of ASPECTS in triaging patients to therapy or predicting outcome has been variable, and its use in prognostication of individual outcomes has been questioned. [12][13][14] Similarly, CS and CBS have shown promise as rapid approaches to assessment but may underperform when compared with perfusion imaging metrics. [9][10][11] Recently, the ASPECTS methodology was applied to CTP parametric maps in an effort to impart standardization and mitigate subjective elements of perfusion analysis; however, even in this context, the strengths of perfusion imaging may be attenuated by variability in postprocessing, computational analysis, selection of parametric maps, and the generally qualitative nature of such approaches. 10,11 Expert consensus has emphasized the demand for standardization in the acquisition, processing, and analysis of perfusion imaging. 15,16 The potential for disparate results and the variability in accuracy among competing software platforms have been the subject of recent studies and were thoroughly expounded in a comparative analysis by Kudo et al. 17 In light of recent reports, the primary objective of this study were to examine the predictive performance of several user-defined approaches to NCCT, CTA, and CTP analysis, by comparison with a fast, vendor-and operator-independent computational tool using fully automated lesion segmentation and pixel-wise parametric thresholding for semiquantitation (RApid processing of PerfusIon and Diffusion [RAPID]). 18 The objective of our study was to evaluate these tools to determine their ability to predict 90-day favorable clinical outcome in patients with AIS.

MATERIALS AND METHODS
We retrospectively identified patients with AIS presenting within 12 hours of symptom onset found to have intracranial ICA or MCA occlusion as part of a prospectively collected registry of patients undergoing acute comprehensive stroke imaging with NCCT, CTA, and CTP at Emory University Hospital from February 2011 to December 2013. All patients were initially evaluated by vascular neurology in the emergency setting, with initiation of an institutional stroke protocol facilitating expedited triage, imaging, interpretation, and treatment when appropriate. Patients were excluded from this analysis if they did not successfully complete the comprehensive imaging protocol, had motion or other artifacts rendering imaging nondiagnostic, or lacked data on 90day clinical outcome. Exclusion criteria further included patients in whom large symptomatic intracerebral hemorrhage developed, among whom measurement of final infarct volumes may be confounded or hindered by resultant mass effect. Retrospective analysis was supported under the guidelines of the institutional review board, and written informed consent was waived for this study.
Medical records were reviewed for a clinical history of atrial fibrillation, type 2 diabetes, hypertension, dyslipidemia, and congestive heart failure. Any pre-existing disability (modified Rankin Scale Ͼ0) was determined on initial questioning and during review of the electronic medical record. Initial neurologic screening at presentation included determination of National Institutes of Health Stroke Scale score recorded by a vascular neurologist with NIHSS certification. A subset of patients received intravenous and/or intra-arterial thrombolytic therapy per institutional protocol and at the discretion of the vascular neurologist. The modified Rankin Scale was established at discharge and 90 days. A favorable clinical outcome was defined as mRS Յ2 at 3 months.

Imaging Protocol
All patients underwent an institutional stroke imaging protocol to include NCCT, CTA, and CTP. CT was performed on a 40-mm, 64 -detector row clinical system (LightSpeed VCT; GE Healthcare, Milwaukee, Wisconsin). Helical NCCT (120 kV; 100 -350 auto-mA; CT dose index, ϳ43.15) was performed from the foramen magnum through the vertex at a 5.0-mm section thickness. In the absence of visible intracranial hemorrhage during real-time evaluation by a radiologist and vascular neurologist, 2 contiguous CTP slabs were obtained for 8-cm combined coverage of the supratentorial brain, obtained at eight 5-mm sections per slab. Cine mode acquisition (80 kV; 100 mA; CT dose index, ϳ293.48) permitting high-temporal-resolution (1-second sampling interval) dynamic bolus passage imaging was obtained following the administration of 35-mL iodinated contrast (iopamidol, Isovue 370; Bracco, Princeton, New Jersey) power injected at 5 mL/s through an 18-ga or larger antecubital IV access. Contrast administration was followed by a 25-mL saline flush at the same rate. Last, helical CTA (120 kV; 200 -350 auto-mA; CT dose index, ϳ38.08) was performed from the carina to the vertex (section thickness/interval, 0.625/0.375 mm) following IV administration of 70-mL iodinated contrast injected at 5 mL/s and followed by a 25-mL saline flush. Follow-up in all patients included either MR imaging and/or NCCT, documenting final infarct size before discharge. All images were transferred to a separate workstation for analysis (Mac Pro; Apple, Cupertino, California) by using a third-party DICOM viewer (Osirix 64-bit; http://www.osirix-viewer.com).

Image Analysis
NCCT-ASPECTS. All NCCTs were assigned an ASPECTS by 2 experienced vascular neurologists blinded to patient clinical information, other imaging, and clinical outcome. ASPECTS used a 10-point visual inspection scale assessing the early ischemic burden in the supratentorial brain as detailed previously by Barber et al. 8 CTA Collateral Score and Clot Burden Score. Tan et al 9 proposed a rapid, qualitative visual inspection methodology for brain CTA, emphasizing the importance of both the extent of vascular clot (CBS) and the adequacy of surface vascular enhancement, ostensibly reflecting collateral flow response (CS). Two experienced neuroradiologists with subspecialty certification applied visual inspection CTA scores, blinded to clinical and outcome data and all other imaging, specifically as follows: CBS defining clot extent for anterior circulation disease was assigned a score of 0 -10, whereby 2 points are deducted for thrombus within each of the supraclinoid ICAs, the proximal 50% of the horizontal (M1) MCA trunk, or the distal 50% of the M1; a single point was then deducted for thrombus within the infraclinoid ICA and the anterior cerebral artery and for each individual vertical (M2) MCA branch. 9 As per the original methodology proposed by Tan et al, the thrombus may be partial or complete, with a minimum potential score of zero reflecting extensive anterior circulation occlusion. CTA CS (Fig 1) was described as an ordinal, visual grading system for collateral flow, scored 0 -3; collateral flow was assigned a score of zero for absent surface collateralization, 1 for Ͼ0 but Ͻ50% collateral supply, 2 for 50 -99% collateral supply, and 3 for normal or supranormal surface vasculature of the MCA territory. CTA analysis was performed by using 20-mm axial maximum-intensity-projection and 0.625-mm axial source images, as well as orthogonal and curved multiplanar reformats as needed.
CT Perfusion. All perfusion imaging was postprocessed by using a vendor-independent software platform (RAPID). RAPID is an operator-independent image processing and visualization tool operating on standardized DICOM data and configured for entirely automated image processing. 18 As an extension of earlier iterations compiled for analysis of MRI diffusion and perfusion imaging, the current implementation further permits CTP postprocessing carried through a computational pipeline similar to that initially described. Briefly, following preprocessing steps correcting rigid-body motion, arterial input function selection is performed and deconvolved from the voxel time-attenuation course using a delay-insensitive algorithm for isolation of the tissue residue function. Time-to-maximum of the tissue residue function (Tmax) is determined on a voxelwise basis, with Tmax maps incrementally thresholded between 4 -10 sec-onds at 2-second intervals, and overlaid upon source CTP data for analysis.
Cerebral blood volume maps were computed as outlined elsewhere, and cerebral blood flow was determined in milliliter/100 g/minute. 18 Both CBV and CBF have been proposed as estimates of irreversibly infarcted (core) tissues, with past investigations describing both absolute and relative (rCBV, rCBF) thresholds for the detection of infarct core. 19 For the purposes of this study, separate estimates were supplied for each, thresholded at relative rCBF or rCBV Ͻ30% normal. Color-coded core maps for both parameters were automatically generated and overlaid on source images for review purposes (Fig 2). Processing time from the transfer of perfusion data to the RAPID suite to production of processed maps was approximately 3 minutes.
CTP analysis lastly included examiner-driven qualitative evaluation of perfusion parametric maps. Recent studies have extended ASPECTS methodology to perfusion imaging, reporting both improvements in predictive performance relative to clinical outcome and stronger interrater agreement. 10,20 As detailed by Aviv et al, 20 CBV and CBF maps were assigned CTP-ASPECTS by the same 2 neuroradiologists, again blinded to clinical and other radiographic data, in separate scoring sessions spaced approximately 7 days from the prior analyses. CTP-ASPECTS again ranged from 0 to 10, with a single point deducted for any of 10 supratentorial regions exhibiting apparent hypoperfusion.

Radiographic Outcome
The correlation between the RAPID core infarct estimates, CBV and CBF, and final infarct volume was determined independently. Final infarct volumes were measured following export of raw DICOM data to the Fiji release of the ImageJ software platform (http://imagej.nih.gov/ij/). Analysis included a series of postprocessing steps (Fig 3), beginning with optimization of image contrast for detection of hyperintense regions on follow-up trace DWI or CT images. Binary images were generated following application of intensity thresholds permitting voxelwise measures of final infarct volume in milliliters, cross-referenced with original DWI to exclude regions of EPI-related artifacts, such as those near the skull base or air-tissue interfaces.

Statistical Analysis
Statistical analysis was performed in SAS software, Version 9.3 (SAS Institute, Cary, North Carolina). Pearson correlation coefficients were calculated to assess the relationship between continuous variables (predicted infarct core and final infarct volume). Ordinal (ASPECTS, CS, CBS) and continuous (predicted infarct core volume, at-risk tissue volume, and final infarct volume)  9 Axial 20-mm maximum-intensityprojection images demonstrate the extent and asymmetry in the peripheral leptomeningeal collateral supply graded as relative to the contralateral normal hemisphere (pathologic MCA territory indicated by a red ROI). A, A score of zero suggests near-complete absence of surface collateralization. B, A score of 1 indicates greater than zero but Ͻ50% collateral flow. C, A score of 2 suggests Ͼ50% but Ͻ100% of normal leptomeningeal collaterals. D, A score of 3 suggests normal or, when present, greater than normal surface collaterals.
variables were considered for the input model of univariate and multivariate logistic regression, against dichotomized outcomes (mRS Յ2) for all patients at 90-day follow-up. Receiver operating characteristics (ROCs) were computed; optimal operating values were established for each variable of interest as discriminators of favorable clinical outcome; and sensitivity, specificity, and confidence intervals were determined. Statistical significance among tested variables as predictors of final infarct volume was determined at P Ͻ .05. Interobserver agreement for NCCT-ASPECTS, CBV-ASPECTS, CBF-ASPECTS, CBS, and CS was established by Pearson correlation coefficients. Intraobserver agreement for binary determination of ASPECTS Ͼ7 was established separately by statistics.

RESULTS
Sixty-two patients (36 women; median age, 70 years; range years, 33-94 years) with AIS (Ͻ12 hours) and MCA or intracranial ICA occlusion constituted the study population. Among these, 15 were excluded due to severe, unrecoverable motion artifacts (n ϭ 5) or lack of 90-day clinical outcome data (n ϭ 10), leaving 47 patients for analysis. No patients were observed to harbor bilateral arterial occlusions.
Median NIHSS at admission was 15 (interquartile range, 16); the mean duration from the time of onset/last known healthy to imaging was 210 minutes. Twenty-three (52%) patients received IV tPA, and 10 (23%) underwent endovascular treatment for intra-arterial clot lysis or retrieval. The mean mRS at both discharge and at 90 days was 3; favorable outcome (mRS Յ2) at 90 days was recorded in 16 patients (34%). An ICA or M1 occlusion was present in 41 (72%) patients, with MCA M2 segment occlusion observed in the remainder. Final infarct volume was measured by DWI in 77% of patients with the remainder established by NCCT; median final infarct volume was 34 mL (interquartile range, 94 mL).
Results of adjusted multivariable logistic regression analysis and odds ratios for dichotomized favorable outcome (mRS, Յ2) are presented in Table 1. Patients with favorable clinical outcome at 90 days exhibited higher baseline NCCT-ASPECTS (P ϭ .03). Similarly, significant interactions were observed relative to 90-day favorable clinical outcome for CBV-ASPECTS (P ϭ .01), CBF-ASPECTS (P ϭ .04), and CTA clot burden score (P ϭ .02), but not for CTA CS (P ϭ .09).
Among RAPID parameters interrogated, predicted core infarct determined both by CBV and CBF demonstrated significant correlations with favorable clinical outcome at 90 days (P ϭ .01). Estimates of hypoperfusion at thresholded values of Tmax  Optimization of image contrast for intensity-wise lesion segmentation was performed with manual thresholding toward production of a binary image, with cross-reference to the original DWI data to exclude spurious areas related to susceptibility or EPI distortions. exhibited variable results; statistical significance was observed for the default threshold Tmax of Ͼ6 seconds (P ϭ .03) as well as Tmax of Ͼ4 seconds (P ϭ .01), but not for the remaining Tmax thresholds.

DISCUSSION
The results herein support the feasibility of integrating a semiquantitative analytic and visualization tool for advanced CT triage in the time-sensitive domain of AIS. The findings highlight the potential benefits of advanced image processing in this context, whereby enhancements to typically qualitative and sometimes subjective interpretive algorithms were shown to augment the predictive value of acute stroke imaging as it relates to clinical outcome.
Among the variables tested, the semiquantitative predictors of irreversible ischemia, CBV core and CBF core, exhibited the strongest performance by ROC. By comparison with qualitative approaches to core estimation, relying on visual inspection and subjective definition of lesion extent, core estimates derived from the RAPID suite were produced following a series of normalization and quantitation steps, with automated lesion segmentation and determination of lesion volumes at predefined thresholds. Past reports and expert consensus have proposed the benefits of streamlined and reproducible approaches to postprocessing and analysis; however, rigorous postprocessing may be impractical in the clinical setting. 15,16 Accordingly, diagnostic paradigms in a NCCT-ASPECTS, CBV-ASPECTS, and CBF-ASPECTS represent ASPECTS applied respectively to noncontrast CT, cerebral blood volume maps, or cerebral blood flow maps. b Statistical significance in logistic regression observed with all tested variables with the exception of RAPID-derived Tmax of Ͼ8 seconds and Ͼ10 seconds and the CTA collateral score (P Ͼ .05). Note:-OOP indicates optimal operating point, optimal cut-off from ROC analysis for the variable of interest. a Results of ROC analysis relative to dichotomized favorable clinical outcome (90-day mRS Յ2). Performance indicated by ROC AUC in descending-order performance. b Significant relationships were observed for all tested variables, with the exception of the CTA collateral score.
multiparametric CTP commonly involve the subjective determination of relative mismatch between putative infarct core and the remaining at-risk tissues. With regions of irreversibly injured, at-risk, and modestly hypoperfused tissues potentially overlapping by visual inspection alone, voxelwise thresholding may provide some immunity to incorrect tissue classification. Heterogeneity in reperfusion of the study population precludes conclusive judgments as to a singular, ideal Tmax threshold for outcome prediction, for which past reports have documented variability in performance between a 4-and 6-second delay, depending on reperfusion status, in line with the best performing Tmax thresholds above. 6 NCCT-ASPECTS, while providing a rapid algorithm for determination of the extent of infarcted tissues, may have insensitivity to infarction in the early stages of injury, before the bulk water shift detection on noncontrast imaging. 10,12,14,20 We suspect that the clustering of abnormal brain regions in ASPECTS methodology, while providing for some uniformity in analysis, may have further limited the dynamic range for discriminating between lesions with similar ASPECTS but differing in actual extent or size. The inability of ASPECTS to provide an estimate of the extent or severity of tissues at ongoing ischemic risk may additionally limit its utility in patient selection. Application of ASPECTS methodology to CTP imaging aims to leverage some benefits of physiologic and flow imaging, while providing uniformity in analysis and reporting as in NCCT-ASPECTS. While showing improvements compared with NCCT-ASPECTS in prior reports, the analysis remains fundamentally subjective and again lacks in its ability to subclassify tissues within ASPECTS regions. 10,20 Two previously proposed CTA-derived measures, the clot burden score and collateral score, were investigated among triage variables in this study. 9,11,21 While previously having shown promise as outcome predictors, their ability to supplant tissuelevel information provided by perfusion imaging may be limited by uncertainty as to the nutritive capacity of surface-level collaterals vis-à-vis the tissue perfusion approximated by dynamic bolus passage deconvolution with an arterial input function. The importance of leptomeningeal collateral flow in preserving ischemic neuronal substrate and potentially protecting against hem-orrhage has been discussed, and numerous multimodal collateral scoring methodologies have been proposed but may lack in their ability to characterize the dynamic nature of collateral flow or discriminate nonperfusion from flow delayed at the moment of acquisition. 22 Such vulnerabilities may, in combination, have contributed to the lack of statistical significance observed with CS relative to dichotomized favorable clinical outcome.
Stroke imaging protocols commonly include some combination of CT or MR imaging to exclude the presence of acute hemorrhage and for characterization of infarct core, as well as CT or MR imaging hemodynamic-sensitized techniques to identify the presence/extent of tissue at ongoing ischemic risk. 4,23,24 Several studies have aimed to apply such methodology to imaging triage; however, no consensus has emerged regarding the optimal combination of qualitative and quantitative metrics to accurately identify the ischemic penumbra. 15,16 Image-based patient selection has shown success in some recent multicenter trials favoring interrogation with perfusion parameters. 1,2 Variability among vendor-based and locally developed perfusion analysis tools is known to produce considerable differences in results from perfusion data and may engender variable conclusions from common datasets. 17 The relative performance of several tools was compared in a digital phantom in a recent work by Kudo et al 17 ; with this in mind, a study of predictive performance against commonly used qualitative approaches was conducted, suggesting enhancement to performance with the use of a standardized, user-independent platform. The disparate outcomes following re-analysis of data from the Echoplanar Imaging Thrombolytic Evaluation Trial support the idea that such differences in analysis may not be purely trivial. 25 We acknowledge several study limitations. The relatively small sample population may have impacted the ability to identify significant relationships among some tested variables; however, significant trends among most tested parameters in our study are consistent with the generally strong performance of such approaches in past reports. Within the statistical limits of the study, we believe the findings offer generalizable support for the strengths of advanced, automated computational tools for clinical use. We acknowledge that ROC results in this study may suggest generally stricter and more conservative thresholds for favorable outcome compared with past studies. We suspect that the small sample size and heterogeneity in treatment and revascularization status account substantially for such differences; however, because the primary study aim was establishing the feasibility, utility, and superiority of an automated platform by comparison with user-driven approaches, we do not suspect that these observations have significantly affected the study conclusions. The aims in this study included principally the characterization of varying approaches to predicting clinical outcome. The design and statistical limits did not permit assessment of interactions between study variables and revascularization or tPA administration among patients and, in this respect, cannot specifically be assumed as generalizable to the merits of treatment in such patients. We, however, maintain that within the study aims, specifically to highlight the advantages of robust automation over qualitative analysis regimens in AIS imaging triage, the interaction term relating reperfusion to outcomes is not likely to have influenced the reported a Correlational analysis between tested variables and final infarct volume (MRI or CT, in milliliters). The strongest overall correlations were observed with RAPID-derived core estimates (rCBF core and rCBV core). Negative correlations were observed between all ASPECTS and CTA-derived measures relative to the final infarct volume. b Significant relationships were observed for all tested variables with the exception of the CTA clot burden score.
results. A larger scale study permitting dichotomization between treated and untreated patients is necessary and is underway for the assessment of such diagnostic approaches as predictors of a beneficial therapeutic effect in AIS.

CONCLUSIONS
Automated approaches to imaging in AIS may offer some immunity to limitations inherent in qualitative methodologies. These findings support the benefits of high-speed automated analysis in AIS, outperforming conventional methodologies while avoiding delays in management.