Abstract
BACKGROUND AND PURPOSE: In the clinical setting, there is a need to perform mismatch measurements quickly and easily on the MR imaging scanner to determine the specific amount of treatable penumbra. The objective of this study was to quantify the agreement of the ABC/2 method with the established planimetric method.
MATERIALS AND METHODS: Patients (n = 193) were selected from the NINDS Natural History Stroke Registry if they 1) were treated with standard intravenous rtPA, 2) had a pretreatment MR imaging with evaluable DWI and PWI, and 3) had an acute ischemic stroke lesion. A rater placed the linear diameters to measure the largest DWI and MTT lesion areas in 3 perpendicular axes—A, B, and C—and then used the ABC/2 formula to calculate lesion volumes. A separate rater measured the planimetric volumes. Multiple mismatch thresholds were used, including MTT volume − DWI volume ≥50 mL versus ≥60 mL and (MTT volume − DWI volume)/MTT volume ≥20% versus MTT/DWI = 1.8.
RESULTS: Compared with the planimetric method, the ABC/2 method had high sensitivity (0.91), specificity (0.90), accuracy (0.91), PPV (0.90), and NPV (0.91) to quantify mismatch by use of the ≥50 mL definition. The Spearman correlation coefficients were 0.846 and 0.876, respectively, for the DWI and MTT measurements. The inter-rater Bland-Altman plots demonstrated 95%, 95%, and 97% agreement for the DWI, MTT, and mismatch measurements.
CONCLUSIONS: The ABC/2 method is highly reliable and accurate for quantifying the specific amount of MR imaging–determined mismatch and therefore is a potential tool to quickly calculate a treatable mismatch pattern.
ABBREVIATIONS:
- CI
- confidence interval
- IQR
- interquartile range
- NPV
- negative predictive value
- PPV
- positive predictive value
- SD
- standard deviation
In the clinical setting, there is a need to perform quantitative mismatch measurements quickly and easily to determine the specific amount of treatable penumbra. The ABC/2 method of measuring quantitative mismatch is a viable option because it can be performed immediately on the MR imaging scanner and has precedence in other diseases. Prior studies by use of the ABC/2 method to measure intracerebral hemorrhage or subdural hematoma volumes have been extensively applied and validated.1⇓–3 Kothari et al1 demonstrated that the ABC/2 method had excellent correlation with the planimetric method when applied to the measurements of intracerebral hemorrhage volumes. Furthermore, they demonstrated excellent inter-rater and intrarater reliability for the ABC/2 method.1 Gebel et al2 adapted the ABC/2 method successively to measure subdural hematoma volumes and demonstrated excellent correlation with a computerized technique. Huttner et al3 applied the ABC/2 method to more complicated intracerebral hemorrhage patterns and found that modification of the formula to ABC/3 produced more accurate measurements. In acute ischemic stroke, automated mismatch measurements by use of postprocessing software are advantageous4 for quantifying mismatch but are not generally available. Quality or format limitations of some scans may prohibit automated software from producing usable mismatch results. The feasibility of qualitative evaluation of mismatch on MR imaging before thrombolysis has been presented.5,6 However, in recent stroke clinical trials, specific imaging thresholds for mismatch beyond visual confirmation were required to make enrollment decisions.7⇓⇓⇓⇓–12 Optimization of thresholds including mismatch of >20%10,11 and PWI/DWI >1.8,12 have been investigated retrospectively by many investigators to apply these conditions prospectively for automated measurement.10⇓–12 Validation of the ABC/2 mismatch method may prove to be less challenging than validating the numeric methods for automated mismatch measurements. Furthermore, the ABC/2 method is a possible alternative to automated methods when image quality restricts their usage. However, the agreement of the ABC/2 method with the planimetric method has not been fully defined. Sims et al13 established that the ABC/2 method provided the best estimation of infarction and MTT volumes. The prior study demonstrated a high PPV of 92%; however, the results had a poor NPV of 33%.13 Pedraza et al14 recommended that a larger study looking at broader range of mismatch volumes and clinical outcomes ultimately should be performed. Vogt et al15 used the ABC/2 method mainly in CT scans in >1800 patients with ischemic stroke and hemorrhage but demonstrated in a subset of MR imaging scans that their infarct volume results were stable regardless of imaging technique. Warach et al16 demonstrated a difference in favorable clinical outcome in desmoteplase-treated patients versus placebo-treated patients in a post hoc analysis of the Desmoteplase in Acute Ischemic Stroke Trial-2 when applying a specific mismatch threshold of >60 mL. However, no study has compared these various mismatch definitions across both the ABC/2 and planimetric methods to quantify their agreement and feasibility for application in the clinical trial setting. Therefore, the primary focus of this study was to compare the agreement across definitions when quantifying the amount of mismatch by use of both the ABC/2 and planimetric methods.
The objectives of this study were to compare the ABC/2 and planimetric methods to determine the agreement between the mismatch volumes and resulting classifications. Our hypotheses were 1) the ABC/2 measurements were equivalent to the planimetric measurements and therefore the ABC/2 method was an accurate tool for quick quantification of treatable mismatch patterns, and 2) the ABC/2 mismatch classifications predicted the same clinical outcomes as the planimetric mismatch classifications.
Materials and Methods
Patients
This is an analysis of data from the National Institute of Neurological Disorders and Stroke Natural History registry. The NINDS Natural History registry is a dataset formed from 2 acute stroke centers: Suburban Hospital in Bethesda, Maryland, and Medstar Washington Hospital Center in Washington, DC. The appropriate ethics and institutional review boards approved the study. For this study, all patients were treated with standard intravenous rtPA within 3 hours of time last known well between December 2000 and October 2009. Patients were included if they 1) were treated with standard intravenous rtPA, 2) had a pretreatment MR imaging with evaluable DWI and PWI, and 3) had an acute ischemic stroke lesion.
Imaging Series
Imaging was performed by use of 1.5T (TwinSpeed; GE Healthcare, Milwaukee, Wisconsin) or 3T (Achieva; Philips, Best, the Netherlands) clinical MR imaging scanners. DWI and PWI series were acquired co-localized over the entire brain with a superior to inferior coverage of 14 cm. Typical imaging parameters for DWI spin-echo echo-planar series included either 40–3.5-mm- or 20–7-mm-thick contiguous axial oblique sections with b = 0 and b = 1000 seconds/mm2, trace or isotropically weighted, TR/TE = 6000–7000/72–90 ms, acquisition matrix of 64 × 64–128 × 128, and 22 cm FOV. The PWI was a dynamic susceptibility contrast series with the use of a single dose of 0.1 mmol/kg of gadolinium (Magnevist; Bayer Schering Pharma, Wayne, New Jersey). Typical imaging parameters for PWI gradient echo-planar series included 20 contiguous axial oblique sections with single-dose gadolinium contrast injection of 0.1 mmol/kg through a power injector by using 25–40 phase measurements, TR/TE = 2000–2200/45 ms, acquisition matrix of 64 × 64–128 × 128, 7-mm section thickness, and 22-cm FOV.
Image Analysis
ABC/2 Volume Method for Quantitative Mismatch.
A rater (J.H.) measured the ABC/2 volume of DWI and MTT lesions by using a DICOM image viewer. After optimizing the window level settings, the 2 longest perpendicular linear diameters (A and B) on the section where the abnormality on DWI appeared largest were measured (Fig 1A). The same process was repeated on the section with the largest area of MTT abnormality (Fig 1B). With the use of these diameters, the product of the section thickness, and the total number of sections containing the lesion (C), the ABC/2 DWI and MTT volumes were calculated with the formula: volume = ABC/2. For discontinuous lesions, only the largest lesion area was measured. Multiple measurements were not performed across the discontinuous lesions. However, if the lesion was continuous and included multiple vascular territories, the measurement included the entire lesion area. For punctate lesions, the largest lesion area was measured.
Corresponding measurements for the ABC/2 method (A, DWI = 116.1 mL; B, MTT = 194.7 mL) and planimetric method (C, DWI = 116.2 mL; D, MTT = 248.4 mL) on paired sections placed independently by 2 different readers.
Planimetric Volume Method for Quantitative Mismatch.
A rater (M.L.) with extensive experience and established rater reliability statistics measured the lesion volumes on the DWI and MTT maps by using a semi-automated quantitative, planimetric method in Cheshire (Boulder, Colorado).17 The intrarater and inter-rater reliability of the planimetric measurements of DWI and MTT was validated as a highly consistent and repeatable method by use of Cheshire in the Luby et al study.17 Lesion areas were segmented on a section-by-section basis, with user-selected seed points followed by user-driven editing (Fig 1C, -D).
Thresholds for Quantitative Mismatch.
Patients were classified as having a quantitative mismatch when the difference in the volumes on the MTT and the DWI was ≥50 mL2 versus ≥60 mL16 and (MTT volume − DWI volume)/MTT volume ≥20%10,11 versus MTT/DWI ≥1.8.12
Inter-Rater Reliability of ABC/2 Volume Method.
For the purposes of determining inter-rater reliability of the ABC/2 method, the rater (M.L.) who performed the planimetric volume measurements independently measured the ABC/2 lesion volumes on DWI and MTT, blinded to the quantitative results already generated for the study.
Statistical Analysis
The following analyses were performed: 1) lesion volumes for the ABC/2 and planimetric methods and associated mismatch classifications, based on the MTT−DWI ≥50 mL versus ≥60 mL and (MTT volume − DWI volume)/MTT volume ≥20% versus MTT/DWI ≥1.8 definitions, 2) agreement of mismatch classifications as determined by the ABC/2 and planimetric methods, 3) inter-rater reliability measures for the DWI and MTT and measurements by use of the ABC/2 method, and 4) functional outcome rates by use of modified Rankin Scale scores among individuals across the ABC/2 and planimetric methods. SPSS Statistics (v17.0; SPSS, Chicago, Illinois) was used for all statistical analyses.
For the lesion volume statistics, only cases with positive lesions, that is, >0 mL volumes, were included. Values are reported as mean (±SD) or median (IQR, 25–75) when appropriate. Agreement rates were defined as the number of cases agreed divided by the total number of cases (n = 193) except as noted. Linear regression of volumes was performed to demonstrate the correlation between the ABC/2 and planimetric measurements. The Bland-Altman plots were generated to display the spread of the lesion volumes and the limits of agreement between the ABC/2 and planimetric measurements, specifically to illustrate how many of the measurements were within 2 SD from the mean volume difference. Inter-rater reliability of the lesion volumes was quantified by Spearman correlation coefficients and Bland-Altman plots. The Bland-Altman plots were generated to display the spread of the lesion volumes and the limits of agreement between the 2 independent sets of ABC/2 measurements. Cohen κ coefficients were calculated. The Bland-Altman and linear regression plots were on the logarithmic scale. Contingency (2 × 2) tables were used to calculate specificity, sensitivity, accuracy, PPV, and NPV of the ABC/2 method compared with the planimetric method.
Results
Patients
From December 2000 through October 31, 2009, 385 patients were treated with standard intravenous rtPA. Of these, 234 patients had a pretreatment MR imaging. Forty-one patients were excluded because they did not include PWI (n = 18) or the DWI and MTT maps were not available or not evaluable (n = 23). The final sample in this study includes 193 patients. In 143 patients, the MR imaging was performed at 1.5T and in 50 at 3T. Fifty-two percent of the patients were women (n = 102). The mean age of the patients was 70.8 (±15.7) years, and the median time from stroke onset to baseline MR imaging was 103 (IQR, 79–128) minutes. The median baseline NIHSS score in the 192 patients for whom these data are available was 9 (IQR, 4–18).
ABC/2 Volumes
Using the ABC/2 method, DWI (n = 184), MTT (n = 170), and mismatch (MTT-DWI, n = 150) median volumes were 18.9 mL (IQR, 3.1–60.0), 116.3 mL (IQR, 33.0–249.5), and 78.9 mL (IQR, 26.8–183.6), respectively. Overall, 48.7% of patients of the entire sample (n = 94/193) were classified as having a mismatch by the ABC/2 method by use of the MTT−DWI ≥50 mL definition. Table 1 contains the volume statistics for the 94 patients with mismatch ≥50 mL versus the 56 patients without measurable mismatch. Forty-four percent of patients (n = 85/193) were classified as having a mismatch by use of the MTT−DWI ≥60 mL definition. Seventy-four percent of patients (n = 142/193) were classified as having a mismatch by use of the MTT−DWI ≥20% definition versus 60% (n = 116/193) by use of the MTT/DWI ≥1.8 definition.
Comparison of volume statistics between patients classified as mismatch versus no mismatch by use of the 50-mL mismatch definition for the ABC/2 and planimetric methods
Planimetric Volumes
With the use of the planimetric volume method, DWI (n = 186), MTT (n = 167), and mismatch (MTT-DWI, n = 154) median volumes were 12.4 mL (IQR, 2.6–51.8), 103.9 mL (IQR, 30.7–208.6), and 74.2 mL (IQR, 21.8–165.9), respectively. Overall, 48% of patients (n = 92/193) were classified as having a mismatch by the planimetric volume method by use of the MTT−DWI ≥50 mL definition. Table 1 contains the volume statistics for the 92 patients with mismatch ≥50 mL versus the 62 patients without measurable mismatch. Forty-five percent of patients (n = 87/193) were classified as having a mismatch by use of the MTT−DWI ≥60 mL definition. Seventy-four percent of patients (n = 142/193) were classified as having a mismatch by use of the MTT−DWI ≥20% definition versus 60% (n = 116/193) by use of the MTT/DWI ≥1.8 definition.
Comparison of ABC/2 and Planimetric Measurements
The linear regression plots (Fig 2) of ABC/2 with planimetric measurements are displayed for DWI (Fig 2A), MTT (Fig 2B), and mismatch (Fig 2C). The plots are on the logarithmic scale. The R2, slope and confidence intervals for DWI, MTT, and mismatch are 0.752, 0.867 (CI, 0.83–0.99), 0.822, 0.906 (CI, 0.82–0.95), and 0.69, 0.83 (CI, 0.65–0.83), respectively. The Bland-Altman plots (Fig 3) of the ABC/2 versus planimetric measurements demonstrated that 93.4%, 94.9%, and 93.3% for the DWI, MTT, and mismatch measurements were within the thresholds defined by 2 SD from the mean differences. The Spearman correlation coefficients were 0.84 and 0.87 for the DWI and MTT measurements, respectively (P < .01), between the ABC/2 and planimetric methods. The sensitivity (0.91), specificity (0.90), accuracy (0.91), PPV (0.90), and NPV (0.91) were excellent for the ABC/2 method by use of the planimetric mismatch definition ≥50 mL. Volume statistics for the discrepant patients (n = 17) are included in Table 2. On the basis of the ABC/2 measurements of cases FN 6, FN 7, and FN 8, treatment decisions requiring a mismatch may have changed. However, there is a positive mismatch for the remaining 14 discrepant patients across both methods; therefore, changes in treatment decisions for thrombolysis are not likely.
Linear regression plots of ABC/2 with planimetric measurements displaying the respective regression lines with R2 and slope values for DWI (A), MTT (B), and mismatch (C). Plots are on the logarithmic scale.
Bland-Altman plots of the ABC/2 and planimetric volume measurements demonstrate the difference between the log volume of the planimetric measurement and the log volume of the ABC/2 measurement over the mean for DWI (A), MTT (B), and mismatch (C). Threshold lines above and below plots represent values that are 2 SD from the mean difference. Plots are on the logarithmic scale.
Volume statistics for discrepant patients (n = 17, nine false-positives and eight false-negatives) on the basis of the ABC/2 method compared with the planimetric method by use of the mismatch definition ≥50 mL
Correlation Between Mismatch and Functional Outcome
Patients who were classified with a positive mismatch by visual or ABC/2 methods were more likely to have a good outcome as defined by modified Rankin Scale score = 0 or 1 (Table 3). The median last follow-up modified Rankin Scale score was 3 (IQR, 1–5), on the basis of the available data (n = 177), with a total of 66 patients (37.3%) with a good outcome. Among patients classified with a mismatch ≥50 mL by either the ABC/2 or planimetric method, 27–28% had a favorable outcome (Table 3). Considering only patients with age ≤80 years, the favorable outcomes were still consistent and, as expected, higher (38–39%). The ABC/2 and planimetric methods demonstrated nearly identical favorable outcome rates on the basis of the presence of positive mismatch defined as ≥50 mL.
Summary of functional outcome (modified Rankin Scale) at follow-up of all patients on the basis of mismatch definition of ≥50 mL by use of the ABC/2 and planimetric methods
Inter-Rater Reliability of ABC/2 Volume Method
There was strong inter-rater correlation and agreement between the 2 independent sets of ABC/2 measurements. The Spearman correlation coefficients were 0.89, 0.91, and 0.82 for the DWI (n = 193), MTT (n = 180), and mismatch (n = 125) measurements, respectively (P < .01). The Bland-Altman plots (Fig 4) of the 2 independent ABC/2 measurements demonstrated that 95%, 95%, and 97% for the DWI, MTT, and mismatch measurements were within the thresholds defined by 2 SD from the mean differences.
Bland-Altman plots of the 2 independent ABC/2 volume measurements demonstrate the difference between the log volume of the measurements over the average for DWI (A), MTT (B), and mismatch (C). Threshold lines above and below plots represent values that are 2 SD from the mean difference. Plots are on the logarithmic scale.
Discussion
Our study demonstrates that mismatch volume calculated by the ABC/2 quantitative method is equivalent to the planimetric method. This expands on our prior study that established the equivalence between the visual and planimetric methods.5 However, as used in prior clinical trials, it is likely that a specific amount of penumbra must be calculated beyond visual confirmation of mismatch only. The equivalence between the ABC/2 and planimetric methods was supported by 3 main results. The Bland-Altman analysis demonstrated that 93% of the 193 patients in this study had mismatch volumes consistent between the ABC/2 and planimetric methods. The Spearman correlation coefficients of the DWI and MTT measurements were both high between the ABC/2 and planimetric methods. The sensitivity (0.91), specificity (0.90), accuracy (0.91), PPV (0.90), and NPV (0.91) were excellent for the ABC/2 method by use of the planimetric mismatch definition ≥50 mL. The ABC/2 method had both high PPV and NPV for the measurement of mismatch compared with the planimetric method, both of which have not been previously demonstrated in similar studies by Sims et al13 and Pedraza et al.14 This is also the largest ischemic stroke study looking at MR imaging–determined mismatch by both ABC/2 and planimetric methods. This study provides a reference for selection of mismatch thresholds and the comparability across these methods for future stroke trial design.
The ABC/2 and planimetric measurements of DWI and MTT lesion volumes used in this study are highly dependent on the “eyeball” image interpretation by the raters. The “eyeball” approach has been used and generally agreed to approximate the 20% mismatch threshold.10 In most stroke centers, this visual confirmation of mismatch is the most commonly applied method. However, the mismatch definition ≥50 mL was the focus of this study, based on the prior study5 that demonstrated equivalence between the visual method, that is, the 20% eyeball method, and the planimetric method by use of this threshold. The intrarater and inter-rater reliability of these methods have been documented as highly reliable.5,6,13 ABC/2 measurements can be performed on any scanner, and, as shown in this study and others, are a reliable and accurate method to assess brain volumes.1⇓–3,5,13 The ABC/2 has the advantage that it can be performed in real time by clinicians caring for patients with acute ischemic stroke. The average computation time of automated algorithms varies across centers; not all centers have access to automated algorithms, and not all centers acquire the images or format suitable for processing by these algorithms. If future clinical trials have inclusion criteria that are based on specific mismatch volumes, a valid, reliable, fast, and accessible method beyond visual confirmation will be required. Gómez-Mariño et al18 recommended that the ABC/2 method be applied routinely in acute stroke because it is a fast and low-cost method. We propose that the ABC/2 method is an alternative method when automated mismatch measurements are not available.
Our study has several limitations. We attempted to replicate the methods readily available on a MR imaging scanner; however, the measurements were not actually performed in an acute clinical setting. Image interpretation differences between the raters were a source for error, independent of the methods. Some of these differences probably were due to suboptimal diffusion and perfusion acquisitions compounded by the differences in the experience of the 2 raters in this study. As shown in Table 2, discrepancy case FN6 was identified as a negative perfusion case by the ABC/2 rater but was measured as a significant perfusion deficit by the planimetric rater. One specific limitation of the ABC/2 method was how discontinuous lesions were evaluated; only the largest lesion was measured rather than combining measurements across the multiple lesions. This probably contributed to some of the discrepancies seen between the ABC/2 and planimetric methods listed in Table 2.
We found that mismatch is common among thrombolytic-treated patients, whether visually confirmed (35%) or defined by mismatch ≥50 mL by use of the ABC/2 method (31%) and is associated with a favorable outcome. We conclude that the ABC/2 method is accurate for classifying the presence of MR imaging–determined quantitative mismatch in patients with acute stroke and therefore a potential tool to quickly determine a treatable mismatch pattern. One possible future study is to apply the ABC/2 method in the acute clinical setting to demonstrate the feasibility of the use of this method when making clinical trial inclusion decisions. Ideally, this study would be performed in conjunction with the application of an automated mismatch method to determine the agreement between these methods.
ACKNOWLEDGMENTS
The authors would like to acknowledge and thank the NIH Stroke Team and the patients for their valuable participation and cooperation.
Footnotes
This research was supported by the Division of Intramural Research of the National Institutes of Health and the National Institute of Neurological Disorders and Stroke.
Indicates open access to non-subscribers at www.ajnr.org
REFERENCES
- Received July 27, 2012.
- Accepted after revision December 3, 2012.
- © 2013 by American Journal of Neuroradiology
Jump to section
Related Articles
Cited By...
- Use of the ABC/2 Method to Select Patients for Thrombectomy After 6 Hours of Symptom Onset
- Adjunctive Efficacy of Intra-Arterial Conebeam CT Angiography Relative to DSA in the Diagnosis and Surgical Planning of Micro-Arteriovenous Malformations
- Predictors of Dysphagia in Acute Pontine Infarction
- ASPECTS (Alberta Stroke Program Early CT Score) Assessment of the Perfusion-Diffusion Mismatch