Abstract
BACKGROUND AND PURPOSE: Although various revascularization scales are used in the angiographic evaluation of acute ischemic stroke, observer reliability tests of these scales have been rarely performed for posterior circulation stroke. We aimed to evaluate inter- and intraobserver variability of 2 scales, the modified Treatment in Cerebral Ischemia and the Arterial Occlusive Lesion, in posterior circulation stroke.
MATERIALS AND METHODS: Three independent readers interpreted pre- and postthrombolytic angiographies of 62 patients with posterior circulation stroke by using the modified Treatment in Cerebral Ischemia and Arterial Occlusive Lesion scales. The κ statistic was used to measure observer agreement for both scales, and κ > 0.6 was considered substantial agreement.
RESULTS: For the Arterial Occlusive Lesion scale, inter- and intraobserver agreement was >0.6. While intraobserver agreement of the modified Treatment in Cerebral Ischemia scale was >0.6 except for 1 reader, interobserver agreement was lower in dichotomized and original scales. In 49 cases with solely basilar artery occlusion, inter- and intraobserver agreement of both scales was similar to that in all 62 patients with posterior circulation stroke. In 2 consecutive readings, there was a significant decrease in the proportion of mTICI 2a reads (22.58% in the first versus 13.44% in the second session, P < .03) and a reciprocal increase in the sum of proportions for modified Treatment in Cerebral Ischemia 2b and modified Treatment in Cerebral Ischemia 3 reads (62.37% in the first versus 72.58% in the second session, P < .046).
CONCLUSIONS: In angiographic assessment of posterior circulation stroke, inter- and intraobserver agreement for the Arterial Occlusive Lesion scale was reliable, while the modified Treatment in Cerebral Ischemia failed to achieve substantial interobserver agreement. The clinical impact of this result needs to be validated in future studies.
ABBREVIATIONS:
- AOL
- Arterial Occlusive Lesion
- BAO
- basilar artery occlusion
- mTICI
- modified Treatment in Cerebral Ischemia
- TICI
- Thrombolysis in Cerebral Infarction
- TIMI
- Thrombolysis in Myocardial Infarction
In the treatment of acute ischemic stroke with large intracranial arterial occlusion, endovascular techniques are becoming the mainstream with higher revascularization rates.1⇓–3 While various grading schemes, such as the Thrombolysis in Myocardial Infarction (TIMI) or Thrombolysis in Cerebral Infarction (TICI), are widespread, their reliability in angiographic assessment of anterior circulation stroke remains controversial.4⇓⇓⇓–8 Recently, the modified Treatment in Cerebral Ischemia (mTICI) and the Arterial Occlusive Lesion (AOL) scales were strongly recommended as standards of reperfusion and recanalization in the angiographic evaluation of anterior circulation stroke.9
Despite applying similar scales to the posterior circulation, it is also unclear which scales might be reliably implemented for the vertebrobasilar territory. Recently, Gerber et al10 questioned whether it is right to use the TIMI or TICI scale in posterior circulation stroke and demonstrated that interobserver variability tests of these scales had never been performed in case of posterior circulation stroke.
Therefore, we aimed to evaluate intra- and interobserver agreement of the mTICI and AOL scales in the angiographic evaluation of posterior circulation stroke.
Materials and Methods
Patients
After approval of the institutional review board for this retrospective study, 62 patients (men/women = 37:25, mean age = 68 ± 11.4 years) with acute posterior circulation stroke, who underwent intra-arterial thrombolysis from April 2004 to December 2013, were consecutively enrolled from a single institutional data base. All patients underwent digital subtraction angiography before and after the procedure.
In these patients, the mean NIHSS score at admission was 17 ± 8, and the levels of arterial occlusion in the basilar (n = 49) and vertebral arteries (n = 13) were 79% and 21%, respectively. At 3 months, 21 patients (33.9%) had a good outcome (modified Rankin Scale score 0–2) and 12 (19%) had died.
Image Acquisition
DSA images from angiograms of the bilateral vertebral arteries were acquired by using a biplane angiography system. Consecutive anteroposterior and lateral angiographic images before and after intra-arterial thrombolysis, from the arterial to the delayed venous phase, were obtained in JPEG format, and were converted to a movie file format (Adobe Flash authoring file). Each reader interpreted them via on-line storage by making a comparison between preoperative and postoperative DSA images.
Image Interpretation
Three experienced readers (S.H.S., C.J., and W.Y.), who worked in 3 different tertiary medical centers as interventional neuroradiologists (with >10 years of experience), independently reviewed all images of 62 cases twice, 3–4 months apart. There was no training or required consensus of readers to perform this task.
Regardless of the others' reads, each reader assessed his own read by using the mTICI and AOL scales. In Table 1, the mTICI scale is defined in 5 grades according to the Stroke Treatment Academic Industry Roundtable consensus9 and the AOL scale is classified into 4 grades.11 For this study, we did not provide any special information to the readers.
The modified Treatment in Cerebral Ischemia and the Arterial Occlusive Lesion scale scores
Statistical Analysis
As parameters of intraobserver and interobserver agreement, the κ statistic was used for the mTICI and AOL scales. The κ value was interpreted according to Landis and Koch12 with a κ value of 0 = poor, 0.01–0.20 = slight, 0.21–0.40 = fair, 0.41–0.60 = moderate, 0.61–0.80 substantial, and 0.81–1.0 = almost-perfect agreement. The κ statistic was also calculated for 49 cases with basilar artery occlusion (BAO) and the dichotomized groups, including those with poor revascularization (AOL = 0–2, mTICI = 0–2a) versus good revascularization (AOL = 3, mTICI = 2b–3). Comparison of the κ values was performed by using 95% CIs for the difference between the κ statistics, with 1000 bootstrapped samples. The difference was statistically significant if 95% CIs did not include zero. Statistical analyses were performed by using SAS (Version 9.2; SAS Institute, Cary, North Carolina).
Results
Contrary to the AOL scale, all pair-wise κ values for the mTICI were lower than 0.6 in interobserver agreement (Table 2). Intraobserver agreement for the AOL and mTICI scales was substantial to almost perfect in most cases, except for 1 reader for the mTICI scale (reader A, κ = 0.444 in all and 0.462 in BAO, respectively; Table 3). Regardless of the scale used, intraobserver and interobserver agreement for 49 cases with BAO was not significantly different from that for all 62 patients. For the dichotomized scales, there was an increasing trend of κ values in the AOL, but not in the mTICI scale.
Interobserver agreement of 62 cases and subset of 49 cases with basilar artery occlusion by using original versus dichotomized outcomes
Intraobserver agreement between 62 cases and subset of 49 cases with basilar artery occlusion
Table 4 shows the distribution of all reads by using either the mTICI or the AOL scale. In 2 consecutive readings, there was a significant decrease in the proportion of mTICI 2a (42/186, 22.58%, in the first session versus 25/186, 13.44%, in the second session; P < .03) and a reciprocal increment in the sum of proportions for mTICI 2b and mTICI 3 (116/186, 62.37%, in the first session versus 135/186, 72.58%, in the second session; P < .046). In the proportion of unanimity, the mTICI was also significantly lower than the AOL (14/62, 23%, in mTICI, versus 44/62, 71%, in the AOL; P < .001).
Proportions in all ratings of 3 readers by categories of the scales
Discussion
In the 17 recent studies regarding intra-arterial thrombolysis in acute posterior circulation stroke,13⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓–29 5 (29.4%) did not provide any scheme and 12 (70.6%) used the TIMI or TICI scale, of which 58% (7/12) used TIMI ≥ 2 and 42% (5/12), TICI ≥2b as a cutoff level of “successful reperfusion.” In 10 studies with only BAO cases,14,17,20⇓⇓–23,25,27⇓–29 the TIMI scale was used in 5 and the TICI scale in 4. However, observer reliability tests in those previous studies had never been performed for angiographic assessment of posterior circulation stroke. If such heterogeneous and inconsistent scoring systems are used in defining end points of revascularization success, it is difficult to compare or combine results of clinical studies.30 Therefore, selection of a reliable biomarker for revascularization is relevant to the prediction of procedural efficacy and the outcome in posterior circulation stroke. To our knowledge, this is the first study to evaluate inter- and intraobserver agreement for 2 common scales in the angiographic evaluation following intra-arterial thrombolysis of posterior circulation stroke.
In this study, we found the observer variability of the mTICI inherent in the angiographic assessment of posterior circulation stroke. While inter- and intraobserver variability for the AOL was reliable, the mTICI failed to achieve substantial interobserver agreement with a low concordance rate of 23% (14/62 cases). We also demonstrated a significant difference in the proportion of mTICI 2a reads that had a direct influence on the sum of proportions of mTICI 2b and 3, which may affect the judgment of angiographic end points in intra-arterial thrombolysis. The mTICI was inferred inferior to the AOL for the following reason1: the relative complexity of the mTICI with more responses and semiquantitative descriptors. Gaha et al4 explained that inter- and intraobserver disagreement in adjudicating treatment results may be caused by multiple problems: intrinsic ambiguities in the definitions of the classifications; discrepancies in the various ways the definitions are interpreted by various readers; and even if the definitions were understood in the same way, discrepancies in applying the definitions to individual cases. Kundel and Polansky31 also showed that the κ value in observer agreement was likely to increase as the number of categories decreased.2 It is possible to have inconsistency in interpreting the mTICI 2 grade, such as mTICI 2a versus mTICI 2b, especially for posterior circulation. In fact, angiographic evaluation of the vertebrobasilar territory has some limitations, such as interference with abundant collateral flows, incomplete visualization of the perforating arteries to the brain stem, and the necessity to consider the antegrade flow from the anterior circulation.
Because endovascular revascularization therapy is becoming the main strategy for acute ischemic stroke, it is important to use the optimal scale with high reliability in decisions of revascularization end points. In fact, revascularization can be understood as angiographic recanalization of the primary arterial occlusive lesion or reperfusion in the arterial bed distal to the occlusion (TIMI, TICI, mTICI). The AOL scale has been the sole scoring system for measuring the degree of recanalization at the target arterial lesion since its introduction in the Interventional Management of Stroke trials,32⇓–34 and the posterior circulation occlusions were categorized according to AOL recanalization in Interventional Management of Stroke III.32 Although Gaha et al4 reported that observer variability for the AOL was “moderate” in anterior circulation stroke, we found that this scheme had high reliability in posterior circulation stroke. Considering its ease of use and consistency, it is possible to evaluate the AOL as a recanalization scale in a further posterior circulation stroke study.
This study had some limitations. First, the study design was retrospective with a limited number of cases. Second, the predictive power of both scales in this study was not analyzed due to the sampling heterogeneity. In fact, whether to choose the recanalization or reperfusion scale as a determinant of clinical outcome in posterior circulation stroke is still controversial. Cho et al35 reported that reperfusion was a reliable surrogate and the strongest predictor of clinical outcome in anterior circulation stroke, and Singer et al36 reported that independent predictors of clinical outcome were not the TICI scale, but the collateral status in BAO. In contrast, Mourand et al37 proposed a DWI brain lesion score for prediction of clinical outcome in patients with BAO by using brain MR imaging. Finally, unfortunately in this study, only bilateral vertebral angiographies were used for interpretation, which may cause underdiagnosis of the mTICI because of the imperfect evaluation of collateral flows from the circle of Willis circulation.
Conclusions
In angiographic assessment of posterior circulation stroke, this is the first study to evaluate inter- and intraobserver variability for 2 commonly used scales; while the AOL as a recanalization scale showed a higher reliability, the mTICI, as a reperfusion criterion, failed to achieve substantial interobserver agreement among readers. In future studies, it will be necessary to validate the clinical impact of this result in posterior circulation stroke.
Acknowledgments
We thank Kyung Hwa Han, MS, for her assistance with statistics for this study.
Footnotes
This study was supported by a faculty research grant from Yonsei University College of Medicine (6-2015-0051).
References
- Received June 3, 2015.
- Accepted after revision July 22, 2015.
- © 2016 by American Journal of Neuroradiology
Jump to section
Related Articles
Cited By...
- Endovascular treatment for isolated posterior cerebral artery occlusion stroke in the MR CLEAN registry
- Thrombectomy for secondary distal, medium vessel occlusions of the posterior circulation: seeking complete reperfusion
- Effect of first pass reperfusion on outcome in patients with posterior circulation ischemic stroke
- Biomarkers of Technical Success After Embolectomy for Acute Stroke
- Interobserver Agreement in Scoring Angiographic Results of Basilar Artery Occlusion Stroke Therapy
- Feasibility and safety of thrombectomy for isolated occlusions of the posterior cerebral artery: a multicenter experience and systematic literature review
- Endovascular treatment of tandem occlusions in vertebrobasilar stroke: technical aspects and outcome compared with isolated basilar artery occlusion
- Mechanical thrombectomy for basilar artery occlusion: efficacy, outcomes, and futile recanalization in comparison with the anterior circulation