Inter-rater reliability of the simplified Modified Rankin Scale as an outcome measure for treated cerebral aneurysm patients

Background: The modified Rankin scale (mRS) is commonly used as a clinical outcome measure in aneurysm trials, but inter-observer reliability in treated patients has not been tested.

Methods: We reviewed the literature on inter-observer reliability studies of the mRS. Sixty patients with ruptured (n=47) or unruptured (n=13) aneurysms treated with endovascular methods (n=34) or surgical clipping (n=26) were independently evaluated by a neurosurgeon, a stroke neurologist, and a novice research assistant, and a simplified mRS score assigned. Results were analyzed using Gwet's AC_1/2 reliability coefficients (K_G).

Results: No previous reports validating the reliability of the mRS in treated aneurysm patients were identified. Using the mRS 0-5, inter-rater agreement was almost perfect (K_G=0.89 [0.86-0.93]). Agreement between raters remained almost perfect regardless of the rater's expertise. Agreement was almost perfect (K_G=0.87 [0.77-0.96] when the mRS was dichotomized 0-2 vs 3-5, but fell to moderate when dichotomized 0-1 vs 2-5 (K_G=0.59 (0.42-0.75). Agreement using the 0-2 vs 3-5 dichotomized mRS remained almost perfect for coiled (K_G=0.90), clipped (K_G=0.82), ruptured (K_G=0.84), and unruptured (K_G=0.95) aneurysms. Dichotomization of results at 0-1 vs 2-5 would have resulted in an (undesirable) significant difference in good outcomes between raters (P=.003), but not at 0-2 vs 3-5 (P=.52).

Conclusion: The simplified mRS appears to be a reliable clinical outcome measure for treated cerebral aneurysm patients. When needed, dichotomization is more reliable at mRS 0-2 vs 3-5 than at 0-1 vs 2-5. The simplified mRS is a promising tool in the functional assessment of aneurysm patients recruited in pragmatic care trials.