The First Examination of Diagnostic Performance of Automated Measurement of the Callosal Angle in 1856 Elderly Patients and Volunteers Indicates That 12.4% of Exams Met the Criteria for Possible Normal Pressure Hydrocephalus

BACKGROUND AND PURPOSE: Many patients with dementia may have comorbid or misdiagnosed normal pressure hydrocephalus, a treatable neurologic disorder. The callosal angle is a validated biomarker for normal pressure hydrocephalus with 93% diagnostic accuracy. Our purpose was to develop and evaluate an algorithm for automatically computing callosal angles from MR images of the brain. MATERIALS AND METHODS: This article reports the results of analyzing callosal angles from 1856 subjects with 5264 MR images from the Open Access Series of Imaging Studies and the Alzheimer’s Disease Neuroimaging Initiative databases. Measurement variability was examined between 2 neuroradiologists (n1⁄4 50) and between manual and automatic measurements (n1⁄4 281); from differences in simulated head orientation; and from real-world changes in patients with multiple examinations (n1⁄4 906). We evaluated the effectiveness of the automatic callosal angle to differentiate normal pressure hydrocephalus from Alzheimer disease in a simulated cohort. RESULTS: The algorithm identified that 12.4% of subjects from these carefully screened cohorts had callosal angles of ,90°, a published threshold for possible normal pressure hydrocephalus. The intraclass correlation coefficient was 0.97 for agreement between neuroradiologists and 0.90 for agreement between manual and automatic measurement. The method was robust to different head orientations. The median coefficient of variation for repeat examinations was 4.2% (Q1 1⁄4 3.1%, Q3 1⁄4 5.8%). The simulated classification of normal pressure hydrocephalus versus Alzheimer using the automatic callosal angle had an accuracy, sensitivity, and specificity of 0.87 each. CONCLUSIONS: In even the most pristine research databases, analyses of the callosal angle indicate that some patients may have normal pressure hydrocephalus. The automatic callosal angle measurement can rapidly and objectively screen for normal pressure hydrocephalus in patients who would otherwise be misdiagnosed. ABBREVIATIONS: AD 1⁄4 Alzheimer disease; ADNI 1⁄4 Alzheimer’s Disease Neuroimaging Initiative; CA 1⁄4 callosal angle; DESH 1⁄4 disproportionately enlarged subarachnoid space hydrocephalus; ICC 1⁄4 intraclass correlation coefficient; NPH 1⁄4 normal pressure hydrocephalus; OASIS 1⁄4 Open Access Series of Imaging Studies Normal pressure hydrocephalus (NPH) is a treatable form of dementia that can be difficult to diagnose. Clinical features of NPH are gait disturbance, postural instability, cognitive deterioration, and urinary incontinence or urgency, but these features are frustratingly nonspecific in elderly patients. Classic neuroimaging findings show differences from the atrophy routinely Received March 5, 2021; accepted after revision July 3. From the Fetal and Neonatal Institute, Division of Neonatology (M.B.), Department of Radiology (A.S., S.B.), and Division of Neurosurgery (J.G.M.), Children’s Hospital Los Angeles, Los Angeles, California; Department of Pediatrics (M.B.), and Neurological Surgery (J.G.M.), Keck School of Medicine, University of Southern California, Los Angeles, California; Rudi Schulte Research Institute (M.B., A.S., S.B., K.S.K.), Santa Barbara, California; and Department of Neuroradiology (J.H., K.S.K.), Barrow Neurological Institute, Phoenix, Arizona. This work was supported by the Rudi Schulte Research Institute. Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative database (adni.loni.usc.edu). Thus, the investigators within the Alzheimer’s Disease Neuroimaging Initiative contributed to the design and implementation of Alzheimer’s Disease Neuroimaging Initiative and/or provided data but did not participate in the analysis or writing of this report. A complete listing of Alzheimer’s Disease Neuroimaging Initiative investigators can be found at http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_ Acknowledgement_List.pdf. Data collection and sharing for this project were funded, in part, by the Alzheimer Disease Neuroimaging Initiative (National Institutes of Health Grant U01 AG024904) and Department of Defense Alzheimer Disease Neuroimaging Initiative (Department of Defense award numberW81XWH-12–2-0012). The Alzheimer’s Disease Neuroimaging Initiative is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: AbbVie; Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica; Biogen; Bristol-Myers Squibb; CereSpir; Cogstate; Eisai; Elan Pharmaceuticals; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech; Fujirebio; GE Healthcare; IXICO Ltd; Janssen Alzheimer Immunotherapy Research & Development; Johnson & Johnson Pharmaceutical Research & Development LLC; Lumosity; Lundbeck; Merck & Co; Meso Scale Diagnostics, LLC; NeuroRx Research; NeuroTrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support the Alzheimer Disease Neuroimaging Initiative clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Therapeutic Research Institute at the University of Southern California. Alzheimer Disease Neuroimaging Initiative data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California. Data were provided, in part, by OASIS. OASIS-3: Principal Investigators: T. Benzinger, D. Marcus, J. Morris; NIH P50 AG00561, P30 NS09857781, P01 AG026276, P01 AG003991, R01 AG043434, UL1 TR000448, R01 EB009352. AV-45 doses were provided by Avid Radiopharmaceuticals, a wholly owned subsidiary of Eli Lilly. 1942 Borzage Nov 2021 www.ajnr.org observed in senescent adults. Specialized physiologic imaging of water diffusion, CSF flow, or cerebrovascular reactivity may help with diagnosing NPH, but each method requires acquiring additional prospective images that are not commonly included in clinical assessments. Lumbar drain trials have diagnostic utility but are invasive; noninvasive biomarkers with strong evidence of therapeutic benefits are preferred before attempting treatment by shunting, which has an 11% risk of serious adverse events. Fortunately, numerous putative NPH imaging biomarkers exist including the following: anatomic assessments of the relative size and shape of the ventricles and subarachnoid spaces; disproportionately enlarged subarachnoid space hydrocephalus (DESH); volume-based assessments of CSF and ratios versus intracranial volume; distance-based assessment of the ventricles versus intracranial width, ie, the Evans index; and angle-based measurements of the parietal portion of the lateral ventricles, ie, the callosal angle (CA). Measurement of the CA, at times used in concert with the Evans index, is a validated biomarker for NPH, with diagnostic accuracies of 93%, 77.8%, and 88.9% for threshold angles of 90°, 90.8°, and 100°, respectively, as validated in studies of 102, 90, and 318 patients, respectively. With an abundance of useful biomarkers, deploying them into clinical practice entails manipulation of the images using 3D software, which requires an investment of precious time by the interpreting radiologist. Since NPH may not be among the most likely differential diagnoses for an elderly patient, manually measuring these biomarkers for screening purposes is impractical. An alternative approach is to perform automated analysis of images to measure these biomarkers and present results to radiologists to interpret. The advantages of such an approach are that the measurement eliminates the need to perform manual assessment and removes observer variability. For these putative imaging biomarkers, automated solutions exist for calculating the DESH, Evans index, and CSF volumes and ratios, but to the best of our knowledge, measurements of the CA have not yet been automated. Therefore, our objective was to automate CA measurements and then do the following: 1) assess the agreement between 2 neuroradiologists measuring the CA and between manual and automatic measurements; 2) evaluate the variation of automated CA under both simulated and real-world conditions; 3) use the algorithm to analyze MR images to identify patients with possible NPH in studies with different scanners, vendors, and imaging parameters; and 4) characterize the performance of the automated measurement for the differentiation of NPH from other dementia. MATERIALS AND METHODS Subjects We included data from 2 imaging databases: the Open Access Series of Imaging Studies (OASIS), (n1⁄4 1015 subjects, 567 women, 448 men) and the Alzheimer Disease Neuroimaging Initiative (ADNI), (n1⁄4 841 subjects, 354 women, 487 men). The mean ages of the subjects at the time of their MR imaging were 68.5 (SD, 9.3) and 75.3 (SD, 6.9) years, respectively. OASIS subjects included groups for cognitively normal and any stage of cognitive decline in aging. New OASIS subjects underwent a clinical assessment, which included a family history of Alzheimer disease (AD), medical history, physical examination, neurologic evaluation, and MR imaging. Stages of cognitive decline were determined using the Clinical Dementia Rating Scale, and patients were excluded if the primary cause of dementia was not AD. ADNI subjects included groups for cognitively healthy, early mild cognitive impairment, late mild cognitive impairment, and AD. New ADNI subjects in all groups were excluded for NPH during an initial screening visit, which included neuropsychological testing andMR imaging. Neuroimaging OASIS neuroimaging was performed on 1.5T (Vision) and 3T (TIM Trio, BioGraph mMR) Siemens clinical scanners. The images were T1-weighted MPRAGE (TE 1⁄4 4.0ms, TR 1⁄4 9.7ms, TI 1⁄4 20ms, slice thickness1⁄4 1.25mm, matrix1⁄4 256 256). ADNI neuroimaging was performed on 1.5T (Signa Excite, Signa HDxt) and 3T (Discovery, Signa Premier) GE Healthcare, 1.5 T (Intera) and 3T (Achieva, Ingenia) Philips Healthcare, and 1.5T (Avanto, Espree, Sonata, Symphony) and 3T (Allegra, Prisma, Skyra, Trio/TIM, Verio, Vida) Siemens clinical scanners. The images were T1weighted MPRAGE with variations in protocol depending on the scanner and software (TE 1⁄4 2.86–4.61ms, TR 1⁄4 2300–3000ms, TI 1⁄4 853–1000ms, slice thickness 1⁄4 1.2mm, matrix 192–256 192–256). Manual CA Measurement Two board-certified neuroradiologists (K.S.K., J.H.) acquired manual CA measurements using established methods: identifying a midsagittal section, creating a reference plane through the anterior commissure and posterior commissure, creating a coronal reference plane perpendicular to the bicommissural plane at the level of the posterior commissure, drawing 2 straight lines along the medial walls of the left and right lateral ventricles on the coronal image, and calculating the angle between the lines. Automated CA Measurement Images were preprocessed with FreeSurfer (http://surfer.nmr.mgh. harvard.edu) to align to a standard orientation and extract the ventricles. The extracted ventricles included the left and right lateral ventricles, choroid plexus, and the third ventricle. CA measurements were calculated in Matlab (R2020A; MathWorks). The algorithm applied to the ventricles creates an axial reference plane by using the centroid of the ventricles and the most distal points on the left and right anterior horns; makes a coronal reference plane perpendicular to the axial plane; pitches the coronal reference plane backward to make it oblique; slices the ventricles; and computes the angle between the medial walls of the lateral ventricles. See the algorithm details in Fig 1. Parameter Optimization Radiologists use the anterior/posterior commissure plane as a reference plane when performing manual CA measurements. This bicommissural plane is more difficult to reliably compute with automated methods than the centroid and distal points of Please address correspondence to A. Saunders, MS, Children’s Hospital of Los Angeles, Department of Radiology, 4650 Sunset Blvd., Los Angeles, CA 90027; e-mail: saundera@usc.edu Indicates open access to non-subscribers at www.ajnr.org http://dx.doi.org/10.3174/ajnr.A7294 AJNR Am J Neuroradiol 42:1942–48 Nov 2021 www.ajnr.org 1943 the left and right lateral ventricles. However, using our different reference planes would lead to systematic bias in the automatically measured CA. To reduce this bias, we identified the key difference in the reference planes as a coronal pitch. Thus, we systematically varied the pitch in the coronal plane with angles from 45° to 85° in increments of 5°. At each angle of pitch, we remeasured the CA for all subjects for whom we had manual CA measurements and determined the pitch that minimized error between the manual and automated CAs. For the optimized angle, we used the linear trend from the correlation of manual and automatic CAs to adjust all subsequent automatic

This work was supported by the Rudi Schulte Research Institute.
Data used in preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative database (adni.loni.usc.edu). Thus, the investigators within the Alzheimer's Disease Neuroimaging Initiative contributed to the design and implementation of Alzheimer's Disease Neuroimaging Initiative and/or provided data but did not participate in the analysis or writing of this report. A complete listing of Alzheimer's Disease Neuroimaging Initiative investigators can be found at http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_ Acknowledgement_List.pdf.
Data collection and sharing for this project were funded, in part, by the Alzheimer Disease Neuroimaging Initiative (National Institutes of Health Grant U01 AG024904) and Department  observed in senescent adults. Specialized physiologic imaging of water diffusion, CSF flow, or cerebrovascular reactivity may help with diagnosing NPH, 3,4 but each method requires acquiring additional prospective images that are not commonly included in clinical assessments. Lumbar drain trials have diagnostic utility but are invasive; 5 noninvasive biomarkers with strong evidence of therapeutic benefits are preferred before attempting treatment by shunting, which has an 11% risk of serious adverse events. 4,6 Fortunately, numerous putative NPH imaging biomarkers exist including the following: anatomic assessments of the relative size and shape of the ventricles and subarachnoid spaces; disproportionately enlarged subarachnoid space hydrocephalus (DESH); 7 volume-based assessments of CSF and ratios versus intracranial volume; 8 distance-based assessment of the ventricles versus intracranial width, ie, the Evans index; 3 and angle-based measurements of the parietal portion of the lateral ventricles, ie, the callosal angle (CA). 9 Measurement of the CA, at times used in concert with the Evans index, is a validated biomarker for NPH, with diagnostic accuracies of 93%, 77.8%, and 88.9% for threshold angles of 90°, 90.8°, and 100°, respectively, as validated in studies of 102, 90, and 318 patients, respectively. [10][11][12] With an abundance of useful biomarkers, deploying them into clinical practice entails manipulation of the images using 3D software, which requires an investment of precious time by the interpreting radiologist. Since NPH may not be among the most likely differential diagnoses for an elderly patient, manually measuring these biomarkers for screening purposes is impractical. An alternative approach is to perform automated analysis of images to measure these biomarkers and present results to radiologists to interpret. The advantages of such an approach are that the measurement eliminates the need to perform manual assessment and removes observer variability. For these putative imaging biomarkers, automated solutions exist for calculating the DESH, 13 Evans index, 14 and CSF volumes and ratios, 15 but to the best of our knowledge, measurements of the CA have not yet been automated. Therefore, our objective was to automate CA measurements and then do the following: 1) assess the agreement between 2 neuroradiologists measuring the CA and between manual and automatic measurements; 2) evaluate the variation of automated CA under both simulated and real-world conditions; 3) use the algorithm to analyze MR images to identify patients with possible NPH in studies with different scanners, vendors, and imaging parameters; and 4) characterize the performance of the automated measurement for the differentiation of NPH from other dementia.
The mean ages of the subjects at the time of their MR imaging were 68.5 (SD, 9.3) and 75.3 (SD, 6.9) years, respectively. OASIS subjects included groups for cognitively normal and any stage of cognitive decline in aging. New OASIS subjects underwent a clinical assessment, which included a family history of Alzheimer disease (AD), medical history, physical examination, neurologic evaluation, and MR imaging. Stages of cognitive decline were determined using the Clinical Dementia Rating Scale, 19 and patients were excluded if the primary cause of dementia was not AD. ADNI subjects included groups for cognitively healthy, early mild cognitive impairment, late mild cognitive impairment, and AD. New ADNI subjects in all groups were excluded for NPH during an initial screening visit, which included neuropsychological testing and MR imaging.

Manual CA Measurement
Two board-certified neuroradiologists (K.S.K., J.H.) acquired manual CA measurements using established methods: 9,10 identifying a midsagittal section, creating a reference plane through the anterior commissure and posterior commissure, creating a coronal reference plane perpendicular to the bicommissural plane at the level of the posterior commissure, drawing 2 straight lines along the medial walls of the left and right lateral ventricles on the coronal image, and calculating the angle between the lines.

Automated CA Measurement
Images were preprocessed with FreeSurfer (http://surfer.nmr.mgh. harvard.edu) to align to a standard orientation and extract the ventricles. 20 The extracted ventricles included the left and right lateral ventricles, choroid plexus, and the third ventricle. CA measurements were calculated in Matlab (R2020A; MathWorks). The algorithm applied to the ventricles creates an axial reference plane by using the centroid of the ventricles and the most distal points on the left and right anterior horns; makes a coronal reference plane perpendicular to the axial plane; pitches the coronal reference plane backward to make it oblique; slices the ventricles; and computes the angle between the medial walls of the lateral ventricles. See the algorithm details in Fig 1.

Parameter Optimization
Radiologists use the anterior/posterior commissure plane as a reference plane when performing manual CA measurements. This bicommissural plane is more difficult to reliably compute with automated methods than the centroid and distal points of the left and right lateral ventricles. However, using our different reference planes would lead to systematic bias in the automatically measured CA. To reduce this bias, we identified the key difference in the reference planes as a coronal pitch. Thus, we systematically varied the pitch in the coronal plane with angles from À45°to 85°in increments of 5°. At each angle of pitch, we remeasured the CA for all subjects for whom we had manual CA measurements and determined the pitch that minimized error between the manual and automated CAs. For the optimized angle, we used the linear trend from the correlation of manual and automatic CAs to adjust all subsequent automatic CA measurements.

Assessing Measurement Variability
We performed 3 different assessments of measurement variability: 1) We used Bland-Altman analysis to ascertain the limits of agreement between CAs manually measured by both neuroradiologists (n ¼ 50) and between 1 neuroradiologist's manual CA measurements and linearly corrected automatic CA measurements (n ¼ 281). 2) We performed a Monte Carlo simulation to assess potential sources of variability due to misalignment and localization errors during image acquisition and FreeSurfer preprocessing. We selected 24 subjects and reoriented their heads multiple times. Each head was randomly translated in three directions (SD 8 mm in each direction) and randomly rotated in three directions (SD 10°, 3°, and 6°for pitch, roll, and yaw, respectively). After reorienting their heads, we automatically computed the CA. Leverage plots were used to determine the sensitivity of each angle measurement to these translations and rotations (not shown). 3) We computed the coefficient of variation of subjects who had $3 MRIs to assess real-world intrapatient variability. Our assumption was that these repeat scans would include real-world variability in image quality, patient orientation, scanner drift, and ventricle morphology. For each subject, we automatically measured the CA for each repeat scan and computed the coefficient of variation of the angles.

Simulated Effectiveness of the CA to Differentiate NPH and AD
We performed a Monte Carlo simulation to examine the effectiveness of the automatic CA measurement to differentiate NPH and AD. We simulated CAs for 1 million patients. Each patient was randomly assigned to NPH or AD with equal likelihood. Ground truth CAs were generated from a Gaussian distribution for patients from the NPH (mean 66°[SD, 14°]) and AD (mean 104°[SD, 15°]) cohorts as given by Ishii et al. 10 Measurement error due to automatic CA calculation was randomly added to each angle: This error was determined by the distribution of error in our comparison of manual and linearly corrected automatic measurements. We calculated the probability that the random angle belonged to the NPH or AD group and used the higher probability to classify the patient as having NPH or AD. When the probabilities were within 5% of each other, we classified the patient as indeterminate. Classification performance was quantified using accuracy, sensitivity, and specificity.

Analysis of NPH and AD Prevalence and Comorbidity
We estimated the probability of comorbid or misdiagnosed NPH in the evaluated databases and clinical practice. We used literature values for prevalence and diagnostic accuracies of NPH and AD in a relevant population. This analysis used the following assumptions: Our relevant population is cognitively impaired patients 65 years of age and older; all patients in this cognitively impaired population have either NPH (NPH1AD-), AD (NPH-AD1), or comorbid NPH and AD (NPH1AD1); the biologic processes and thus the probabilities of having NPH or AD are independent.  (2). These 3 points are used to calculate the axial reference plane (3) and the coronal reference plane (4), shown here without pitch correction. B, The coronal plane pitch correction is optimized by finding the angle that maximizes the correlation of manual and automated CA measurements (30°). For each pitch correction, the percentage of examinations with the least error is also shown. C, The pitch-corrected coronal section is analyzed by finding the most superior point to the left and right of the centroid (5). A greedy pathfinding algorithm connects the 2 superior reference points (5) to identify the medial walls of the lateral ventricles (red points). The inferior 20% and superior 20% of red points are excluded, selecting the middle 60% for angle calculation; this was found empirically to exclude the portions of the ventricle walls with higher curvature. A first-order polynomial fit produces the fit lines (blue lines), and the angle between them is calculated. Figure 1 demonstrates the automatic CA measurement algorithm and determination of correction factors. Figure 1B shows the comparison of manual and automatic CA measurements (n ¼ 281) to optimize the coronal reference plane pitch. The highest correlation was found when adjusting the pitch of the automatic coronal reference plane back (ie, the most superior part of the plane moved posterior) by 30°, and the highest percentage of examinations with the least error between manual and automatic measurements occurred at a 50°pitch correction. We selected the best correlation (30°) for our optimization; the correlation protects against large measurement errors by minimizing the squared error, whereas the percentage of examinations with the fewest errors does not. However, the correlation and percentage of examinations with the fewest errors changed only modestly, with pitch corrections ranging from 25°to 50°, meaning that pitch correction is relatively insensitive over this range. At a 30°pitch correction, the intraclass correlation coefficient (ICC) was 0.87, the coefficient of determination was R 2 ¼ 0.82, the median absolute error was 8.1°( Q1 ¼ 3.4°, Q3 ¼ 13.2°), and the line of fit was Manual CA ¼ Automatic CA Â 1:27 À 3:93. The pitch correction and linear correction factors were applied to all subsequent automated CA measurements.

RESULTS
The Bland-Altman analysis of the 2 neuroradiologists' manual measurements (Fig 2A) had a bias of -3.78°with 95% limits of agreement of À14.95°to 17.39°, a median absolute error of 4°( Q1 ¼ 3°, Q3 ¼ 7°), an ICC of 0.97, and coefficient of determination of R 2 ¼ 0.95. Bland-Altman analysis of the manual and linearly corrected automatic measurements ( Fig 2B) had no bias, with 95% limits of agreement of about 622°, median absolute error of 6.5°(Q1 ¼ 2.9°, Q3 ¼ 12.0°), an ICC of 0.90, and coefficient of determination R 2 ¼ 0.82. For the Monte Carlo analysis of head position sensitivity, we re-oriented each subject's head (mean, n ¼ 1122 [SD, 381] times). The automatically measured CA was sensitive to initial orientation, with significant change in the CA measurements due to pitch, roll, and yaw rotations (P , .001, P ¼ .028, and P ¼ .004 respectively) and left-right translation (P , .001), but not anterior-posterior or superior-inferior translations (P ¼ .222, P ¼ .350, respectively). However, the parameter estimates for these significant factors indicate that the effects are negligible: Estimates for pitch, roll, and yaw rotations are À0.006°, 0.004°, and À0.003°, respectively, and leftright translation is À0.001°, indicating that a 1°of pitch, roll, or yaw corresponds to a CA change of #0.006°, and a 1-mm translation corresponds to a CA change of #0.001°. For the real-world intrapatient variability analysis, we identified n ¼ 906 subjects (n ¼ 213 OASIS, n ¼ 693 ADNI), each with between 3 and 8 repeat MRIs. For these subjects, we determined that the median coefficient of variation was 4.2% (Q1 ¼ 3.1%, Q3 ¼ 5.8%). The median time between repeat examinations was 214 days (Q1 ¼ 183 days, Q3 ¼ 396 days). See Fig 3 for details on the distribution of the coefficient of variation in these subjects. A visual inspection of all automatic measurements (A.S., with 3 years' experience) found that 52 angles (1%) had fit lines that deviated from the expected placement.
When applying the algorithm to all available images, we computed 5264 CA measurements. The median CA was 113°F IG 2. Measurement variability. A, We found a slight bias (À3.78°) with 95% limits of agreement of À14.95°to 17.39°between 2 radiologists. This translates to an intraclass correlation coefficient of 0.97, demonstrating good reproducibility of the callosal angle biomarker between 2 readers. B, Comparison of manual and linearly corrected automatic CA measurements for n = 281 images had 95% limits of agreement of about 622°and an ICC of 0.90. We observed lower agreement between manual and automatic measurements than between the 2 neuroradiologists, which highlights the impact of differences in measurement methods.
For the Monte Carlo analysis of classification using the automatic CA measurement, we calculated the accuracy, sensitivity, and specificity to all be 0.87. The percentage of indeterminate measurements was 5.4%. For the analysis of NPH and AD comorbidity, we used literature values for the following: the prevalence of NPH of 2.9% (mean of 3 studies), 21-23 the prevalence of AD of 11.7%, 24 and the diagnostic accuracy for AD of 77%. 25 Thus, if a patient is diagnosed with AD, the probabilities of the true disease processes are 96.3% NPH-AD1, 2.8% NPH1AD1 (comorbid), and 0.9% NPH1AD-(misdiagnosed). We calculated the sum of the latter 2 values and estimate that 3.7% of patients diagnosed with AD actually were NPH1.

DISCUSSION
To the best of our understanding, this is the first automated CA measurement and the largest number of CA measurements made. Thus, the distribution of the CA provides a revised and expanded population for comparing individual CA measurements. Obtaining these results through manual CA measurements would be extremely time-consuming and prone to observer variability, which highlights the suboptimal reliability of manual CA measurement. Modest differences in the selection of the bicommissural plane, the anterior-posterior position of the coronal reference plane, and lines that best parallel the medial walls of the lateral ventricles contribute to different measurements of the CA.
One of our most noteworthy findings was that at least 12.4% of the images we measured met the CA criteria for possible NPH. 10 The estimated rate of NPH is 2.1%-3.9% in adults older than 65 years of age. 21-23 Our estimate of comorbid or misdiagnosed NPH among patients diagnosed with AD was 3.7%, which is consistent with the reported value of 3.9%. 26 If the databases we examined contain patients with NPH who are classified as neurotypical or diagnosed with other dementias, it suggests that other analyses using these databases may be skewed by a substantial fraction of the overall sample. Perhaps even more important, if these patients do have NPH or another hydrocephalus indicated by a narrowed CA, their dementias may be treatable.
The Monte Carlo analysis of NPH classification performance allowed us to compute the accuracy, sensitivity, and specificity (all 0.87) of our automated approach, which were lower than results from Ishii et al 10 (0.93, 0.97, and 0.88, respectively). Despite mild performance decreases when using the automatic CA measurement, the potential for rapid evaluation is substantial and may be particularly valuable in cases in which NPH findings are incidental.
The automated measurement is deterministic, eliminating observer variability and establishing a more structured reporting framework between radiologists and clinicians for the CA biomarkers. The benefit of the robust approach we selected was demonstrated in both intrapatient variability analyses. The Monte Carlo analysis demonstrated that our approach is robust to the initial position and orientation of the brains of our subjects; thus, even if FreeSurfer fails to correctly align the heads or subjects have an abnormal orientation of their ventricles, the effect on the CA measurements would be small (,1°) across the entire range of rotations and translations tested. This finding appears to be sufficient to handle most scenarios in which the head is even crudely oriented. The real-world analysis demonstrated a median coefficient of variation of 4.2%, representing errors which are 8.7-13.6 times smaller than the putative differences between the mean CA in patients with NPH (66°) and AD (104°). 10 It is reasonable to expect that in a clinical radiology practice, these automatic CA measurements  would provide highly repeatable quantitation to support distinguishing NPH from atrophy in patients with a clinical suspicion of movement disorders and dementia. The systematic bias between the automatic and manually computed CA is likely due to different methods for selecting axial reference planes. The automatic method identifies landmarks from the segmented surface representations of the lateral ventricles, while the clinical method uses the anterior/posterior commissures. The finding that a 30°pitch correction improves agreement with manual measurement is consistent with the typical angle of the bicommissural plane relative to the axial plane, and measurement of the angle in the posterior area of the lateral ventricles may better capture pathologically narrow angles as the ventricles expand upward on both sides of the falx cerebri. 27 We chose reference points for the automated method to be reliable but still comparable with those used in manual CA measurements. The centroid of the ventricles represents a global average that is inherently robust, and the anterior horns have less anatomic variability than the posterior extrema or superior extrema, which we speculate may be altered by NPH. Furthermore, the reference points were selected to be reproducible across a wide range of image qualities and modalities, which we suspect includes CT. CT is commonly included in primary imaging studies for dementia work-up; adapting the automatic CA method for use in CT is a promising future option to reach additional patients who may not have undergone MRI.

Limitations
A limitation of the field of NPH research, and thus this article, is the lack of treatment-responsive patients with NPH in a public database. Such a database would enable us to directly assess the accuracy, specificity, and sensitivity of our measures in identifying NPH. Thus, future objectives for those studying NPH, including our group, should include collecting neuroimaging of shuntresponsive patients with NPH. Meanwhile, our algorithm can flag potential cases of a narrowed CA, which can then be verified by a trained reader as we have done in this study. We chose to demonstrate the utility of our efforts on making a CA algorithm that is pragmatic to use by providing secondary analyses of subjects in large existing databases. There are areas of possible improvement in our algorithm: The automated CA measurement had some performance disadvantages when evaluated against manual observations. There may be an opportunity to adjust the algorithm to be more robust to irregular variations in the segmented ventricles, but these errors were infrequent. There are also other imaging biomarkers that might benefit patients with NPH, which we did not use (eg, the Evans Index), an area with potential for future investigation. One minor limitation of our study is that we performed manual measurements on 281 of our total 5264 images (5.3%). This limitation is due to the substantial time requirement for a neuroradiologist to perform CA measurements, which highlights the need for automated tools if CA or other quantitative biomarkers are to be routinely measured.

CONCLUSIONS
NPH is a treatable dementia that is commonly misdiagnosed due to the poor specificity of its neurologic symptoms. CA measurements are an established tool to assess the risk for NPH, but manual measurement is time-consuming. We developed an algorithm for automated CA measurement, applied it to 5264 T1weighted MRIs, and compared its performance with manual CA measurements. We found that agreement between manual and automatic measurements (ICC ¼ 0.90) was lower than the agreement between 2 neuroradiologists (ICC ¼ 0.97). Intrapatient variability was evaluated in subjects with $3 longitudinal imaging examinations; the median coefficient of variation was 4.2%, indicating reliable automatic measurement. Although NPH was an exclusion criterion from these databases, 12.4% of the automatic CA measurements met the criteria for possible NPH. We believe automatic CA measurements can rapidly and objectively assess NPH in patients who would otherwise be misdiagnosed with other dementias, and can create opportunities for successful treatment of dementia.
Disclosures: Kevin S. King-RELATED: Grant: Rudi Schulte Research Institute, Comments: My work on this project was conducted with support of the nonprofit Rudi Schulte Research Institute. The Rudi Schulte Research Institute is a trust established to support hydrocephalus-related research by issuing competitive fellowships in response to submitted research proposals.