Validity Assessment of an Automated Brain Morphometry Tool for Patients with De Novo Memory Symptoms

Participants

Participants were randomly selected from the Open Access Series of Imaging Studies 4 (OASIS-4) cohort, which is publicly accessible through the OASIS brain website: https://central.xnat.org/. Participants were included under the following circumstances: 1) They were referred for clinical assessment due to a de novo cognitive symptom, 2) were 45 years of age or older, and 3) had a structural T1-weighted MR imaging study within a maximum of 1 year of the initial assessment. Diagnosis of dementia was made on the basis of the clinical assessment and a battery of cognitive tests, including the Clinical Dementia Rating (CDR)⁶ and Mini-Mental State Examination.⁷ If dementia was present, an etiologic diagnosis was further determined on the basis of clinical practices for Alzheimer disease (AD), posterior cortical atrophy, dementia with Lewy bodies (DLB), frontotemporal dementia (FTD), and vascular cognitive impairment.^8⇓⇓-11 This diagnosis was made by a neurologist clinician at the end of the recruitment visit and before any imaging assessment. Reflecting the proportion of each disease category in the OASIS-4 cohort, the current sample comprised a random selection of 15 individuals with AD, including 5 participants with early-onset AD; 10 participants with non-neurodegenerative conditions; such as subjective cognitive impairment in the absence of clinical dementia, mood disorders, polypharmacy, and sleep disorders; 5 with posterior cortical atrophy; 5 with DLB; 5 with FTD, and 5 with vascular cognitive impairment (Online Supplemental Data). The 15-item version of the Geriatric Depression Scale (GDS) was used to screen participants for the presence of depressive symptoms in which a cutoff score of 5 has shown 92% sensitivity (Online Supplemental Data).¹²

This study was conducted using a research agreement between Washington University School of Medicine in St. Louis and Siemens Medical Solutions USA and was reviewed and approved by the institutional review board of Washington University in Saint Louis School of Medicine (IRB No. 201912172).

Image Data Collection

The 3D T1WI MPRAGE and T2-weighted FLAIR images were acquired on a 3T scanner (Magnetom Skyra; Siemens) using a TR/TE = 2300/2.95 ms, TI = 900 ms, flip angle = 9°, section thickness = 1 mm, and FOV = 256 × 256 for the T1WI scans, and a TR/TE = 900/81 ms, TI = 2500 ms, flip angle = 150°, section thickness = 5 mm, within a 256 × 256 FOV for FLAIR scans. SWI was performed in the same session and used the following parameters: TR/TE = 27/20 ms, flip angle = 15°, section thickness = 2.4 mm, FOV = 256 × 256.

In all subsequent analyses, the absolute volumes refer to raw estimates produced by each tool, normalized volumes refer to absolute volumes divided by their corresponding estimated total intracranial volume (TIV), and standardized volumes refer to z scores calculated by comparing the normalized volumes with their respective normative database.

AIRC Tool

The AIRC Brain MR tool creates brain morphometry reports using a T1WI MPRAGE series and through a tissue-wise segmentation model, resulting in a considerably reduced computation time (2–5 minutes) compared with other segmentation software such as FreeSurfer.^3,13 This tool produces volumes of 25 different brain regions in both hemispheres (50 total) and compares them with age- and sex-matched normative data from a healthy population. Results are presented as a labeling report consisting of a label map showing the segmentation results (Fig 1A); a deviation report consisting of a deviation map and the corresponding standardized volumes for each region; and a list of evaluated volumes and their corresponding TIV-normalized measures displayed alongside the 10th–90th percentile normative ranges based on the participant’s age and sex group. Regions with normative volumes that are outside this range are indicated by an asterisk (Fig 1B, -C).¹³ Once processed, a visual quality check of the labeling and deviation results is performed to ensure consistent delineation of different cortical and subcortical regions. All of the 45 scans passed this quality control.

• Download figure
• Open in new tab
• Download powerpoint

FIG 1.

AIRC brain MR imaging tool volumetric output for a 60-year-old male participant with early-onset Alzheimer disease. Labeling map (A), deviation map (B), and 1 page of the numeric report (C). Asterisk indicate values outside the normative 10th–90th percentile range for participants age and sex.

The FS/ILP Pipeline

Analytical Approach and Statistics

Statistical analyses were performed by using R software Version 4.0.5 (http://www.r-project.org/). The purpose of these analyses was the following: 1) to assess the magnitude of the correlation, consistency, or agreement between measurements from each tool, and 2) to evaluate the sensitivity and specificity of the AIRC tool compared with FS/ILP as a reference standard. The Pearson correlation and intraclass correlation statistics were used to compare the absolute and normalized regional volumes and z scores derived from the FS/ILP and AIRC tools and their respective normative data sets. When necessary, a summation of various FS-based cortical segmentation volumes was calculated to match the lobar cortical volumes reported by the AIRC tool as detailed in the Online Supplemental Data.¹⁸ The Pearson correlation coefficient and intraclass correlation coefficients (ICCs) in agreement and consistency and their respective P values were calculated by using the “corr” and “icc” functions, respectively. Pearson correlation coefficient values of <0.3, between 0.3 and 0.5, and >0.5 were considered to indicate small, moderate, and large correlations, while ICC values <0.5, between 0.5 and 0.7, between 0.7 and 0.9, and >0.9 were considered to indicate poor, moderate, good, and excellent agreement or consistency.^19,20 Additional details on the definition of these terms can be found in the Online Supplemental Data. Normal distribution of the variables was tested using the Kolmogorov-Smirnov goodness-of-fit test. P values < .05 after correction for multiple comparisons using the Benjamini-Hochberg false-discovery-rate correction rejected the null assumption.²¹ We further performed paired statistics to extract mean differences and the resulting effect sizes between volumes measured by each tool, as detailed in the Online Supplemental Data.

We compared the rates of detection of abnormal findings through comparison of final reports generated by each tool and by using the “chisq.test” function in R. The T1WI MPRAGE scans were evaluated by 3 board-certified neuroradiologists (W.W., C.A.R., and A.N.) with or without additional volumetric information provided by the AIRC or FS/ILP tools. Each participant was rated 3 times with 3 different methods, once using only the T1-weighted image (MPRAGE_Only), once after adding the FS/ILP output (MPRAGE+ILP), and once after adding the AIRC output (MPRAGE+AIRC). Raters independently assessed all 45 cases so that each participant was randomly evaluated by using one of the above 3 methods by each rater. The raters were asked to indicate radiologic impressions in a stepwise manner indicating the following: 1) whether there were any structural abnormalities related to the patient’s cognitive symptoms, 2) whether the observed abnormalities were symmetric and lobar, and 3) whether the abnormalities pointed to a specific neurodegenerative entity (AD, posterior cortical atrophy, DLB, FTD, vascular cognitive impairment). The rate of compatibility between radiologic impressions and clinical diagnoses was calculated as percentages for each method as detailed in the Online Supplemental Data and compared across methods using the “aov” and “TukeyHSD” functions in R.