Abstract
BACKGROUND AND PURPOSE: Identification of mesial temporal sclerosis is critical in the evaluation of individuals with temporal lobe epilepsy. Our aim was to assess the performance of FDA-approved software measures of hippocampal volume to identify mesial temporal sclerosis in patients with medically refractory temporal lobe epilepsy compared with the initial clinical interpretation of a neuroradiologist.
MATERIALS AND METHODS: Preoperative MRIs of 75 consecutive patients who underwent a temporal resection for temporal lobe epilepsy from 2011 to 2016 were retrospectively reviewed, and 71 were analyzed using Neuroreader, a commercially available automated segmentation and volumetric analysis package. Volume measures, including hippocampal volume as a percentage of total intracranial volume and the Neuroreader Index, were calculated. Radiologic interpretations of the MR imaging and pathology from subsequent resections were classified as either mesial temporal sclerosis or other, including normal findings. These measures of hippocampal volume were evaluated by receiver operating characteristic curves on the basis of pathologic confirmation of mesial temporal sclerosis in the resected temporal lobe. Sensitivity and specificity were calculated for each method and compared by means of the McNemar test using the optimal threshold as determined by the Youden J point.
RESULTS: Optimized thresholds of hippocampal percentage of a structural volume relative to total intracranial volume (<0.19%) and the Neuroreader Index (≤−3.8) were selected to optimize sensitivity and specificity (89%/71% and 89%/78%, respectively) for the identification of mesial temporal sclerosis in temporal lobe epilepsy compared with the initial clinical interpretation of the neuroradiologist (50% and 87%). Automated measures of hippocampal volume predicted mesial temporal sclerosis more accurately than radiologic interpretation (McNemar test, P < .0001).
CONCLUSIONS: Commercially available automated segmentation and volume analysis of the hippocampus accurately identifies mesial temporal sclerosis and performs significantly better than the interpretation of the radiologist.
ABBREVIATIONS:
- EEG
- electroencephalography
- MTS
- mesial temporal sclerosis
- NRI
- Neuroreader Index
- %Vol
- percentage of a structural volume relative to total intracranial volume
Mesial temporal lobe epilepsy is one of the most common forms of epilepsy.1 Temporal lobectomies for resection of an epileptogenic lesion are effective in reducing or even eliminating seizures and/or reducing the number of medications required for seizure management.2,3 Seizure semiology and scalp electroencephalography (EEG) remain critical for defining seizure onset. Concordant scalp EEG demonstrating a unilateral mesial temporal onset with unilateral mesial temporal sclerosis (MTS) predicts postresection seizure freedom in up to 78% of patients.4 In the absence of an identifiable abnormality on MR imaging, invasive monitoring with surgical placement of intracranial electrodes is often necessary to confirm the localization of seizures measured by scalp EEG. In these instances, nonlesional MR imaging and intracranial monitoring demonstrating a mesial temporal seizure focus predict postresection seizure freedom in approximately 76% of patients.5 The presence of a positive MR imaging finding with concordant scalp EEG may be equivalent to more invasive monitoring for the localization of seizure and concordant EEG and MR imaging demonstrating MTS are considered sufficient to proceed directly to surgical resection.6 Thus, sensitive identification of MTS is critical to identify patients who may benefit from a resection.
Radiologically, mesial temporal sclerosis is suggested by the recognition of volume loss and T2 signal hyperintensity. This subjective interpretation can be difficult, especially with underlying diffuse cerebral volume loss as is commonly present in elderly individuals or in individuals with prior brain injury from trauma or infection, for example. Furthermore, although radiographically defined MTS relies on the presence of T2 signal abnormality, it is well-established that pathologically defined lesional MTS is discovered in cases in which the radiographic findings are very subtle and resection is primarily directed by invasive monitoring and seizure semiology. Reliable, objective tools to quantify volume loss, accounting for age and sex, would help overcome interobserver variability and false-negative reporting, which can lead to delay or failure to identify an epileptogenic lesion potentially amenable to surgical intervention.
Neuroreader (Brainreader, Horsens, Denmark) calculates volumes and volumetric indices of specified brain regions, including hippocampal formations, on the basis of high-resolution T1-weighted MR imaging of the brain, using a variety of age- and sex-variable templates and normative data bases.7 MR imaging–based estimates of volume correlate with the interpretation of radiologists in a study of hippocampal volume loss in Alzheimer disease but perform better than radiologists when the clinical manifestations (and presumably the pathology) are more subtle.8 A different commercially available software platform, NeuroQuant (CorTech Labs, San Diego, California) was shown to be equivalent to the identification of hippocampal volume asymmetry by a radiologist.9 Manual estimation of hippocampal volumes failed to routinely identify a distinguishing threshold to identify abnormal hippocampi in epilepsy, and is time-prohibitive.10,11 The predictive value of imaging-based hippocampal volume estimation before epilepsy surgery has, however, been established using a variety of research-based automated techniques,12⇓⇓⇓⇓⇓–18 but not with a clinically applicable or standardized solution. Validation of a clinically integrated method is necessary for implementation into routine practice. We chose this software on the basis of its FDA-approved status, robust and automated registration, and availability of normative comparison adjusted for age and sex.
We propose that automated volumetric analysis of MR imaging using the Neuroreader will correctly identify MTS by detecting hippocampal volume loss. The purpose of this study was to assess the performance of automated MR imaging–based measures of hippocampal volume to identify MTS in patients with medically refractory temporal lobe epilepsy compared with the interpretation of a neuroradiologist and to establish normative values to validate the method for clinical application.
Materials and Methods
This study was approved by the University of Pittsburgh institutional review board. Informed consent was not required by the institutional review board given the retrospective nature of the study and the use of data acquired as part of an ongoing quality-assurance initiative.
Study Population
In this retrospective study, all subjects were adults (older than 18 years of age) who had undergone temporal lobectomy between 2011 and 2016 for treatment of epilepsy following consensus recommendation of the operation by the University of Pittsburgh Medical Center Multidisciplinary Epilepsy Patient Management team. Seventy-five consecutive patients were identified during this time period. Preoperative evaluations included a detailed history, physical examination performed by a neurologist specialized in epilepsy care, scalp EEG, and 3T epilepsy protocol MR imaging in all patients. Magnetoencephalography, ictal single-photon emission CT, positron-emission tomography, and/or evaluation in an inpatient epilepsy-monitoring unit, including intracranial EEG monitoring, were added when indicated for seizure localization. Review of the clinical record was performed to obtain age, sex, and side of the operation. Four subjects were excluded for lack of imaging suitable for volumetric analysis.
Pathologic Reference Standard
Pathology reports of the resections were reviewed by a board-certified neuropathologist (C.A.W.) with >25 years of experience, and findings were classified as normal, MTS, or other (non-MTS lesion, including low-grade tumors, cavernous malformations, and dysplasia), with the reviewer blinded to the volumetric results. MTS was identified by the following criterion: discrete hippocampal damage consisting of neuronal loss and astrocytosis predominantly in the CA1 region but potentially including the end folium (CA4) and subiculum.
MR Imaging and Analysis
All included subjects had at least 1 clinical epilepsy protocol MR imaging performed, including an isotropic T1-weighted sequence acquired in either axial or coronal planes. The specific parameters varied slightly during 6 years (Discovery MR750w or Optima MR450w, BRAVO [GE Healthcare, Milwaukee, Wisconsin]: 1.5T–3T 3D fast spoiled gradient (FSPGR) BRAVO [GE Healthcare]; TR, 9.8–10.9 ms; TE, 4.1–4.5 ms; TI, 450 ms; flip angle, 8°–13°, NEX, 1; FOV, 250; matrix, 320 × 256–350 × 288; 1.2-mm thickness; 0.6- to 1.2-mm spacing; Tim Trio [Siemens, Erlangen, Germany]: 3T MPRAGE; TR, 2110 ms; TE, 2.6 ms; TI, 1100 ms; flip angle, 8°; NEX, 1; FOV, 75%; matrix, 256 × 192; 1.5-mm thick). All volumetric series were inspected for artifacts or large structural lesions that would interfere with the segmentation process. The volumetric sequence was then analyzed using Neuroreader.7 Output included measures of total intracranial and hippocampal volumes with calculation of the relative hippocampal volume as a percentage of total intracranial volume (%Vol) and the Neuroreader Index (NRI), a nonparametric index of size compared with a normative data base accounting for age and sex. If a subject had both axially and coronally acquired volumetric studies before the operation, both were analyzed and the resulting values were averaged.
Radiology reports from the relevant imaging acquired before the operation were reviewed and classified as having normal findings, MTS, or other (non-MTS lesion or equivocal abnormalities not meeting the radiologic criteria for MTS as defined by volume loss and T2 signal hyperintensity).19 All MR imaging studies were interpreted by 1 of 12 Certificate of Added Qualification–certified academic neuroradiologists as part of routine clinical practice, to reflect real-world comparison of the software with radiologic interpretation. The volumetric T1 sequence was used along with all other routine clinical pulse sequences in the radiologic assessment. The interpretation of a radiologist was considered positive for MTS if MTS was offered as the most likely diagnosis in the radiology report.
Statistical Analysis
The reference standard was pathologic interpretation of either MTS or non-MTS (including other pathologies or normal findings). Receiver operating characteristic curves were generated for the hippocampal NRI and %Vol for both sides of the operation and contralateral brain. Area under the curve and Youden J point were calculated for each measure. Sensitivity, specificity, and accuracy based on a threshold value determined by the Youden J point were then compared with the radiologic interpretation using the nonparametric McNemar test. All statistics were performed with SPSS Statistics for Windows, Version 24.0 (IBM, Armonk, New York).
Results
Seventy-five subjects underwent temporal lobectomy during the study period. Four subjects were excluded for inadequate imaging. Twenty-six subjects had pathologic findings consistent with MTS, while 45 subjects had either normal findings or demonstrated other pathology. Basic demographics, incidences of non-MTS pathologies, and other features comparing the 2 groups are presented in Table 1. There were no significant differences between relevant demographics between the groups classified as MTS versus non-MTS. Minor variations in the T1 volumetric imaging acquisition parameters did not differ between the 2 groups.
Demographics and details of the operation, pathology, and imaging evaluation
Receiver Operating Characteristic Curves for Identification of MTS
Hippocampal %Vol classified the presence or absence of ipsilateral MTS with an area under the curve of 0.818 (P < 0.001); contralateral %Vol was not significantly correlated with MTS status (area under the curve = 0.602, P = .151). The NRI of the hippocampus classified the presence or absence of ipsilateral MTS with an area under the curve = 0.800 (P < 0.001); the contralateral NRI was not significantly correlated with MTS status (area under the curve = 0.560, P = .399). Receiver operating characteristic curves are demonstrated in Fig 1.
Receiver operating characteristic curve for mesial temporal sclerosis. The volume of the hippocampi corrected for total intracranial volume and the Neuroreader Index, a nonparametric comparative statistic derived from age- and sex-adjusted normative data, were calculated and evaluated for their ability to classify patients with pathology-proved mesial temporal sclerosis. Receiver operating characteristic curves from ipsilateral and contralateral hippocampal volume measures relative to the subsequent side of surgery are presented. Note that contralateral volumes appear skewed away from the 45° diagonal.
Comparison of Automated Segmentation Measures and Radiology Reporting
The Youden J point identified ideal thresholds for hippocampal NRI ≤−3.80 and %Vol <0.19% (Table 2). Pair-wise comparisons with the McNemar nonparametric test demonstrated that automated volumetric techniques performed significantly better than the interpretation of a radiologist for classification of MTS; there was no significant difference between the 2 automated volumetric measures, NRI or %Vol. Figure 2 presents 3 examples of left mesial temporal sclerosis, the corresponding clinical read, and the results from Neuroreader volumetric analysis.
Comparison of Neuroreader measures with radiologist interpretation for detection of MTS
Volumetric analysis. A–C, Three patients with left mesial temporal sclerosis of varying conspicuity on MR imaging. Clinical reads and volumetric measures of the abnormal left and normal right hippocampal formations are presented below the images. D, Representative registration and segmentation. Hippocampal volumes are denoted by bright green (right) and dark blue (left) in a patient with left mesial temporal sclerosis.
Discussion
Objective identification of hippocampal volume loss is associated with improved outcome from an epilepsy operation.12 We used a commercially available software platform to demonstrate that automated volumetric measurements of preoperative brain MR imaging accurately predict pathologic evidence of MTS in TLE. Because MTS is one of the most common lesions identified in this population before an operation, automated methods may be useful adjuncts to help radiologists more confidently identify lesions potentially amenable to surgical resection. Identification of epileptogenic lesions such as MTS can alter the treatment course for individuals with medically intractable epilepsy, prompting a more in-depth evaluation at a center dedicated to epilepsy surgery, informing a plan to confirm a seizure origin, and potentially avoiding more aggressive invasive diagnostic tests before the operation.6 Indeed, intracranial seizure monitoring is associated with a significant risk because up to 9% of patients will experience surgical complications associated with their procedure, such as hemorrhage, infection, or CSF leak.20
Hippocampal volume as a percentage of total intracranial volume and the NRI, a proprietary nonparametric index derived from comparison with normative databases accounting for age and sex, accurately identified MTS on the basis of preoperative imaging. Of note, the NRI did not perform significantly better than the %Vol measure, suggesting that the effect of age and sex did not substantially confound predictions when using this software in this population. Alternatively, the registration and segmentation strategy used by the software, in which best-fit templates are identified before registration, may help to pre-emptively account for age- and sex-related volume differences. A larger analysis of older patients would be helpful to further understand the specific contribution of accounting for these factors. Others have suggested using asymmetry indices. We found these measures to be inferior to individual volumes, potentially due to contralateral temporal lobe volume loss that has been described with MTS21; indeed, Fig 1 demonstrates that the contralateral temporal lobe volume was slightly skewed away from the 45° angle line on the receiver operating characteristic curve, which would potentially degrade the sensitivity of asymmetry indices.
T2 signal intensity in the hippocampal formation is a traditional hallmark of mesial temporal sclerosis, but it was not specifically taken into consideration regarding the volumetric-based classification of disease. Although this approach goes against the traditional interpretation of MTS, we believe that overt MTS characterized by asymmetric volume loss and T2 hyperintensity is usually evident to the interpreting neuroradiologist and the primary added value of volumetric analysis would be to identify more subtle manifestations of MTS, which may not have substantial signal abnormality, thereby increasing sensitivity. Indeed, many of the discordant analyses between the radiologist and the automated volumetric analysis were in the context of false-negative radiology reporting. Conversely, non-MTS pathology such as tumors or focal cortical dysplasia is likely more evident to the radiologist, and false-positives potentially generated by automated segmentation would have little impact.
The incidence of MTS in this cohort (26/71, 37%) is not dissimilar to the report of Cohen-Gadol et al22 of 28% in a cohort of 399 patients without lesions, and the report of Tassi et al23 of approximately 46% (mixed and isolated hippocampal sclerosis). Aggressive intracranial EEG monitoring using depth electrodes and grids has become more common, identifying more subtle, lateralizing temporal lobe epilepsy than before. Furthermore, there is growing recognition of mesial temporal lobe epilepsy amenable to resection, but without specific histopathology.24 The number of cases with normal pathologic findings in this cohort may, in part, reflect nonsclerotic temporal lobe epilepsy, while other normal pathologic findings may simply reflect a damaged or incomplete specimen that pathology could not reliably interpret as anything but normal. Last, there is likely some degree of institutional bias to the overall makeup of the patients included in this cohort, given that different subspecialty services such as oncologic surgery may be involved with resection of more obvious tumor-related seizure onset, and thus were not captured in this cohort.
Inclusion of all temporal lobe resections meant that a small proportion of the studied individuals had mesial temporal involvement of tumors, cavernous malformations, or dysplasia; most important, these were not generally misclassified by volumetric analysis (2 of 18). In one of these cases, an oligodendroglioma was correctly identified by the radiologist irrespective of the volume measure. The single case that was classified incorrectly by both the radiologist and Neuroreader as MTS was diagnosed as a cortical dysplasia by pathology. Thus, it is in this mixed setting that we suggest that the software would best complement interpretation of the radiologist and contribute to greater value of imaging interpretation.
Volumetric analysis of brain regions, particularly the hippocampus, is not new. From manual, semimanual, automated, and machine learning algorithms, the value of volumetric measures of regional brain structures has been recognized in many studies, including in the fields of epilepsy, dementia, and traumatic brain injury.11,16⇓–18,25,26 However, many of these approaches use software or processes that are not FDA-approved or require a prohibitive amount of time to complete, rendering them difficult to incorporate into routine clinical practice or replicate at different sites. The relative merits of accuracy for each of these methods could be debated; however, there is clear value in the consistency and availability of a methodology to assist in traditional interpretation, and these data validate the use of volumetric analysis in this clinical context and establish normative values for clinical implementation. Future studies to determine which commercial applications perform best or most reliably for specific populations and indications would provide helpful guidance.
A limitation of this study was the retrospective nature of analysis, including only those individuals who had undergone temporal lobectomy. Future investigation requiring greater numbers of participants will better address the role of prospective volumetric analysis in epilepsy evaluations, including patients with bitemporal seizure onset, and the contribution to prognostication of surgical outcome. Our study did not directly assess the benefit of prospective interpretation when using the volumetric software. MR imaging interpretation was also made by a group of radiologists rather than a single reader, likely increasing variability but reflecting typical real-world workflow. A comprehensive review of the MRIs was not performed because we wanted to demonstrate the contribution of volumetric analysis to the true clinical read. Last, we did not consider the side of the operation or handedness of the individuals in developing our thresholds, assuming that a single-volume threshold could be applied to both the left and the right hippocampal formations in the general population, which is predominantly right-handed. Previous studies have shown that there are no significant differences between the left and right hippocampal volumes in right-handed individuals,27 though a small-but-statistically significant difference was reported for left-handed individuals. A variety of scanners and volumetric protocols was included in the analysis, but without bias toward one group or the other. The software is built to robustly analyze data across platforms for consistent clinical application.
Future investigation should focus on directly comparing the performance of the volumetric software alone with the performance of a radiologist using the volumetric software in a prospective manner. We propose that the interpretation of a radiologist would be complemented by automated hippocampal volumetric measures for optimal diagnostic accuracy. While the radiologist will likely best identify obvious lesions such as advanced MTS, a cavernous malformation, cortical dysplasia, or tumor, accuracy for identification of subtle mesial temporal sclerosis will be increased given the improved sensitivity provided by the volumetric analysis. We predict that radiologists will experience increased confidence in their interpretations when using this software and describing normal study findings or subtle manifestations of mesial temporal sclerosis based on the thresholds and parameters presented above. The slightly lower specificity of the volumetric analysis is acceptable and may, in some way, be mitigated by the radiologists. In addition, the decision to proceed with an operation takes into account many other data, including scalp and intracranial EEG analysis, seizure semiology, and neurocognitive testing as well as other imaging modalities such as PET, ictal/interictal SPECT, and magnetoencephalography.
Conclusions
Automated segmentation and volumetric analysis using Neuroreader perform significantly better than subjective evaluation by a radiologist for preoperative identification of MTS in patients with mesial temporal lobe epilepsy. On the basis of this dataset, we propose that hippocampal %Vol of ≤0.19% or NRI of ≤−3.8, should be considered strongly suggestive of MTS.
Acknowledgments
The commercial product Neuroreader was made available to our institution at no charge.
References
- Received September 11, 2018.
- Accepted after revision December 23, 2018.
- © 2019 by American Journal of Neuroradiology
