Abstract
BACKGROUND AND PURPOSE: MR imaging is useful for the detection and/or confirmation of optic neuritis. The objective of this study was to determine whether a postprocessing algorithm selectively increases the contrast-to-noise ratio of abnormal optic nerves in optic neuritis, facilitating this diagnosis on MR imaging.
MATERIALS AND METHODS: In this retrospective case-control study, coronal FLAIR images and coronal contrast-enhanced T1WI from 44 patients (31 eyes with clinically confirmed optic neuritis and 28 control eyes) underwent processing using a proprietary postprocessing algorithm designed to detect and visually highlight regions of contiguous increases in signal intensity by increasing the signal intensities of regions that exceed a predetermined threshold. For quantitative evaluation of the effect on image processing, the contrast-to-noise ratio of equivalent ROIs and the contrast-to-noise ratio between optic nerves and normal-appearing white matter were measured on baseline and processed images. The effect of image-processing on diagnostic performance was evaluated by masked reviews of baseline and processed images by 6 readers with varying experience levels.
RESULTS: In abnormal nerves, processing resulted in an increase in the median contrast-to-noise ratio from 17.8 to 85.0 (P < .001) on FLAIR and from 19.4 to 93.7 (P < .001) on contrast-enhanced images. The contrast-to-noise ratio for control optic nerves was not affected by processing (P = 0.13). Image processing had a beneficial effect on radiologists' diagnostic performance, with an improvement in sensitivities for 5/6 readers and relatively unchanged specificities. Interobserver agreement improved following processing.
CONCLUSIONS: Processing resulted in a selective increase in the contrast-to-noise ratio for diseased nerves and corresponding improvement in the detection of optic neuritis on MR imaging by radiologists.
ABBREVIATIONS:
- CIE
- correlative image enhancement
- CNR
- contrast-to-noise ratio
Optic neuritis is an acute-to-subacute, demyelinating optic neuropathy, typically occurring in young women1,2 in association with multiple sclerosis.1,3⇓–5 It presents with changes in visual acuity, eye pain, a relative afferent pupillary defect, and sometimes optic disc swelling.1,2,5⇓⇓⇓⇓–10 While the diagnosis can often be made clinically, misdiagnosis is common, reported in almost 60% of referrals in 1 study.11 MR imaging can help solidify the clinical diagnosis, especially in atypical cases.7 In addition, with increasing recognition of subclinical optic neuropathy in patients with multiple sclerosis,12 the use of MR imaging to detect optic neuritis–related nerve abnormalities might become even more important.
Challenges of MR imaging of the optic nerve include its small size, heterogeneity of the surrounding orbital tissue, and artifacts related to eye movements or the presence of surrounding orbital fat.13⇓⇓⇓⇓⇓–19 The optic nerve signal is best compared with normal white matter in the brain.20 However, the lack of proximity of the optic nerve to the white matter and different intensities of surrounding structures make this comparison difficult. Despite these challenges, some previous studies have reported sensitivities in the detection of optic neuritis by neuroradiologists ranging from 75.7% on noncontrast MR imaging21 to 94% on contrast-enhanced MR imaging.22 Improving the conspicuity of optic nerve abnormalities on MR imaging may help improve the detection of optic neuritis, particularly to minimize the need for intravenous contrast, detect subclinical disease, or improve the sensitivity in communities where scans are interpreted by general radiologists.
Correlative image enhancement (CIE) is a proprietary image postprocessing algorithm based on the correlative properties of contiguous pixels designed to increase the conspicuity of details of interest on existing CT and MR imaging scans.23 It has been previously shown to improve the contrast-to-noise ratio (CNR) between diseased and normal brain tissue in brain infarction and mesial temporal sclerosis.24,25 The objective of this study was to test whether image processing of existing MR imaging scans with CIE can be used to improve detection of optic neuritis by selectively improving the CNR of diseased optic nerves.
Materials and Methods
Institutional review board approval was obtained.
Cases and Controls
This was a retrospective study of 44 patients (31 women, 13 men; mean age, 43.0 ± 14.2 years) who had undergone orbital MR imaging including coronal fat-saturated FLAIR and coronal fat-saturated contrast-enhanced T1-weighted imaging (Table 1) for clinical suspicion of optic neuritis on 1.5T MR imaging scanners at our institution between January 2014 and October 2016. Of these, 31 eyes diagnosed with optic neuritis based on clinical evaluation were included as cases and 28 eyes found not to have any clinical evidence for optic neuritis or optic neuropathy served as controls. Twenty-nine asymptomatic eyes in patients with unilateral optic neuritis were excluded because they could be affected by subclinical disease. Coronal FLAIR images were available for 28 cases and 28 control eyes. Coronal contrast-enhanced images were available for 28 cases and 18 control eyes.
Technical parameters for the coronal FLAIR and coronal contrast-enhanced T1WI used in the study
Image Processing
For each patient, the entire set of coronal FLAIR and contrast-enhanced MR images in a DICOM format was de-identified and transferred to an OsiriX Lite workstation (https://www.macupdate.com/app/mac/14362/osirix-lite). A masked investigator processed these images using the CIE algorithm with a custom-built plug-in. The algorithm required manual placement of an ROI within normal-appearing white matter. If the signal intensity of a contiguous set of pixels exceeded the intensity of the normal appearing white matter by a predetermined threshold, the algorithm markedly increased the signal intensity of those pixels. The processed images were saved as a new DICOM file.
Contrast-to-Noise Ratio Calculations
Using an ROI tool inbuilt in OsiriX Lite, we measured the signal intensities of each optic nerve (SIon), and the ipsilateral normal-appearing white matter (SIwm) for both baseline and processed images at equivalent locations (Fig 1). For optic nerves with obvious signal abnormality, the ROI was placed in the region of abnormal signal. Otherwise, the optic nerve was sampled in its retrobulbar portion on an image that allowed its best visualization free from partial volume averaging effects. The SD of the signal intensity of air in the image was similarly recorded as a measure of noise. These measurements were then used to calculate the CNR for both FLAIR and contrast-enhanced images using the following formula: CNR = (SIon − SIwm)/Noise.
Measurement of the CNR between the optic nerve and ipsilateral white matter. Baseline (A) and processed (B) coronal FLAIR images show equivalent placement of ROIs to measure signal intensities of the optic nerve (a) and the ipsilateral white matter (b). The SD of the intensity in air around the head (c, not shown) was used as a measure of noise. CNR was defined as (a − b) / c. In this patient with left optic neuritis, processing resulted in an increase of the left optic nerve signal intensity from 355 at baseline to 754 after processing. Other measurements were unaffected by processing.
Masked Image Review
Six masked readers, including 2 radiology residents, 2 neuroradiology fellows, and 2 attending neuroradiologists, reviewed baseline and processed images separately in OsiriX Lite. Readers rated optic nerve signal intensity (on FLAIR images) and the presence of contrast enhancement (on contrast-enhanced images) within the optic nerve on a 5-point Likert scale, ranging from 1 (definitely normal) to 5 (definitely abnormal). For subsequent analysis to calculate the diagnostic performance of each reader, ratings of 4 (probably abnormal) or 5 were taken as an abnormal test result, while ratings of ≤3 (possibly normal) were taken as a normal test result. The baseline-versus-processed status of the images was not revealed to the readers. However, readers were asked to comment on whether they saw markedly high signal in the optic nerves and whether this influenced their interpretation of the image.
Statistical Analysis
Descriptive statistics were used to summarize the imaging metrics. Continuous variables were summarized using means and SDs and median (25th percentile, 75th percentile). A paired t test or a signed rank test was used to evaluate differences between baseline and processed images in the optic nerve and control groups. The median of the ratings across all readers was identified for both baseline and processed images, and the difference in the median (processed-baseline image) was computed and compared using a paired t test or signed rank test, to assess improvement in the confidence of categorizing the optic nerve using signal intensity and enhancement. Sensitivity, specificity, positive predictive value, negative predictive value, and diagnostic accuracy were calculated for each reader and were also averaged for all readers. Improvements in diagnostic performances in cases versus controls were compared between the baseline and processed image within a group using a McNemar test.
Interobserver reliability was computed using a model-based measure of agreement, which is robust to the underlying disease prevalence,26 and 95% confidence intervals were obtained from 1000 bootstrapped samples. The pair-wise κ for each pair or readers with similar experience levels (the pair of radiology residents, the pair of neuroradiology fellows, etc.) was also compared using the Cohen and weighted κ. A P value of .05 was considered statistically significant. Statistical analyses were conducted in SAS, Version 9.4 (SAS Institute, Cary, North Carolina) and R statistical and computing software (Version 1.0.136; http://www.r-project.org/).
Results
Effect of Processing on the CNR
For FLAIR images of eyes with optic neuritis, processing resulted in a statistically significant increase in the median (25th to 75th) CNR from 17.8 (8.9–32.3) to 85.0 (64.8–127.2) (P < .001), with an increased CNR seen in 27/28 eyes (Figs 2A and 3). CNRs for control eyes were not significantly affected (P = .625), with a CNR increase noted only in 1/28 control eyes (Fig 2B). For contrast-enhanced images of eyes with optic neuritis, processing resulted in an increase in the median (25th to 75th) CNR from 19.35 (4.0–32.8) to 93.7 (47.2–150.6) (P < .001), with an increased CNR seen in 24/28 eyes (Figs 2C and 3). The CNR for control eyes was not significantly affected (P = 0.13), with the CNR increase noted in only 1/18 control eyes (Fig 2D).
The effect of image processing on the CNR for optic nerves with and without optic neuritis. Scatterplots show the contrast-to-noise ratio of the optic nerve for the baseline images and the postprocessed images in both eyes with optic neuritis and control eyes.
Examples highlighting the effect of processing on FLAIR and contrast-enhanced T1WI in patients with optic neuritis. A, Baseline FLAIR image of a patient with left-sided optic neuritis. B, Processed version of the same FLAIR image (from A) of a patient with left-sided optic neuritis. C, Baseline FLAIR image of a patient with right-sided optic neuritis. D, Processed version of the same FLAIR image (from C) of a patient with right-sided optic neuritis. E, Baseline contrast-enhanced image of a patient with right-sided optic neuritis. F, Processed version of the same contrast-enhanced image (from E) of a patient with right-sided optic neuritis. G, Baseline contrast-enhanced image of a patient with left-sided optic neuritis. H, Processed version of the same contrast-enhanced image (from G) of a patient with left-sided optic neuritis.
Effect of Processing on Confidence Ratings
For FLAIR images, there was a statistically significant increase in the median confidence rating for processed images of eyes with optic neuritis (P < .001). For control eyes, processing did not result in any significant change in the median confidence rating on FLAIR images (P = 0.08). Processing did not result in any significant change (P = .54) in average confidence ratings for eyes with optic neuritis for contrast-enhanced images.
Effect of Processing on Diagnostic Performance of Readers
For FLAIR images, the average sensitivity for the detection of optic neuritis for all readers increased from 71.4% ± 13.6% on baseline images to 94.6% ± 2.0% on processed images. Sensitivity improved for 5 readers and remained unaffected for the sixth (Table 2). Accordingly, the McNemar test revealed a significant increase in the number of additional detections of optic neuritis by 5/6 readers (P ranging from .01 to <.01 for individual readers). The average specificity did not change with image processing (Table 2), with individual specificities unaffected for 3 readers, decreased for 2 readers, and improved for 1 reader after processing (Table 2). The positive predictive value was relatively unaffected (92.3% at baseline and 92.0% after processing), but the negative predictive value improved from 77.8% ± 9% to 94.5% ± 1.9% following processing. The average accuracy for the detection of optic neuritis increased from 0.83 ± 0.03 for baseline images to 0.93 ± 0.02 for processed images.
Effect of image processing on sensitivity and specificity for detection of optic neuritis on FLAIR and contrast-enhanced T1WI by 6 masked readers including 2 radiology residents (readers 1, 2), 2 neuroradiology fellows (readers 3, 4), and 2 attending neuroradiologists (readers 5, 6)
For contrast-enhanced images, the sensitivity for the detection of enhancement in eyes with optic neuritis increased from 72.0% ± 10.5% to 78.6% ± 5.1% after processing, with improved sensitivities in 4/6 readers (Table 2). The McNemar test indicated a significant increase in the detection of enhancement for 2 readers (P = .025). The average specificity did not significantly change with image processing (Table 2). The positive predictive value changed from 96.4 ± 6.7 to 98.6 ± 2.1, and the negative predictive value, from 69.5 ± 7.3 to 74.9 ± 4.1. The average accuracy for the detection of optic neuritis increased from 0.81 ± 0.04 to 0.86 ± 0.02 with processing.
Effect of Processing on Interobserver Agreement
Interobserver agreement in the categorization of optic nerves into abnormal or normal categories on FLAIR images improved following processing (Fleiss κ = 0.59 for baseline images and 0.91 for processed images). The pair-wise κ for readers with similar experience improved from 0.48 for the radiology residents, 0.46 for the neuroradiology fellows, and 0.55 for the neuroradiology attendings, and for baseline images to 0.76, 0.71, and 0.82, respectively, for processed images.
Interobserver agreement in the categorization of optic nerves into enhancing or nonenhancing categories improved following processing (Fleiss κ improving from 0.63 for baseline images to 0.83 for processed images). The pair-wise κ for readers with similar experience improved from 0.85 for the radiology residents, 0.62 for the neuroradiology fellows, and 0.78 for the neuroradiology attendings, and for baseline images to 0.90, 0.69, and 0.88, respectively, for processed images.
Discussion
Our results indicate that this algorithm can improve the sensitivity for the detection of optic neuritis on MR imaging by readers with varied training and experience levels, without a detrimental effect on specificity. Our quantitative assessment validated this benefit by demonstrating a selectively increased CNR of diseased optic nerves while affecting only a small proportion of controls. Eyes affected by optic neuritis can be expected to have higher average signal intensity on T2-weighted images such as FLAIR.27,28 By exaggerating the signal differences between the diseased optic nerve and the normal white matter, this algorithm makes these underlying differences more easily recognizable.
McKinney et al21 reported 75.7%–77.3% sensitivity and 90.5%–93.5% specificity in the detection of optic neuritis on FLAIR images. While the average sensitivity (71.4%) and specificity (94.6%) of readers for baseline FLAIR images in our study was comparable, sensitivity achieved after processing (94.6%) was substantially higher. This improved sensitivity on noncontrast MR imaging may help obviate a contrast-enhanced study in the detection of optic neuritis, thereby minimizing cost and scanning time.
Previous studies have suggested that contrast-enhanced T1-weighted images may be more sensitive than noncontrast sequences, with reported sensitivities of 89.5%21 and 94%.22 In our study, the average baseline sensitivity for the detection of acute optic neuritis on contrast-enhanced sequences was lower (72%), even for expert neuroradiologists (75%, readers 5 and 6; Table 2). The reason for this difference is unclear. While it may result from underlying technical differences, it is possible that it simply reflects a different patient population. Additionally, image processing was also helpful in improving the sensitivity for contrast-enhanced images.
Many previous studies have assessed different pulse sequences, including fat-suppression techniques (STIR and spectral presaturation with inversion recovery), fluid-suppression techniques (FLAIR), artifact-minimizing techniques (fast spin-echo with fat suppression), combined fat- and fluid-suppression techniques (spectral presaturation with inversion recovery–FLAIR and XETA [Cube; GE Healthcare, Milwaukee, Wisconsin] FLAIR), and diffusion-weighted imaging in the detection of optic neuritis.13⇓–15,29 It is not feasible for us to test the effect of postprocessing on images acquired by all these different techniques.
In prior studies, interobserver reliability in identifying optic nerve abnormalities has varied depending on the type of sequences being studied, with the κ reported from 0.60 to 0.96.18,27 For readers with similar experience, the baseline κ in our study ranged from 0.60 to 0.85. Postprocessing improved the interobserver reliability, with κ for postprocessed FLAIR images ranging from 0.69 to 0.90. This result may prove beneficial in settings where the images are interpreted by less experienced readers. An improvement in the Fleiss κ also indicated improved agreement among readers of varying experience. Interobserver agreement for the detection of contrast enhancement remained low for controls, even after processing. This outcome is possibly due to the exaggeration of high signal in vasculature in close proximity to the optic nerves that could be misinterpreted as optic nerve enhancement by some readers.
While our study was restricted to acute optic neuritis, it is possible that improved detection of underlying signal alterations in optic nerve signal intensity may make this processing beneficial for the detection of other causes of optic neuropathy, including patients with remote optic neuritis. In patients being assessed for multiple sclerosis, such image processing may help uncover underlying optic nerve abnormalities that are otherwise not captured on routine clinical assessment or routine MR imaging.30,31
By using patients who underwent imaging due to concern for optic neuritis but were found to have no clinical evidence of optic neuritis and by incorporating a large number of masked reviewers with a range of training experience, our study tries to overcome some of the limitations of prior studies, which include restriction of controls to the contralateral eye in symptomatic patients and only 1 or 2 subspecialist reviewers.13⇓–15,18,21,22,27,32
Some limitations of our study are worth mentioning. Optic neuritis is a clinical diagnosis, and MR imaging is not always required for the diagnosis. Thus, there may be a selection bias toward atypical presentations in the patients who underwent MR imaging. Patients with other types of optic neuropathy were excluded, preventing evaluation of whether the algorithm can distinguish optic neuritis from other forms of optic neuropathy; this exclusion may affect the generalizability of the specificity results. The control patients were those who presented with visual or ocular symptoms but were found not to have optic neuropathies; there were no asymptomatic controls. Finally, this study was limited to coronal FLAIR sequences obtained at a single institution. It remains to be seen whether the incremental benefit seen in our study would hold for scans performed on higher field strength scanners with potentially higher baseline sensitivity.
Conclusions
The described postprocessing algorithm increased the CNR between diseased optic nerves and normal white matter for eyes with optic neuritis on both FLAIR and contrast-enhanced images. This result translated into an improved sensitivity, negative predictive value, diagnostic accuracy, and interobserver reliability for the detection of optic neuritis by readers with varied training and experience levels.
Acknowledgments
We thank the following physicians for participating as masked observers: Devin Vaswani, MD; Ben Strand, MD; Albert Sohn, MD; Jason Carmichael, MD; Hilary Orlowski, MD.
Footnotes
Disclosures: Aseem Sharma—RELATED: Other: Correlative Enhancement LLC, Comments: The algorithms used in this study were coinvented by me. I hold the intellectual property rights to these patented algorithms. I have founded a company (Correlative Enhancement LLC) with the aim of future commercialization of this intellectual property. I am the sole proprietor of the company, and since the inception of the company until now (including the time during which I processed the images for this study), this company has not received funding from any external source. While I used the algorithms to process the images for this study, I did not participate in image review, image analysis, or the subsequent statistical analysis; UNRELATED: Consultancy: Biomedical Systems, Comments: As a consultant, I serve as an independent reviewer for imaging studies performed for research by third parties; Patents (Planned, Pending or Issued): “Method for Medical Image Analysis and Manipulation,” US patent No. 9,846,937, December 19, 2017; Stock/Stock Options: GE Healthcare, Comments: I own publicly traded GE Healthcare stocks (current value of $10–15,000) in a company that makes imaging equipment among other things. Amber Salter—UNRELATED: Consultancy: Circulation Cardiovascular Imaging journal.
Data previously presented at: North American Neuro-Ophthalmology Society Annual Meeting, March 3–8, 2018; Waikoloa Village, Hawaii; and the American Academy of Neurology Annual Meeting, April 21–27, 2018; Los Angeles, California.
REFERENCES
- Received November 29, 2018.
- Accepted after revision March 31, 2019.
- © 2019 by American Journal of Neuroradiology
