Detection of Volume-Changing Metastatic Brain Tumors on Longitudinal MRI Using a Semiautomated Algorithm Based on the Jacobian Operator Field
=============================================================================================================================================

* O. Shearkhani
* A. Khademi
* A. Eilaghi
* S.-P. Hojjat
* S.P. Symons
* C. Heyn
* M. Machnowska
* A. Chan
* A. Sahgal
* P.J. Maralani

## Abstract

**BACKGROUND AND PURPOSE:** Accurate follow-up of metastatic brain tumors has important implications for patient prognosis and management. The aim of this study was to develop and evaluate the accuracy of a semiautomated algorithm in detecting growing or shrinking metastatic brain tumors on longitudinal brain MRIs.

**MATERIALS AND METHODS:** We used 50 pairs of successive MR imaging datasets, 30 on 1.5T and 20 on 3T, containing contrast-enhanced 3D T1-weighted sequences. These yielded 150 growing or shrinking metastatic brain tumors. To detect them, we completed 2 major steps: 1) spatial normalization and calculation of the Jacobian operator field to quantify changes between scans, and 2) metastatic brain tumor candidate segmentation and detection of volume-changing metastatic brain tumors with the Jacobian operator field. Receiver operating characteristic analysis was used to assess the detection accuracy of the algorithm, and it was verified with jackknife resampling. The reference standard was based on detections by a neuroradiologist.

**RESULTS:** The areas under the receiver operating characteristic curves were 0.925 for 1.5T and 0.965 for 3T. Furthermore, at its optimal performance, the algorithm achieved a sensitivity of 85.1% and 92.1% and specificity of 86.7% and 91.3% for 1.5T and 3T, respectively. Vessels were responsible for most false-positives. Newly developed or resolved metastatic brain tumors were a major source of false-negatives.

**CONCLUSIONS:** The proposed algorithm could detect volume-changing metastatic brain tumors on longitudinal brain MRIs with statistically high accuracy, demonstrating its potential as a computer-aided change-detection tool for complementing the performance of radiologists, decreasing inter- and intraobserver variability, and improving efficacy.

## ABBREVIATIONS:

AUC
:   area under the curve
3D-T1-Gad
:   contrast-enhanced 3D T1-weighted
ΔMBT
:   volume-changing MBT
ΔMBTos
:   newly developed or resolved MBT
ΔMBTts
:   changing MBT present on both baseline and follow-up scans
FPR
:   false-positive rate
JOF
:   Jacobian operator field
MBT
:   metastatic brain tumor
ROC
:   receiver operating characteristic
VCR
:   volume change ratio

Metastatic brain tumors (MBTs) occur in 24%–45% of patients diagnosed with primary cancers outside the brain.1 Accurate assessment of MBTs on follow-up imaging is critical for better prognosis and selecting the most appropriate treatment such as chemotherapy, surgery, and radiation therapy or a combination of the aforementioned.2⇓–4 This is becoming more important with increasing use of stereotactic radiosurgery.5

Contrast-enhanced 3D T1-weighted (3D-T1-Gad) MR imaging is commonly used for detection and follow-up of MBTs and is the sequence of choice for stereotactic radiosurgery planning of MBTs.5,6 During follow-up of MBTs, longitudinal volumetric imaging is performed every 2–3 months.5 This results in a large amount of data to process and a demanding workload for radiologists.7 Moreover, the inherent limitations of viewing scans section by section, changes in head position from one scan to another, and user subjectivity result in the potential for increased inter- and intraobserver variability in both detection and volume assessment, especially with small MBTs or subtle volume changes.8

Although several studies have investigated the efficiency of computer-aided detection techniques in MBTs on a single MR scan,9⇓⇓⇓⇓–14 the literature is limited in studies evaluating the efficacy of computer algorithms in the follow-up of MBTs15 and detection of volume-changing MBTs (ΔMBTs) as an indicator of treatment response. Tracking volumetric changes is of high clinical value, and implementing computer-aided techniques in the follow-up of MBTs can improve diagnostic accuracy and efficiency16⇓⇓–19 and complement current single-scan detection algorithms.

While computer-aided detection tools are limited for MBTs, automated change-detection techniques in MS are an active area of research and development.20 Like MBTs, in MS, quantitative analyses are important for assessing disease progression,21 activity,22 and treatment evaluation.23 The literature on automatic change detection of MS lesions broadly divides existing approaches into deformation-based (analysis of deformation fields resulting from nonrigid registration of scans) and intensity-based (voxel-to-voxel comparison between scans) methods and suggests a potentially strong role for deformation field-based and combined techniques.20,23 Furthermore, techniques based on the deformation field have been proposed as promising in detecting structural changes on longitudinal brain MR imaging24⇓⇓–27 and encompass techniques based on vector displacement fields such as those centered around divergence28 and Jacobian operator fields (JOFs).29

To detect ΔMBTs on longitudinal brain MRIs, a semiautomated algorithm based on the JOF is proposed here. The JOF has certain advantages compared with other methods because it can be used independently to detect local volume changes28 compared with divergence techniques30 and is computationally less expensive than other deformation field morphometry approaches.20

In conjunction with other image-processing methods, the JOF is used in this study to identify MBT candidates that have changed in size, independent of changes in intensity or contrast compared with surrounding structures. We hypothesized that our algorithm can detect ΔMBTs with statistically high accuracy compared with a reference standard, defined here as detection by a board-certified neuroradiologist with 5 years of experience.

## Materials and Methods

### Dataset Extraction

This retrospective study was approved by the Research Ethics Board of our institution (Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada) with a waiver of informed consent. Patient datasets from June 2014 to June 2015 were extracted from the PACS of our institution. All patients included in this study were adults (older than 18 years of age) with MBTs, who had undergone 2 consecutive 3D-T1-Gad scans at our institution. Patients with other brain pathologies—such as primary brain neoplasms, MS lesions, or stroke—or those who had undergone a brain operation for any reason were excluded. Datasets were divided into 2 groups based on magnetic field strength (1.5T versus 3T) to assess the performance of the algorithm in 2 different scenarios. The 1.5T group had MR images obtained on 2 identical 1.5T TwinSpeed Excite scanners (GE Healthcare, Milwaukee, Wisconsin) with a standard 8-channel head coil. The parameters for 3D-T1-Gad included the following: TR/TE/flip angle, 8.6 ms/4.2 ms/20°; FOV, 220 × 220 mm; and voxel size, 0.58 × 0.58 × 1.5 mm. The 3T group had MR images obtained on a 3T Achieva TX scanner (Philips Healthcare, Best, the Netherlands) with a standard 8-channel head coil. The parameters for 3D-T1-Gad included the following: TR/TE/flip angle, 9.5 ms/2.3 ms/8°; FOV, 312 × 206 mm; and voxel size, 0.67 × 0.67 × 1.5 mm. The parameters are part of the routine contrast-enhanced brain MR imaging protocol at our institution.

### Image Analysis

#### Overview and Definitions.

In this section, the image-processing pipeline used to detect ΔMBTs is described (Fig 1).

![Fig 1.](http://www.ajnr.org/https://www.ajnr.org/content/ajnr/38/11/2059/F1.medium.gif)

[Fig 1.](http://www.ajnr.org/content/38/11/2059/F1)

Fig 1. 
Steps summarizing the preprocessing of patient datasets (*dotted box*) and calculation of the Jacobian operator field in the forward direction (*A*) and for segmentation of metastatic brain tumor candidates (*dotted box*) and detection of shrinking MBTs in the forward direction (*B*).

ΔMBTs were classified by the authors as either ΔMBTts, which represent ΔMBTs present on both baseline and follow-up scans, or ΔMBTos, which only appear on 1 scan (ie, newly developed or resolved MBTs). The volume change ratio (VCR) of an object across 2 scans was defined as the ratio of its volume change across time over its initial volume. This metric was defined because it is accepted that the values of JOFs are directly associated with the VCR of the brain, and higher JOF values suggest higher VCR and vice-versa.20

Concerning volume changes, “growing” MBTs refer to all MBTs that have grown in volume from baseline to follow-up scans, including newly developed MBTs. “Shrinking” MBTs refer to MBTs that shrank in size from baseline to follow-up, including resolved MBTs. Note that to detect growing MBTs in the forward direction (ie, from baseline to follow-up), our algorithm detected shrinking MBTs in the reverse direction (ie, from follow-up to baseline) because the JOF provides richer information in the shrinking field.30 In addition, this approach allowed detection of newly developed MBTs that do not have a corresponding MBT on baseline, though the processes going in the forward and reverse directions are identical. While our algorithm identified growing MBTs by detecting shrinking MBTs in the reverse direction, for clarity, here we will describe the process going in the forward direction only. SPM12 ([http://www.fil.ion.ucl.ac.uk/spm/software/spm12](http://www.fil.ion.ucl.ac.uk/spm/software/spm12)) and Matlab (MathWorks, Natick, Massachusetts) software packages were used for data processing. All average values are presented as mean ± SD.

#### Spatial Normalization.

To ensure that longitudinal baseline and follow-up scans were comparable and could be registered to one another for analyzing changes, we spatially normalized both scans to the Montreal Neurological Institute template31 with the methods described by Ashburner et al32 (Fig 1*A*) on SPM12 software (Fig 2).33

![Fig 2.](http://www.ajnr.org/https://www.ajnr.org/content/ajnr/38/11/2059/F2.medium.gif)

[Fig 2.](http://www.ajnr.org/content/38/11/2059/F2)

Fig 2. 
Axial sections of a patient's baseline (*A*) and follow-up (*B*) scans. The Jacobian operator field, calculated from the deformation field in the forward (*C*) and reverse (*D*) directions. The final output of our algorithm produced for baseline (*E*) and follow-up (*F*) scans, highlighting volume-changing metastatic brain tumors on each scan. Note that darker voxels on *C* and *D* correspond to negative JOF values and brighter voxels correspond to positive JOF values. The location of a metastatic brain tumor that has shrunk in size across the scans has been *circled* on *A–D*. The green on *E* indicates shrinkage, and the red on *F* indicates growth. Note that while this image is demonstrated in 2D, various operations as described here were performed in 3D.

#### Deformation Field.

Next, a nonrigid registration technique, based on joint diffeomorphic and rigid-body registration and proposed for intrasubject registration,34 was used to align the images using SPM12.33 This generated a 3D displacement field, which determined structural changes between the 2 scans (Fig 1*A*).

#### Calculation of the JOF.

To determine whether a tumor shrank between time points, the JOF—which quantifies local volume changes—was computed from the displacement field (Fig 2*C*, *-D*), using SPM12.33

#### Segmentation of MBT Candidates.

To segment MBT candidates, we used a modified version of the approach proposed by Seghier et al35 for lesion segmentation. First, a probabilistic segmentation of GM and WM was computed with the methods described by Ashburner and Friston.32 These 2 tissue maps were then added together to create a brain mask, which was then converted to binary in which voxels of nonzero value were retained. Because MBTs generally appear on MR images as bright objects fully or partially confined within the brain matter, they are not detected during this step and are therefore excluded from the binary mask, leaving “holes” within the brain mask. To detect MBT candidates, we created a binary mask of the holes, and this mask was multiplied to the original intensity image to identify MBT candidates in the intensity domain. Automatic postprocessing, with no user interaction, was completed to remove irrelevant objects that had an intensity lower than the average intensity of GM and WM.

#### False-Positive Reduction and Detection of ΔMBTs.

To limit MBT candidates to islands that shrank, we converted the MBT candidates to binary, multiplied to the JOF, and discarded 2D islands in the axial dimension with a positive median value, permitting false-positive reduction. To detect ΔMBTs, we then thresholded these MBT candidates using various JOF values. For each threshold value, 3D islands present on at least 1 section in the axial dimension with a median value of the threshold value or less were retained (Figs 1*B* and 2*E*, -*F*).

### Statistical Analysis

The presence and location of ΔMBTs were confirmed and extracted by a board-certified neuroradiologist with 5 years of experience, blinded to the technical details of our detection algorithm using Medical Image Processing, Analysis & Visualization (MIPAV; National Institutes of Health, Bethesda, Maryland); this defined our reference standard. To validate the algorithm, we considered its accuracy in detecting ΔMBTs in the 1.5T and 3T groups separately. For statistical analysis of the accuracy of our algorithm and to find the optimal threshold value of the JOF, we used receiver-operating characteristic (ROC) analysis.36 At each threshold value, a detection was marked as a true-positive if it overlapped an ΔMBT identified by the neuroradiologist; the rest were marked as false-positives. The area under the curve (AUC) of the ROC curve was used to evaluate accuracy, for which an AUC > 0.9 corresponds to a technique with statistically high accuracy.37 For verification of our results, in addition to assessing the algorithm using 2 independent datasets, we used the jackknife approach,38 with evaluation of the AUC, sensitivity, specificity, and false-positive rate (FPR) at the optimal threshold value of each iteration. To assess the potential role of our algorithm in complementing the performance of radiologists and reducing interobserver variability, we compared the true-positives detected by our algorithm against those detected by a different board-certified neuroradiologist with 3 years of experience, blinded to the technical details of our algorithm. The Mann-Whitney *U* test was used to compare the following: 1) the accuracy, sensitivity, specificity, and FPR of our algorithm at 1.5T and 3T; 2) the VCR of missed-versus-detected ΔMBTs; and 3) the VCR of ΔMBTos versus the VCR of ΔMBTts. A *P* < .05 defined statistical significance.

## Results

### Patient MR Imaging Datasets

The 1.5T group, comprising 30 patients, had 74 MBTs. The 3T group, comprising 20 patients, had 76 MBTs. Table 1 summarizes information about patient demographics, scan timeline, and MBT details.

View this table:
[Table 1:](http://www.ajnr.org/content/38/11/2059/T1)

Table 1: 
Summary of information on patients, scans, and MBTs

### Algorithm Performance

#### Detecting ΔMBTs.

For the 1.5T group, an ROC curve with 233 points (0 to −0.232, separated by −0.001) and, for the 3T group, an ROC curve with 656 points (0 to −0.655, separated by −0.001) were constructed by thresholding MBT candidates using various values of the JOF (Fig 3 and Table 2). Both ROC curves showed statistically high accuracy with an AUC of 0.925 and 0.965 for the 1.5T and 3T groups, respectively. There was no significant difference in the VCR between the detected and missed ΔMBTs for either group (*P* > .05).

![Fig 3.](http://www.ajnr.org/https://www.ajnr.org/content/ajnr/38/11/2059/F3.medium.gif)

[Fig 3.](http://www.ajnr.org/content/38/11/2059/F3)

Fig 3. 
*A*, Illustration of the receiver operating characteristic curve of our algorithm for detecting volume-changing metastatic brain tumors at 1.5T, constructed from 233 different thresholding values of the Jacobian operator field (from 0 to −0.232, separated by −0.001). The *arrow* shows the optimal point of balance. *B*, Illustration of the ROC curve of our algorithm for detecting ΔMBTs at 3T, constructed from 656 different thresholding values of the Jacobian operator field (from 0 to −0.655, separated by −0.001). The *arrow* shows the optimal point of balance between sensitivity and specificity, which happens at the Jacobian value of −0.182.

View this table:
[Table 2:](http://www.ajnr.org/content/38/11/2059/T2)

Table 2: 
Summary of ROC analysis, for detecting all 1.5T and 3T ΔMBTs and 1.5T ΔMBTts only, and the VCR of detected and missed ΔMBTs

### False-Positives

False-positives detected by the algorithm were divided into 5 categories (Table 3): 1) arteries: branches of anterior and middle cerebral arteries and vertebrobasilar system; 2) veins: superficial cortical veins, deep veins including internal cerebral veins, and basal veins of Rosenthal; 3) dural venous sinuses; 4) dura, tentorium, and falx cerebelli; and 5) choroid plexus. Vessels, including arteries, veins, and dural venous sinuses, were responsible for 79.0% ± 24.0% and 77.8% ± 18.5% of false-positives in the 1.5T and 3T groups, respectively. Visual inspection of the datasets suggested that at times, subtle structural differences such as pulsation of vessels between scans resulted in high JOF values and, consequently, detection as false-positives. However, these can easily be dismissed on visual inspection by a radiologist.

View this table:
[Table 3:](http://www.ajnr.org/content/38/11/2059/T3)

Table 3: 
Categories and distribution of false-positives at the optimal Jacobian threshold

### False-Negatives

At its optimal threshold value, the algorithm did not detect 11 ΔMBTs in the 1.5T group and 6 ΔMBTs in the 3T group. ΔMBTts false-negatives could subsequently be divided into the following (Table 4): 1) those that had a size of ≤2 voxels on one of the scans, or 2) those segmented poorly during the candidate segmentation portion of the algorithm, which consequently negatively affected the evaluation of their JOF values.

View this table:
[Table 4:](http://www.ajnr.org/content/38/11/2059/T4)

Table 4: 
Categories and distribution of false-negatives at the optimal Jacobian threshold

In the 1.5T group, the sensitivity of the algorithm in detecting ΔMBTos showed inferior values compared with its sensitivity in detecting ΔMBTts (75% versus 87.9%, respectively). This is even though ΔMBTts had a significantly lower VCR than ΔMBTos (*P* < .0001). These results suggested that ΔMBTos may be a major source of false-negatives for our algorithm. To assess this, we evaluated the performance of the algorithm with ΔMBTos omitted. This analysis was only performed in the 1.5T group because all instances of ΔMBTos were detected in the 3T group. Like ΔMBT detection, an ROC curve with 302 points (0 to −0.301, separated by −0.001) was generated with an AUC of 0.929. At its optimal performance (achieved at a threshold value of −0.035), the algorithm had a sensitivity of 87.9%, specificity of 86.6%, and FPR of 0.210 per section, equivalent to 25.4 per scan. Note that detected the ΔMBTos was included in the calculation of specificity and FPR.

### Verification

In addition to performing all analyses on 2 independent patient datasets, as demonstrated above, we conducted jackknifing for further verification of findings. Results from jackknifing were like the findings before the procedure, verifying initial results (Table 5). Furthermore, statistical analysis of the mean AUC of the algorithm and the sensitivity of the algorithm when applied to jackknifed 1.5T ΔMBTts-only data versus jackknifed 1.5T ΔMBT data showed significant improvement (*P* < .0001 for both AUC and sensitivity).

View this table:
[Table 5:](http://www.ajnr.org/content/38/11/2059/T5)

Table 5: 
Summary of ROC analysis after jackknifing datasets, for detecting all 1.5T and 3T ΔMBTs and 1.5T ΔMBTts only

### Effect on Performance

Of the 150 ΔMBTs in our datasets, a second neuroradiologist could detect 144 ΔMBTs, equivalent to 96% sensitivity. Of the 6 ΔMBTs missed by the neuroradiologist, 2 ΔMBTs, consisting of 1 ΔMBTos and 1 ΔMBTts, were missed by both the algorithm and neuroradiologist. However, 1 ΔMBTos and 3 instances of ΔMBTts were missed by the neuroradiologist but detected by the algorithm. Inspection of those 4 ΔMBTs revealed subtle volume changes, ranging from 9.1 × 10−2 to 2.5 × 10−1 mL. The addition of these detections would improve the neuroradiologist's sensitivity to 98.7%.

## Discussion

In this article, we have presented our semiautomated algorithm for detecting ΔMBTs in longitudinal brain MRIs with statistically high accuracy at 1.5T and 3T (AUC > 0.9). Our approach uses the JOF—a vector displacement field operator—which quantifies longitudinal structural changes of the brain on the basis of the deformation field. At its optimal performance, our algorithm had a sensitivity of 85.1% and 92.1% and a specificity of 86.7% and 91.3% for the 1.5T and 3T groups, respectively. Although performances at the optimal thresholds were similar, the difference in optimal JOF threshold values for the 2 groups indicated the effect of scan parameters on optimal threshold values and the need to identify a unique optimal threshold value for each set of scan parameters. Moreover, missed and detected ΔMBTs had a statistically similar VCR, suggesting that the detection ability of our algorithm is not dependent on VCR values. Compared with manual reading, this can be considered an advantage of our algorithm because manual reading of longitudinal brain MR images is biased against subtle changes.16 Thus, the findings in this study supported the potential of our algorithm to aid performance by identifying lesions with subtle volume change that could otherwise be missed by the user. Although it is accepted that the values of the Jacobian operator field are directly associated with the VCR of the brain,20 we speculate that this association may not be the case for VCR values of individual brain regions because the missed and detected ΔMBTs of our algorithm had statistically similar VCRs despite differences in JOF values. Although this difference may limit the information provided by the JOF about the amount of volume change, our results demonstrate the adequacy of the JOF in detecting ΔMBTs, even for relatively small VCR values, and support its potential to serve as a diagnostic aid.

At present, the literature is limited in studies concerning the follow-up and volume-change detection of MBTs. Chitphakdithai et al15 proposed a method for tracking MBTs, relying on a 4-level label map to denote the intensity-correspondence relation between baseline and follow-up images. Although authors reported a sensitivity of 92% in detecting ΔMBTs, this was verified on only a limited dataset comprising 3 patients. In addition, the authors did not report specificity or FPR; this omission prevented accurate comparison with our algorithm.

With our algorithm, nonlinear registration of baseline and follow-up scans was used to detect structural changes. One of its main limitations is the infinite number of displacement fields for a given pair of scans, depending on the registration technique used.16 More important, these techniques rely on the assumption that a lesion or tissue region exists on both scans with similar intensity.16 This assumption limits the detection of ΔMBTts of negligible size on one of the scans and ΔMBTos. Reviewing the detections of the algorithm showed that most false-negatives were of the aforementioned 2 categories. Moreover, although our algorithm could detect 75% of ΔMBTos at 1.5T, this was inferior to the general sensitivity of our algorithm; however, this problem was not isolated to our algorithm.

A similar problem exists in MS, in which computer-aided detection tools are more developed and prevalent in the clinical environment. Although MS computer-aided detection tools focus more on the amount of volume change as opposed to mere detection of change, both techniques use automated quantitative analysis techniques that can be compared. A recent study by Cabezas et al39 combined subtraction and deformation field analyses to detect new MS lesions on T2-weighted images. Using this approach on 36 patient datasets, Cabezas et al reported a 70.8% sensitivity.39 Comparatively, our 1.5T datasets consisted of 30 patients with 16 instances of ΔMBTos. When these were omitted, the AUC and sensitivity of our algorithm improved; this change suggests that ΔMBTos has a negative effect on its performance. However, because detecting ΔMBTos is equivalent to detecting MBTs on a single scan, this limitation can be addressed by combining our algorithm with previously described MBT computer-aided detection tools.9⇓⇓⇓⇓–14

Further enhancement of the clinical utility of our algorithm rests on improving its FPR. Like other studies concerned with automatic MBT detection on 3D-T1-Gad,9⇓⇓⇓–13 bright vessels were a major source of false-positives. Differentiating vessels and MBTs could be achieved by incorporating 3D template matching–based algorithms for MBT segmentation9⇓⇓⇓–13 in place of our current method. This may also improve the sensitivity of our algorithm because some of the false-negatives of our technique were a direct result of poor MBT segmentation. Further false-positive reduction may be achieved with vessel filters40,41 or cerebrovascular atlases.42 The use of black-blood MR imaging can also address this problem14 because this technique has been shown to be superior in capturing smaller MBTs compared with other MR images.43,44 More recently, Pérez-Ramírez et al13 proposed the use of a degree of anisotropy to distinguish blood vessels and MBTs.

## Conclusions

The proposed semiautomated algorithm presented in this article could detect volume-changing MBTs on longitudinal brain MR imaging with high accuracy. With the growing quantity and quality of automated techniques for MBT detection, implementation of some of these elements may further improve the sensitivity, specificity, and FPR of our algorithm. The clinical role of our technique lies in its potential to improve the high workload7 and ambiguity16 associated with manual reading, which is crucial for appropriate treatment.5 Through demonstrating the potential of deformation-based techniques and our algorithm, this study serves as an initial step in developing a computer-aided change-detection tool to complement the performance of radiologists.

## Footnotes

*   Disclosures: Omid Shearkhani—*RELATED*: *Grant*: Brain Tumour Foundation of Canada Research Studentship, Radiological Society of North America Medical Student Grant. Arjun Sahgal—*UNRELATED*: *Consultancy*: Varian Medical Systems, Hoffmann-La Roche; *Grants/Grants Pending*: Elekta; *Payment for Development of Educational Presentations*: Medtronic, Elekta, Accuray Inc, Varian Medical Systems; *Travel/Accommodations/Meeting Expenses Unrelated to Activities Listed*: Medtronic, Elekta, Varian Medical Systems; *OTHER RELATIONSHIPS*: past educational seminars with Medtronic, Elekta, Accuray Inc, and Varian Medical Systems; consulting/advisory role with Varian Medical Systems, Hoffmann-La Roche; research grant with Elekta; travel accommodations/expenses by Medtronic, Elekta, and Varian Medical Systems. Dr Sahgal also belongs to the Elekta MR Linac Research Consortium. Pejman Jabehdar Maralani—*UNRELATED*: *Grants/Grants Pending*: grants from the Radiological Society of North American and the Brain Tumour Foundation of Canada.* *Money paid to the institution.

*   This work was supported by the Radiological Society of North America (RMS1640 to O.S.) and the Brain Tumour Foundation of Canada (to O.S).

## References

1.  1. Nussbaum ES, Djalilian HR, Cho KH, et al. Brain metastases: histology, multiplicity, surgery, and survival. Cancer 1996;78:1781–88 pmid:8859192
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1002/(SICI)1097-0142(19961015)78:8<1781::AID-CNCR19>3.0.CO;2-U&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=8859192&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=A1996VL24400019&link_type=ISI) 

2.  2. Okunieff P, Schell MC, Ruo R, et al. Long-term management of patients with multiple brain metastases after shaped beam radiosurgery: case report and review of the literature. J Neurosurg 2004;101(suppl 3):46–12 doi:10.3171/jns.2004.101.supplement 3.0406 pmid:15537197
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.3171/jns.2004.101.supplement 3.0406&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=15537197&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

3.  3. Patchell RA, Tibbs PA, Walsh JW, et al. A randomized trial of surgery in the treatment of single metastases to the brain. N Engl J Med 1990;322:494–500 doi:10.1056/NEJM199002223220802 pmid:2405271
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1056/NEJM199002223220802&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=2405271&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=A1990CP69600002&link_type=ISI) 

4.  4. Smith ML, Lee JY. Stereotactic radiosurgery in the management of brain metastasis. Neurosurg Focus 2007;22:1–8 doi:10.3171/foc.2007.22.6.7 pmid:17608358
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.3171/foc.2007.22.3.1&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=17613227&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

5.  5. Patel SH, Robbins JR, Gore EM, et al; Expert Panel on Radiation Oncology–Brain Metastases. ACR Appropriateness Criteria® follow-up and retreatment of brain metastases. Am J Clin Oncol 2012;35:302–06 doi:10.1097/COC.0b013e31824be246 pmid:22609733
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1097/COC.0b013e31824be246&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=22609733&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

6.  6. Kakeda S, Korogi Y, Hiai Y, et al. Detection of brain metastasis at 3T: comparison among SE, IR-FSE and 3D-GRE sequences. Eur Radiol 2007;17:2345–51 doi:10.1007/s00330-007-0599-9 pmid:17318603
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1007/s00330-007-0599-9&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=17318603&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

7.  7. Prastawa M, Bullitt E, Moon N, et al. Automatic brain tumor segmentation by subject specific modification of atlas priors. Acad Radiol 2003;10:1341–48 doi:10.1016/S1076-6332(03)00506-3 pmid:14697002
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/S1076-6332(03)00506-3&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=14697002&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

8.  8. Loganathan AG, Chan MD, Alphonse N, et al. Clinical outcomes of brain metastases treated with Gamma Knife radiosurgery with 3.0 T versus 1.5 T MRI-based treatment planning: have we finally optimised detection of occult brain metastases? J Med Imaging Radiat Oncol 2012;56:554–60 doi:10.1111/j.1754-9485.2012.02429.x pmid:23043576
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1111/j.1754-9485.2012.02429.x&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=23043576&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

9.  9. Ambrosini RD. *Automated Detection and Growth Rate Determination of Metastatic Tumors in the Brain and Lungs* [dissertation]. Rochester: University of Rochester; 2010:134
    
    

10. 10. Ambrosini RD, Wang P, O'Dell WG. Computer-aided detection of metastatic brain tumors using automated three-dimensional template matching. J Magn Reson Imaging 2010;31:85–93 doi:10.1002/jmri.22009 pmid:20027576
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1002/jmri.22009&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=20027576&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

11. 11. Farjam R, Parmar HA, Noll DC, et al. An approach for computer-aided detection of brain metastases in post-Gd T1-W MRI. Magn Reson Imaging 2012;30:824–36 doi:10.1016/j.mri.2012.02.024 pmid:22521993
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.mri.2012.02.024&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=22521993&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

12. 12. Pérez-Ramírez Ú, Arana E, Moratal D. Computer-aided detection of brain metastases using a three-dimensional template-based matching algorithm. In: Proceedings of the 36th Annual International Conference of the Institute of Electrical and Electronics Engineers Engineering in Medicine and Biology Society, Chicago, Illinois. August 26–30, 2014:2384–87
    
    

13. 13. Pérez-Ramírez Ú, Arana E, Moratal D. Brain metastases detection on MR by means of three-dimensional tumor-appearance template matching. J Magn Reson Imaging 2016;44:642–52 doi:10.1002/jmri.25207 pmid:26934581
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1002/jmri.25207&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=26934581&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

14. 14. Yang S, Nam Y, Kim MO, et al. Computer-aided detection of metastatic brain tumors using magnetic resonance black-blood imaging. Invest Radiol 2013;48:113–19 doi:10.1097/RLI.0b013e318277f078 pmid:23211553
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1097/RLI.0b013e318277f078&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=23211553&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

15. 15.1.  Durrleman S, 
    2.  Fletcher T, 
    3.  Gerig G, 
    4.  Niethammer M
    
     Chitphakdithai N, Chiang VL, Duncan JS. Tracking metastatic brain tumors in longitudinal scans via joint image registration and labeling. In: Durrleman S, Fletcher T, Gerig G, Niethammer M, eds. Spatio-temporal Image Analysis for Longitudinal and Time-Series Image Data: STIA 2012. Berlin: Springer; 2012:124–36
    
    

16. 16. Patriarche J, Erickson B. A review of the automated detection of change in serial imaging studies of the brain. J Digit Imaging 2004;17:158–74 doi:10.1007/s10278-004-1010-x pmid:15534751
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1007/s10278-004-1010-x&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=15534751&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

17. 17. Doi K. Computer-aided diagnosis in medical imaging: historical review, current status and future potential. Comput Med Imaging Graph 2007;31:198–211 doi:10.1016/j.compmedimag.2007.02.002 pmid:17349778
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.compmedimag.2007.02.002&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=17349778&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

18. 18. Doi K. Diagnostic imaging over the last 50 years: research and development in medical imaging science and technology. Phys Med Biol 2006;51:R5–27 doi:10.1088/0031-9155/51/13/R02 pmid:16790920
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1088/0031-9155/51/13/R02&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=16790920&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

19. 19. Takahashi R, Kajikawa Y. Computer-aided diagnosis: a survey with bibliometric analysis. Int J Med Inform 2017;101:58–67 doi:10.1016/j.ijmedinf.2017.02.004 pmid:28347448
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.ijmedinf.2017.02.004&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=28347448&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

20. 20. Lladó X, Ganiler O, Oliver A, et al. Automated detection of multiple sclerosis lesions in serial brain MRI. Neuroradiology 2012;54:787–807 doi:10.1007/s00234-011-0992-6 pmid:22179659
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1007/s00234-011-0992-6&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=22179659&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

21. 21. Martola J, Bergström J, Fredrikson S, et al. A longitudinal observational study of brain atrophy rate reflecting four decades of multiple sclerosis: a comparison of serial 1D, 2D, and volumetric measurements from MRI images. Neuroradiology 2010;52:109–17 doi:10.1007/s00234-009-0593-9 pmid:19774369
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1007/s00234-009-0593-9&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=19774369&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

22. 22. Tian W, Zhu T, Zhong J, et al. Progressive decline in fractional anisotropy on serial DTI examinations of the corpus callosum: a putative marker of disease activity and progression in SPMS. Neuroradiology 2012;54:287–97 doi:10.1007/s00234-011-0885-8 pmid:21567135
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1007/s00234-011-0885-8&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=21567135&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

23. 23. Calcagno G, Staiano A, Fortunato G, et al. A multilayer perceptron neural network-based approach for the identification of responsiveness to interferon therapy in multiple sclerosis patients. Information Sciences 2010;180:4153–63 doi:10.1016/j.ins.2010.07.004
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.ins.2010.07.004&link_type=DOI) 

24. 24. Südmeyer M, Pieperhoff P, Ferrea S, et al. Longitudinal deformation-based morphometry reveals spatio-temporal dynamics of brain volume changes in patients with corticobasal syndrome. PLoS One 2012;7:e41873 doi:10.1371/journal.pone.0041873 pmid:22848644
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1371/journal.pone.0041873&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=22848644&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

25. 25. Durand-Dubief F, Belaroussi B, Armspach J, et al. Reliability of longitudinal brain volume loss measurements between 2 sites in patients with multiple sclerosis: comparison of 7 quantification techniques. AJNR Am J Neuroradiol 2012;33:1918–24 doi:10.3174/ajnr.A3107 pmid:22790248
    
    [Abstract/FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpuciI7czo1OiJyZXNpZCI7czoxMDoiMzMvMTAvMTkxOCI7czo0OiJhdG9tIjtzOjIxOiIvYWpuci8zOC8xMS8yMDU5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

26. 26. Fuentes D, Contreras J, Yu J, et al. Morphometry-based measurements of the structural response to whole-brain radiation. Int J Comput Assist Radiol Surg 2015;10:393–401 doi:10.1007/s11548-014-1128-3 pmid:25408306
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1007/s11548-014-1128-3&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=25408306&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

27. 27. Unay D. Local and global volume changes of subcortical brain structures from longitudinally varying neuroimaging data for dementia identification. Comput Med Imaging Graph 2012;36:464–73 doi:10.1016/j.compmedimag.2012.03.006 pmid:22658230
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.compmedimag.2012.03.006&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=22658230&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

28. 28. Thirion JP, Calmon G. Deformation analysis to detect and quantify active lesions in three-dimensional medical image sequences. IEEE Trans Med Imaging 1999;18:429–41 doi:10.1109/42.774170 pmid:10416804
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1109/42.774170&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=10416804&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

29. 29.1.  Höhne KH, 
    2.  Kinikis R
    
     Davatzikos C, Vaillant M, Resnick S, et al. Morphological analysis of brain structures using spatial normalization. In: Höhne KH, Kinikis R. Visualization in Biomedical Computing: 4th International Conference, VBC ‘96, Hamburg, Germany, September 22–25, 1996: Proceedings. Berlin: Springer-Verlag; 1996:355–60
    
    

30. 30. Rey D, Subsol G, Delingette H, et al. Automatic detection and segmentation of evolving processes in 3D medical images: application to multiple sclerosis. Med Image Anal 2002;6:163–79 doi:10.1016/S1361-8415(02)00056-7 pmid:12045002
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/S1361-8415(02)00056-7&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=12045002&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

31. 31. Evans AC, Collins DL, Mills S, et al. 3D statistical neuroanatomical models from 305 MRI volumes. In: Institute of Electrical and Electronics Engineers. IEEE Nuclear and Plasma Sciences Society. Nuclear Science Symposium and Medical Imaging Conference, 1993; 1993 IEEE Conference Record. Piscataway: Institute of Electrical and Electronics Engineers; 1993:1813–17
    
    

32. 32. Ashburner J, Friston KJ. Unified segmentation. Neuroimage 2005;26:839–51 doi:10.1016/j.neuroimage.2005.02.018 pmid:15955494
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.neuroimage.2005.02.018&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=15955494&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=000230211100020&link_type=ISI) 

33. 33. Ashburner J, Barnes G, Chen C, et al. SPM12 Manual. London: Wellcome Trust Centre for Neuroimaging; 2016
    
    

34. 34. Ashburner J, Ridgway GR. Symmetric diffeomorphic modeling of longitudinal structural MRI. Front Neurosci 2012;6:197 doi:10.3389/fnins.2012.00197 pmid:23386806
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.3389/fnins.2012.00197&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=23386806&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

35. 35. Seghier ML, Ramlackhansingh A, Crinion J, et al. Lesion identification using unified segmentation-normalisation models and fuzzy clustering. Neuroimage 2008;41:1253–66 doi:10.1016/j.neuroimage.2008.03.028 pmid:18482850
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.neuroimage.2008.03.028&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=18482850&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=000256620400009&link_type=ISI) 

36. 36. Greiner M, Pfeiffer D, Smith R. Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Prev Vet Med 2000;45:23–41 doi:10.1016/S0167-5877(00)00115-X pmid:10802332
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/S0167-5877(00)00115-X&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=10802332&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=000087096600003&link_type=ISI) 

37. 37. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29–36 doi:10.1148/radiology.143.1.7063747 pmid:7063747
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1148/radiology.143.1.7063747&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=7063747&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=A1982NG95400006&link_type=ISI) 

38. 38. Miller RG. The jackknife–a review. Biometrika 1974;61:1–15
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1093/biomet/61.1.1&link_type=DOI) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=A1974S717000001&link_type=ISI) 

39. 39. Cabezas M, Corral J, Oliver A, et al. Improved automatic detection of new T2 lesions in multiple sclerosis using deformation fields. AJNR Am J Neuroradiol 2016 Jun 9. [Epub ahead of print] doi:10.3174/ajnr.A4829 pmid:27282863
    
    [Abstract/FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpuciI7czo1OiJyZXNpZCI7czoxMDoiMzcvMTAvMTgxNiI7czo0OiJhdG9tIjtzOjIxOiIvYWpuci8zOC8xMS8yMDU5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

40. 40.1.  Wells WM, 
    2.  Colchester AC, 
    3.  Delp S
    
     Frangi AF, Niessen WJ, Vincken KL, et al. Multiscale vessel enhancement filtering. In: Wells WM, Colchester AC, Delp S. Medical Image Computing and Computer-Assisted Interventation: MICCAI'9. Berlin: Springer-Verlag; 1998:130–37
    
    

41. 41. Li Q, Sone S, Doi K. Selective enhancement filters for nodules, vessels, and airway walls in two- and three-dimensional CT scans. Med Phys 2003;30:2040–51 doi:10.1118/1.1581411 pmid:12945970
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1118/1.1581411&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=12945970&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

42. 42. Forkert N, Fiehler J, Suniaga S, et al. A statistical cerebroarterial atlas derived from 700 MRA datasets. Methods Inf Med 2013;52:467–74 doi:10.3414/ME13-02-0001 pmid:24190179
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.3414/ME13-02-0001&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=24190179&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

43. 43. Park J, Kim EY. Contrast-enhanced, three-dimensional, whole-brain, black-blood imaging: application to small brain metastases. Magn Reson Med 2010;63:553–61 doi:10.1002/mrm.22261 pmid:20187162
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1002/mrm.22261&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=20187162&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=000274938000002&link_type=ISI) 

44. 44. Park J, Kim J, Yoo E, et al. Detection of small metastatic brain tumors: comparison of 3D contrast-enhanced whole-brain black-blood imaging and MP-RAGE imaging. Invest Radiol 2012;47:136–41 doi:10.1097/RLI.0b013e3182319704 pmid:22104961
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1097/RLI.0b013e3182319704&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=22104961&link_type=MED&atom=%2Fajnr%2F38%2F11%2F2059.atom) 

*   Received January 5, 2017.
*   Accepted after revision June 15, 2017.


*   © 2017 by American Journal of Neuroradiology