Optimal Presentation Modes for Detecting Brain Tumor Progression

Discussion

We found an improvement in rater performance when using advanced tools for display. Automated change detection and subtraction significantly increased the ability to detect subtle progression of primary brain tumors, relative to the traditional side-by-side image comparison method. They also had value in determining that a given case was truly negative: if a case was considered nonprogressive after using these tools, there was a greater TTP than if these tools were not used. Although change detection and subtraction showed similar performance, it is important to note that they are fundamentally different, in that subtraction highlights changes, but the observer must always determine whether this change is “real.” Change detection includes a step where that decision threshold is set and probably accounts for the trend for greater agreement between observers.

We had expected that the automated change detector might have an efficiency advantage (more rapid time to rate a case) because it integrates information from several images into a single image to determine possible areas of progression, but this was not demonstrated here. It may be that the change detector was overly sensitive and labeled areas as progression that, after careful study, were not progression. This may be worth a separate investigation.

It was expected that flicker would show a significant advantage in determining tumor progression, particularly because flicker mode was the method preferred by the panel to decide difficult cases. We did not see an advantage for flicker, perhaps because this display method was new to 2 of the 3 raters, and they may not have used the technique optimally. NF should have taken longer than N, because when using flicker as a first-line strategy (rather than a confirmatory strategy) on images with changing acquisition parameters, the rater should have stared at various regions of the flicker image for every section, for several seconds per region. That an increased time was not observed suggests that the raters did not benefit from flicker because they failed to use it optimally. We do note that the rater that had more experience with flicker (from other research projects using flicker display mode) showed a greater advantage for flicker; so, it may be that if there had been a greater training period, there might have been an advantage detected. Because flicker has shown an advantage for other perception tasks, we believe that further investigation of this negative result is warranted.

Radiologists have long compared examinations from different time points in chronic diseases such as cancer. Historically, hardcopy images have been compared side-by-side as a natural extension of the method of image acquisition. With digital acquisition, the requirement for side-by-side comparison was removed, but the convention has continued, even though human perception is less effective at detecting differences in images when a saccade is required.²

Studies of the efficiency of interpreting imaging examinations on computer displays versus film have been performed. In most such studies, the raters were relatively new to computer-based interpretation, there was little effort to implement alternative display modes for computers, and therefore these largely emulated the film comparison format.⁷ Reports on the benefits of electronic interpretation times generally show a benefit, ranging from dramatic to moderate⁸ to none.⁹ However, those studies generally did not evaluate the mode of display, but simply the speed. Additional studies have documented an advantage in understanding complex, spatial relationships for “stack-mode” display versus “tile-mode (film mode)”¹⁰ as well as substantially faster interpretation times for cross-sectional studies.^8,11

Image registration is more challenging to implement in clinical practice, limiting its use in clinical practice. This study used only rigid body registration, which is faster and more reliable than image warping. Rigid registration works well for certain anatomies where the body parts remain fairly fixed, including the brain and skull. Schellingerhout et al.¹² demonstrated the clinical utility of image registration applied to head CT, for both agreement of radiologic interpretation and a reduction in time to report an examination. For other body parts such as the abdomen, rigid registration does not work well, and either subregions must be selected or warping must be applied.

Once registration is complete, other processing and display options are available, the most obvious being image subtraction. Image subtraction has been used in some cases where the images were acquired in the same spatial setting (eg, subtraction angiography), though algorithms have been proposed for adjusting for different patient positions in projection radiographs.¹ Postacquisition computed registration opens up many new possible applications. The value of subtraction has been demonstrated in several MR applications, including multiple sclerosis^13,14 and brain tumors.¹⁵ In the Tan et al¹⁴ study, they used a custom application that always showed old and new examinations in a combined registered and subtracted image; but the value of each display mode was not evaluated.

The report by Alpert et al¹⁶ describes the use of image registration and subtraction of CT scans without and with contrast material for improving diagnosis of vascular lesions. They also made the argument that the technology was mature and should be routinely applied in clinical practice. However, their report was focused on the accuracy of registration with few clinical examples and did not document the clinical value in terms of sensitivity, specificity, or efficiency. Takao et al¹⁷ describe the application of a nonrigid registration method to chest CT and describe good visual results. They also presented subtraction images as a good way to allow improved perception of changes over time. Unfortunately, they did not take the next step of formally testing it versus other display methods with multiple radiologists. Other applications described¹³ include detection of posttraumatic changes, as well as changes due to inhalation of 100% oxygen. We note, however, that this report did not document any comparison to other display modes, nor evaluate its possible clinical value.

We did not study the value of detecting small changes in brain tumors. The difference in TTP in this study suggests that there may at least be prognostic value and that may be important in deciding how aggressively to manage patients, especially with continuing advancements in therapy. The value for low-grade tumors may be greater, though we did not separately study low- versus high-grade tumors. It is not uncommon practice to observe tumors until they dedifferentiate or “go bad.” Having the ability to detect subtle changes may allow us to detect the change earlier and possibly allow intervention before the tumor grows so large that it cannot be completely resected or aggressively treated.

These display modes also may be of value in clinical trials and may lead to new treatment algorithms. The cost of conducting a clinical trial is directly proportional to the duration of a study and the number of subjects. These are both directly affected by the tumor measurement method. A technique that could reliably detect small progressions might allow clinical trials to be conducted on smaller cohorts (because of less variability) with a shorter duration (because of higher sensitivity).

There are some important limitations to this study. We recognize that 1 of the standards used here was based on the opinion of the panel and not on biopsy proof. Biopsy proof is hard to obtain in change analysis method studies, because getting tissue to measure the state at the baseline time point will disturb the follow-up images. It is possible that all 3 radiologists will incorrectly assign an examination pair to 1 category. In that case, the method that most agrees with the 3 raters will look good but will be equally wrong. We also note that we did not power this study to account for multiple comparison adjustments.

We did compare the results with TTP, which is broadly used in clinical trials, but it is partially based on imaging measures and therefore is somewhat circular. Because RECIST is based on large imaging changes (ie, >20% increase in the maximum dimension of the tumor versus the baseline examination), it is rather insensitive, and because it is based on 1 unidimensional measure, it also has a large amount of noise due to variations in positioning of the measurement line. In our case, we did show that having a negative examination (“no progression” based on this study's definition) by using the registered change detection or subtraction images with or without flicker display did predict stability for a longer period than without these tools. There was not a difference for cases where there was progression. We suspect that this may be due to the fact that detecting small changes early when presently there is no effective treatment may not have an impact on outcome. In addition, this information about early progression was not available to the treating physician, so there was no opportunity to alter treatment. We should also note that the ability to detect early progression might allow for new treatment algorithms or yet to be developed therapies that could result in some improved outcome, though this is speculation.

The display application was not incorporated into our clinical PACS. Translating this research application into a clinical software application will be challenging unless vendors provide support for these tools and display modes. We believe this represents an opportunity for vendors to develop innovative solutions that could potentially improve patient care and might give them a competitive advantage.

Another limitation is that these results cannot be generally applied to all body parts. The brain is a relatively fixed structure and is amenable to rigid registration. Other studies of rigid registration have demonstrated little advantage or even disadvantages for structures that move, such as those in the abdomen.¹⁸ Therefore, the applicability of this study is limited to the rigid structures such as the head or spine, until a suitable registration method has been found.