Abstract
BACKGROUND AND PURPOSE: The NI-RADS lexicon doesn't use ADC parameters and T2 weighted signal for ascribing categories. We explored ADC, DWI, and T2WI to examine the diagnostic accuracy in primary sites of postsurgical oral cavity carcinoma in the Neck Imaging Reporting and Data System (NI-RADS) categories 2 and 3.
MATERIALS AND METHODS: We performed a retrospective analysis in clinically asymptomatic post-surgically treated patients with oral cavity squamous cell carcinoma who underwent contrast-enhanced MRI between January 2013 and January 2016. Histopathology and follow-up imaging were used to ascertain the presence or absence of malignancy in subjects with “new enhancing lesions,” which were interpreted according to the NI-RADS lexicon by experienced readers, including NI-RADS 2 and 3 lesions in the primary site. NI-RADS that included T2WI and DWI (referred to as NI-RADS A) and ADC (using the best cutoff from receiver operating characteristic curve analysis, NI-RADS B) was documented in an Excel sheet to up- or downgrade existing classic American College of Radiology NI-RADS and calculate diagnostic accuracy.
RESULTS: Sixty-one malignant and 23 benign lesions included in the study were assigned American College of Radiology NI-RADS 2 (n = 33) and NI-RADS 3 (n = 51) categories. The recurrence rate was 90% (46/51) for NI-RADS three, 45% (15/33) for NI-RADS 2, and 73% (61/84) overall. T2WI signal morphology was intermediate in 45 subjects (53.5%) and restricted DWI in 54 (64.2%). Sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of the American College of Radiology NI-RADS were the following: NI-RADS (75.4%, 78.3%, 90.1%, 54.5%, and 76.1%); NI-RADS A (79.1%, 81.2%, 91.9%, 59.1%, and 79.6%); and NI-RADS B (88.9%, 72.7%, 91.4%, 66.7%, and 85.1%), respectively.
CONCLUSIONS: Adding MR imaging diagnostic characteristics like T2WI, DWI, and ADC to the American College of Radiology NI-RADS improved diagnostic accuracy and sensitivity.
ABBREVIATIONS:
- ACR
- American College of Radiology
- AUC
- area under the ROC curve
- CE
- contrast-enhanced CT
- HNSCC
- head and neck squamous cell carcinoma
- NI-RADS
- Neck Imaging Reporting and Data System
- NPV
- negative predictive value
- PPV
- positive predictive value
- ROC
- receiver operating characteristic
Head and neck squamous cell carcinomas (HNSCCs) recur in up to 15%–50% of patients, most frequently in the first 2–3 years after treatment, and surveillance imaging is critical during this period to detect recurrence as quickly as possible, contributing to optimistic salvage treatment choices.1 Salvage surgery may be undertaken in the setting of recurrent disease, and if this is not practicable, chemotherapy and a biologic agent may be used.2 Patients with recurrences of early-stage HNSCC who undergo salvage surgery have a 70% 2-year relapse-free survival rate, as opposed to others with advanced-stage HNSCC, who have a 22% 2-year relapse-free survival rate.3
The National Comprehensive Cancer Network suggests posttreatment baseline imaging after 6 months of therapy for advanced HNSCC but does not formally advocate imaging for asymptomatic patients.4 Few studies show a clear benefit or survival advantage to using ubiquitous surveillance imaging after the initial baseline assessment.5 However, the ideal timing and frequency for using PET with contrast-enhanced (CE) CT in the posttreatment context have not yet been established.6
The American College of Radiology (ACR) in 2016 released the first iteration of a standardized reporting system dubbed the Neck Imaging Reporting and Data System (NI-RADS).7 The lexicon defines NI-RADS 2b lesions as deep soft-tissue lesions with ill-defined or mildly enhancing soft tissue or soft tissue with only mild FDG uptake. These are indeterminate lesions and should have a short-term-interval follow-up. Depending on the degree of suspicion for treatment failure, NI-RADS 3 lesions should have a short-term-interval follow-up or biopsy. In past studies, recurrence rates in NI-RADS 2b and NI-RADS 3 lesions were 5.6% and 60%–80%, respectively.8
The template was initially designed for CECT with a concentration on PET/CT for detecting recurrence; MR imaging has been recently explored in various studies.9⇓-11 ADC has potential value for distinguishing recurring tumors and posttreatment changes in head and neck squamous cell carcinoma.12 Although DWI has become a routine sequence in head and neck imaging, it is not yet a criterion in NI-RADS;13 until recently, when a study11 evaluated the additive role of T2WI DWI in the NI-RADS MR imaging lexicon. Past studies5,8,11,14,15 have appraised MR imaging in NI-RADS but with numerous limitations like heterogeneous imaging with a more miniature representation of MR imaging8 and a smaller amount of oral cavity squamous cell carcinoma as a subsite,14,15 small NI-RADS 3 in a representative sample population,14 and lack of a fixed interval for inclusion in the sample population.15
We aimed to explore the incremental diagnostic accuracy using ADC, DWI, and T2WI in NI-RADS categories 2 and 3 in primary sites of postsurgical oral cavity carcinoma.
MATERIALS AND METHODS
Patient Selection
After institutional review board approval, a retrospective analysis was performed in patients with post-surgically treated oral cavity squamous cell carcinoma who presented between January 2013 and January 2016 to the surgical outpatient services as a part of the institution-based protocol for surveillance contrast-enhanced MR imaging. The final cohort consisted of clinically asymptomatic patients who were determined to have “new enhancing lesions” on surveillance imaging. A time interval of >3 months and <2 years from the point of surgery and/or completion of chemoradiation was considered suitable clinically for inclusion. If imaging surveillance examination findings are worrisome but not conclusive of recurrence, a multidisciplinary meeting every week determines the decision to shorten the next surveillance interval. A routine follow-up imaging study was performed every 8–12 weeks for the new enhancing lesion.
Occult recurrence was without troubling symptoms or physical examination evidence consistent with posttreatment changes. Suspicious clinical examination results include new palpable abnormalities on physical examination or a suspicious finding by surgeons performing flexible fiber optic endoscopy. Clinically asymptomatic patients determined to have new enhancing lesions on surveillance imaging were assessed for tissue-sampling feasibility. Histopathology was the criterion standard to confirm the presence or absence of malignancy. If not feasible due to any reason, the new enhancing lesion was assigned “benign” if it resolved or shrank spontaneously in the absence of therapy within 6–12 months as determined by a follow-up imaging study. New enhancing lesions were labeled malignant if confirmed histopathologically or on follow-up imaging. Only NI-RADS categories 2 and 3 were included in the study. Figure 1 summarizes the flow chart of the study enrolling process and reasons for patient exclusion.
Flow chart of the enrolling process of the study and reasons for patient exclusion. OCSCC indicates oral cavity squamous cell carcinoma.
In the stipulated study period, a total of 84 patients were enrolled. Informed consent was waived after approval from the institutional review board and the scientific committee.
Imaging Study and Image Analysis
We used a 1.5T Avanto (Siemens) machine with a circular polarization surface coil to perform traditional diffusion-weighted scans. The patients had DWIs performed using a multisection spin-echo single-shot EPI sequence. An average of 15 sections was obtained in the axial plane, covering the study area. The parameters for the imaging were as follows: TR/TE of 10,000/108 ms, a FOV of 23 × 23 cm, a 128 × 128 matrix, a section thickness of 5 mm, and a 1- to 2-mm gap between sections when they met. Diffusion-probing gradients were used in all 3 orthogonal directions (x, y, and z). With a diffusion-weighted factor, factor b, the MR images were obtained at 0, 400, and 800 s/mm2. They were obtained with a factor of b = 0. Radiologists reviewed DWIs and decided whether they were good enough to be used for more study, paying attention to how susceptibility artifacts could distort the image. Finally, T1WIs (TR/TE of 800/15 ms) were performed after the patients had gadolinium-based contrast injected into their veins. Before giving the patients contrast and getting their permission, the renal function parameters were checked.
New enhancing lesions were in proximity to the primary tumor site, visualized as areas of T2 prolongation that could be edema, fibrous-inflammatory reaction, neoplasm, and different levels of mass effect. While the initial radiologic report used nonstandardized formats because they were studied before 2016, new enhancing lesions were graded and interpreted following the NI-RADS lexicon by experienced readers (20 years’ experience). Our focus in this study was only on NI-RADS 2 and 3 lesions in the primary site.
Using a previously described methodology,11 we manually placed 3 polygonal ROIs on the highest b-value (800) images, either on the same or subsequent axial sections, according to the size and extent of the primary foci, and a head and neck radiologist with 20 years of experience in the interpretation of oral cavity squamous cell carcinoma MR images automatically copied them on the appropriate ADC maps. In the presence of enhancement, ROIs were placed on the most enhancing regions of new enhancing lesions, excluding necrotic areas, using contrast-enhanced T1WI and T2WI in the same axial sections. The average ADC mean value was determined using the 3 measurements for each target.
A review of related literature11,13,16 helped us classify T2WI signal morphology as iso- to hypo-intense, intermediate and hyperintense to muscle signal intensity. Similar to a previous study,11 if the T2WI had an intermediate tumor signal with corresponding diffusion restriction (diffusion restriction was defined as the presence of high signal intensity on DWI [b-value = 800 s/mm2] and low signal intensity on the ADC map in the corresponding tumor, an upgrade was assigned to the NI-RADS category. Similarly, a downgrade was assigned in the absence of both features. The best cutoff for the study population was determined by the Youden index of the receiver operating characteristic (ROC) curve for ADC values. The new NI-RADS taking into account T2WI and DWI (referred to as NI-RADS A for purposes of simplicity) and ADC (choosing the best cutoff yielded by ROC, referred to as NI-RADS B) was documented in an Excel (Microsoft) sheet for calculation of diagnostic accuracy.
Statistical Analysis
SPSS 22.0 (IBM) was used for statistical evaluation. The 2-sided Fisher exact test was used to calculate P values for categoric variables, while the differences in medians test were used to obtain P values for continuous variables.
A Diagnostic Test Calculator (http://araw.mede.uic.edu/cgi-bin/testcalc.pl) was used to calculate specificity, sensitivity, and positive and negative likelihood ratios for different diagnostic categories. The statistical program MedCalc 12.2.1 (MedCalc Software) was used to generate ROC curves and compare the areas under the ROC curves (AUCs). ROC and AUC were used to predict malignancy. The AUC, a measure of the diagnostic accuracy of the individual parameter, is shown with 95% CIs. ROC curves were compared using the DeLong test. A 2-tailed P < .05 was considered statistically significant.
RESULTS
Among the 84 subjects enrolled in the study with a median age of 59 years (range, 32–83 years), 61 had a malignant new enhancing lesion and 23 had benign lesions. The histologic findings of 61 malignant and 23 benign lesions were verified by tissue sampling in 69 subjects and follow-up imaging within 8–12 weeks in the remaining 15 subjects. The follow-up imaging revealed unequivocal progression in 5 cases, which were labeled “malignant,” while resolution in the remaining 10 cases was labeled benign. The histopathology of the benign lesions was granulation tissue (n = 5), fibrous elements (n = 3), abscess (n = 3), and osteoradionecrosis (n = 2). The median time to recurrence in our study was 8 months (range, 5–37 months). Surgery with or without radiation therapy as a part of management for recurrence was performed in 24/61 subjects after discussion in a multidisciplinary tumor board committee. The remaining 30 subjects were managed by radiation with or without chemotherapy, while 7 were provisionally started on oral metronomic therapy.
Thirty-three subjects were assigned ACR NI-RADS 2, while 51 were assigned the NI-RADS 3 category based on the lexicon and previous studies. Our recurrence rate was 90% (46/51) for NI-RADS-3, 45% (15/33) for NI-RADS-2, and 73% (61/84) overall.
The median ADC of lesions was 1.12 × 10−3 mm2/s (range = 0.48–2.18 × 10−3 mm2/s), and T2WI signal morphology was hypointense (n = 10, 11.9%), intermediate (n = 45, 53.5%), and hyperintense (n = 29, 34.5%). DWI was interpreted as positive for restriction in 54 (64.2%) patients. There was a statistically significant difference between the benign and malignant new enhancing lesions concerning the ACR NI-RADS category and DWI, T2WI, and ADC values, as elaborated in Table 1.
Baseline characteristics of study population (n = 84)
The ROC curve produced an AUC that measured the overall diagnostic accuracy (Fig 2), with the best diagnostic performance by DWI (AUC = 0.793). ROC analysis showed an AUC of 0.72 (0.611–0.811 CI) for ADC, with a cutoff ADC mean of >1.3 × 10−3 mm2/s, which showed the highest sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) (88.5%, 56.5%, 84.3%, and 65.1%, respectively). There were 34/37 (92%) correct upgrades and 9 incorrect downgrades in NI-RADS A, with 32/35 (91%) correct upgrades and 8 correct downgrades in NI-RADS B. The addition of T2 signal morphology and ADC yielded higher diagnostic accuracy than ACR NI-RADS. NI-RADS A and NI-RADS B had a higher diagnostic performance (sensitivity, specificity, PPV, NPV, and accuracy of 79.1%, 81.2%, 91.9%, 59.1%, and 79.6%; and 88.9%, 72.7%, 91.4%, 66.7%, and 85.1%, respectively) compared with the ACR NI-RADS (75.4%, 78.3%, 90.1%, 54.5%, and 76.1%). The diagnostic parameters alone and in combination are summarized in Table 2.
ROC comparison curves for ACR NI-RADS, TW2I, DWI, and ADC show the highest diagnostic accuracy with DWI (0.793), followed by ACR NI-RADS (0.768), ADC (0.720), and T2WI (0.536), with a statistically significant difference between DWI and T2WI (P value = .002) and ACR NI-RADS and T2WI (P value = .005).
Diagnostic parameters for different criteria included in the study
DISCUSSION
Our study shows that a definite incremental sensitivity is achieved with concomitant diagnostic accuracy by incorporating T2WI, DWI, and ADC metrics into the existing ACR NI-RADS lexicon descriptors. There was a noticeable improvement in the false-positives, which would help the surgeon obviate unnecessary biopsies, and the false-negatives that would correctly identify actual disease without delay, which is essential to provide the window of salvage opportunities.
Our study had a 72% recurrence rate in our population of asymptomatic patients. The recurrence rate was 54% (46/84) in NI-RADS 3 and 17% (15/84) in NI-RADS 2, similar to findings in a previous study,14 which documented 54.6% in a large subset of 618 subjects. Our study is quintessential because it deals with a subset of the population who were clinically occult and asymptomatic, which was a shortfall in the previous studies.5
In a previous study,5 more than one-third of all imaging-detected recurrences occurred in patients with no clinical symptoms, with a median time of 4–11 months,15 similar to our findings, re-emphasizing the critical nature of extending the current National Comprehensive Cancer Network imaging surveillance guidelines beyond the first 6 months of treatment. It has been found that more than one-third of the recurrences identified on imaging were clinically indistinguishable from recurrences that were clinically occult, and 80% of those occurred >6 months after therapy.5
Previous studies14,15 documented an overall PPV of 54%–56% for NI-RADS 3 posttreatment PET/CT, which was considered low. Compared with these previous studies, our diagnostic parameters using MR imaging had a high PPV (90%). We observed that the PPV did not show any improvement among the DWI, ACR NI-RADS, NI-RADS A, and NI-RADS B diagnostic categories but exhibited a high PPV compared with previous studies using CECT/PET/CT (90% versus 56%). We hypothesize that this high PPV could be due to our institutional imaging protocol because a study found that CECT alone was more likely to correctly identify recurrence than CECT PET (91.7% versus 40.0%).14 However, this finding lowers the NPV of the NI-RADS 2 category, which is undesirable (67% NPV versus 91% PPV), as we found in our results.
NI-RADS favors NPV over PPV to potentially capture treatable lesions, even if it comes at the cost of performing additional biopsies. One possible explanation for our findings was that our experienced readers were overly cautious, favoring specificity over sensitivity. NI-RADS can serve as an assessment tool, with which individual radiologists or groups of radiologists can use the PPV and NPV to evaluate their own performance. We found improved detection of recurrence using ADC and T2WI/DWI parameters in conventional NI-RADS (88% and 79% versus 75%).14 The overall diagnostic accuracy for NI-RADS at the primary site was 0.786, comparable with ours (AUC = 0.768). Although a previous study11 showed higher specificity than ours (90.7 versus 56.5%) for a comparable sensitivity (92% versus 88%) for the diagnostic parameter ADC, it included most nonsurgical candidates not specific to subsite of oral cavity subsite specific patients. Our study highlights the importance of imaging in this subsite, which is more subject to misreading due to the complex altered anatomy in post-surgically treated cases.
Among the 9 false, downgrades in NI-RADS A in our study, 5 subjects were due to superadded infection amounting to abscess formation in 3 and osteoradionecrosis of the mandible in 2, keeping with the previous literature.6 The use of ADC and deploying NI-RADS B correctly identified 5 of these, as depicted in representative cases in Fig 3. Functional imaging like DWI and ADC parameters derived from DWI have shown promise in identifying true disease recurrence with higher specificity than conventional PET/CT, with a reported pooled sensitivity and specificity of 85% and 93% for PET/CT.8 However, in our institution, the cost and radiation17 factors propel the multidisciplinary team to use PET/CT in reserve cases. The cost-effectiveness and overall survival advantage have not been discussed here because they are beyond the scope of this study and will be addressed in future studies.
A post-surgically treated case of left buccal mucosa squamous cell carcinoma in a symptomatic 45-year-old male subject with a mild-to-moderate new enhancing lesion in the deep tissue along the superior margin of the flap occult to clinical examination, interpreted as NI-RADS 2. Corresponding ADC shows low values (<1.3 × 10−3 mm2/s) with intermediate gray T2WI. It was a histopathologically-confirmed malignancy.
One potential limitation in our study was the lack of interreader agreement, but studies18 have found that assigning NI-RADS categories to findings and impressions has moderate-to-solid interreader and intrareader reliability, even when readers with different levels of experience from different institutions read the studies. We could not determine how early imaging can detect recurrences or its effect on an outcome. This determination would need controls that were beyond the scope of the study. Our study focused on the primary site without addressing the neck. Abnormal and palpable lymphadenopathy in a post-surgically treated population by clinical examination would be subjected to sonography-guided tissue sampling for confirmation without MR imaging. Last, although our study has a clinically occult population, some subjects had trismus due to postsurgical and radiation therapy. This would mean an improper or unjust “clinically occult” classification on our part in this subset of patients. Last, further differentiation and suggestions for follow-up imaging for NI-RADS categories 2b and 3 would be suitable, as mentioned in a previous study,10 but we did not have a large enough sample size to arrive at these conclusions.
CONCLUSIONS
Improved diagnostic accuracy was achieved in classic ACR NI-RADS by including MR imaging diagnostic parameters like T2WI, DWI, and ADC, with a gain in sensitivity. Standardization of associated treatment recommendations and their relevance to patient outcomes should validate performance and emphasize the radiologist’s added value in in patient care. Before implementing the NI-RADS imaging template, a standardized approach to develop a consensus surveillance imaging algorithm should be undertaken.
Footnotes
Disclosure forms provided by the authors are available with the full text and PDF of this article at www.ajnr.org.
References
- Received December 21, 2021.
- Accepted after revision May 3, 2022.
- © 2022 by American Journal of Neuroradiology
