Brain Tumors Images: Interrater Reliability in Pediatric Features of Visually AcceSAble Rembrandt

BACKGROUND AND PURPOSE: At present, no evidence-based lexicon exists for pediatric intracranial tumors. The Visually AcceSAble Rembrandt Images terminology describes reproducible MR imaging features of adult gliomas for prediction of tumor grade, molecular markers, and survival. Our aim was to assess the interrater reliability of the pre-resection features of Visually AcceSAble Rembrandt Images in pediatric brain tumors. MATERIALS AND METHODS: Fifty consecutive pre-resection brain MR imaging examinations of pediatric intracranial neoplasms were independently reviewed by 3 neuroradiologists. The intraclass correlation coef ﬁ cient for continuous variables and the Krippendorf alpha were used to evaluate the interrater agreement. Subgroup analysis was performed for 30 gliomas. RESULTS: Parameters with almost perfect agreement ( a . .8) included tumor location (F1) and proportion of enhancing tumor (F5). Parameters with substantial agreement ( a ¼ .61 – .80) were side of tumor epicenter (F2), involvement of eloquent brain (F3), enhancement quality (F4), proportion of non-contrast-enhancing tumor (F6), and deep white matter invasion (F21). The other parameters showed either moderate ( a ¼ .41 – .60; n ¼ 11), fair ( a ¼ .21 – .40; n ¼ 5), or slight agreement ( a ¼ 0 – .20; n ¼ 1). Subgroup analysis of 30 gliomas showed almost perfect agreement for tumor location (F1), involvement of eloquent brain (F3), and proportion of enhancing tumor (F5); and substantial agreement for side of tumor epicenter (F2), enhancement quality (F4), proportion of noncon-trast enhancing tumor (F6), cysts (F8), thickness of enhancing margin (F11), and deep white matter invasion (F21). The intraclass correlation coef ﬁ cient for measurements in the axial plane was excellent in both the main group (0.984 [F29] and 0.982 [F30]) and the glioma subgroup (0.973 [F29] and 0.973 [F30]). CONCLUSIONS: Nine features of Visually AcceSAble Rembrandt Images have an acceptable interrater agreement in pediatric brain tumors. For the subgroup of pediatric gliomas, 11 features of Visually AcceSAble Rembrandt Images have an acceptable interrater agreement. The low degree of reproducibility of the remainder of the features necessitates the use of features tailored to the pediatric age group and is likely related to the more heterogeneous imaging morphology of pediatric brain tumors.

T he terminology for Visually AcceSAble Rembrandt Images (VASARI) was created for describing visual and subjective MR imaging features of adult gliomas. 1 It was developed by The Cancer Genome Atlas radiology working group as part of The REpository for Molecular BRAin Neoplasia DaTA (REMBRANDT) project, 2 which was, in turn, established to facilitate collaboration among researchers across the globe.The REMBRANDT data set comprises imaging, clinical, and genetic information from 630 patients with gliomas. 2 VASARI features were developed following review of 88 MR imaging studies from this data set. 1 The goal of the VASARI terminology was to facilitate accurate and reproducible MR imaging interpretations of gliomas across different sites with the help of a controlled terminology incorporating the major subjective features of these tumors.A total of 30 features (F1-F25 [with the exception of F15, which is no longer in use], and F26-F30) form the terminology.4][5][6] Currently, no evidence-based standardized vocabulary exists for imaging features of pediatric brain tumors.We, therefore, aimed to assess the interrater reliability of VASARI features in a sample of consecutive pediatric brain tumors.

MATERIALS AND METHODS
This retrospective study was approved by the institutional research ethics board of the Hospital for Sick Children (REB 1000077073), and informed consent was waived due to the retrospective nature of the study.

Patient Population
A retrospective review of the data base of our institution was performed for patients who had undergone neuro-oncologic brain MR imaging between January 2018 and March 2020.Patients were included according to the following criteria: 1) age younger than 18 years, 2) presence of a de novo diagnosis of an intra-axial tumor on MR imaging, and 3) pretherapeutic status of the intracranial tumor.Exclusion criteria were image degradation by artifacts and incomplete image acquisition.

MR Imaging Technique
MR imaging of all except 4 patients was performed at our institution.MR imaging was performed on a 3T magnet (Magnetom Skyra, Siemens, or Achieva, Philips Healthcare) using a 32-channel head coil or on a 1.5T magnet (Achieva) with pediatric 8-or 16channel head coils.The standardized sequence protocol at our institution included the following sequences: a sagittal 3D T1-weighted sequence with axial and coronal reformats; axial DWI, axial FLAIR, and coronal T2-weighted sequences; an axial SWI or a multiplanar gradient recalled acquisition; a gadolinium-based contrast agentenhanced axial 3D T1-weighted sequence with coronal and sagittal reformats; and a gadolinium-based contrast agent-enhanced coronal T1-weighted spin-echo sequence.Other sequences performed variably for different patients on the basis of suspected tumor pathology or for preoperative purposes were axial TOF-MRA, contrast-enhanced axial FLAIR, and orbital or spinal imaging.For the 4 patients (glial series neoplasms ¼ 2; embryonal tumor with multilayered rosettes ¼ 1; atypical teratoid/rhabdoid tumor ¼ 1) who were referred to our hospital following MR imaging performed at external sites, the images were uploaded onto our institutional PACS server, with all examinations including T1-weighted, T2-weighted, FLAIR, DWI, and contrast-enhanced T1-weighted sequences.

Analysis of the Study Cohort
Three pediatric neuroradiologists (A.B., M.W.W., B.B.E.-W.), blinded to clinical and neuropathologic data, were provided with the study accession IDs in random reading order (randomized in Excel [Microsoft]).These were read independently by each reader.Assessment of 26 pre-resection VASARI features (F1-F14, F16-F25, and F29-F30) was performed on the PACS workstations.The VASARI features for each study were entered into Google Forms (https://www.google.com/forms/about/)created and shared by The Cancer Imaging Archive Team. 1 On completion of all 50 studies by each reader, the data were exported into Excel for further analysis.

Statistical Analysis
All statistical analyses were performed using R (Version 4.0.0;http://www.r-project.org/). 7The sample size (n ¼ 50) was calculated using the kappaSize package in R with the confidence interval approach with binary outcome for a 1-sided 95% confidence interval, 0.2 precision, and an a level of .05.Intraclass correlation coefficients for continuous variables and the Krippendorf alpha 8 were used to evaluate the interrater agreement.Subgroup analysis was performed for the 30 gliomas of the cohort.

Patient Demographics and Diagnoses
A total of 1097 consecutive neuro-oncologic brain MR imaging studies between January 1, 2018, and March 31, 2020, were reviewed for this study.A total of 1018 studies were acquired after initiation of therapy and were not considered for this study.We further excluded 18 studies with MRIs demonstrating image degradation by artifacts and 11 studies with incompletely acquired MRIs.The first 50 patients (mean age, 8.47 [SD, 5.33] years; range, 1-17 years; 28 males) who fulfilled the inclusion criteria were included for further analysis.The cohort consisted of biopsy-proved glial series neoplasms (n ¼ 30), presumed glial series neoplasms (n ¼ 7), biopsyproved medulloblastomas (n ¼ 6), a presumed medulloblastoma (n ¼ 1), biopsy-proved embryonal tumors with multilayered rosettes (n ¼ 2), biopsy-proved atypical teratoid/rhabdoid tumors (n ¼ 2), a biopsy-proved ependymoma (n ¼ 1), and a brain stem tumor that was presumed to be either a diffuse midline glioma or an embryonal tumor with multilayered rosettes (n ¼ 1).

DISCUSSION
In this study, we analyzed the reliability of the VASARI terminology in pediatric brain tumors.In a group comprising 50 consecutive pediatric brain tumors, we found almost perfect interreader agreement for 2 features, substantial agreement for 5 features, and slight-to-moderate agreement for the other features.There was an excellent intraclass correlation coefficient for tumor measurement.In the subgroup comprising 30 consecutive pediatric gliomas, we found almost perfect interreader agreement for 3 features, substantial agreement for 6 features, and slight-to-moderate agreement for the other features.Similar to findings in the main group, an excellent intraclass correlation coefficient was found for tumor measurement.
In the adult population, imaging interpretation using VASARI features has been validated for reproducibility, 3 prediction of molecular subtype, 3,9 and prediction of survival [10][11][12][13] in glioblastoma multiforme; and in predicting astrocytoma grade 5 and molecular markers and survival in diffuse lowgrade gliomas. 4The development of a controlled, reproducible terminology has allowed collaboration in adult glioma imaging across multiple sites. 4,14,15ur results for tumor location are similar to the results of the VASARI research project, 1 in which there was high agreement for this parameter.However, there was relatively less agreement for side of tumor epicenter in both the main and subgroups of our study, possibly due to the higher incidence of midline/off-midline tumors such as medulloblastoma and diffuse midline gliomas in our cohort compared with the adult population.
In our study, in both the main group and the glioma subgroup, we showed higher agreement for ordinal parameters such as proportion of enhancing tumor and proportion of nonenhancing tumor but lower agreement for proportion of necrosis, proportion of edema, T1/FLAIR ratio, and thickness of enhancing margin.The lower interrater agreement for proportion of necrosis compared with the adult study was likely due to the difficulty in differentiating tiny cysts from areas of necrosis (Fig 1).There was also difficulty in differentiating peripheral cysts and necrosis from CSF clefts, especially in posterior fossa tumors.Re-definition of these parameters in the context of pediatric tumors, therefore, needs consideration, with perhaps separate terminologies  for supratentorial and infratentorial tumors.The concept of the T1/FLAIR ratio was developed to differentiate expansive from infiltrative tumors. 1 It is unclear, however, if this parameter has an important role in pediatric tumors.Moreover, there was uncertainty as to whether the FLAIR hyperintense signal resulted from tumor-related mass effect or infiltration, particularly in the posterior fossa.The role of this parameter may, therefore, be useful only in the setting of adult gliomas.Certain lesions had a poorly defined rim with scattered pockets of enhancement or focal nodular peripheral enhancement (Fig 2).This issue likely led to poor agreement for parameter F11 (thickness of enhancing margin).Modifying the definition to account for these enhancement patterns in pediatric brain tumors may help to better characterize this feature and may lead to better interreader agreement.
For nominal parameters, there was near-perfect agreement for tumor location, with substantial agreement for side of tumor epicenter, eloquent brain, enhancement quality, and deep white matter invasion and lower agreement for other parameters such as cysts.In the subgroup of gliomas, however, there was substantial agreement for cysts.As we alluded to earlier, there was disparity in differentiating small cysts from necrosis and also peripherally located cysts from CSF clefts.A unifying definition incorporating fluid-filled structures as a single entity, with necrosis, cysts, and CSF clefts as subcategories, may help in better categorizing these components, especially if tumor location (supratentorial versus infratentorial) is also taken into account.Further evaluation of the cystic component (such as number of cysts, dominant cyst, peripheral cyst, cyst wall characteristics) may also prove useful in pediatric tumors, given the higher proportion of tumors with cystic components such as pilocytic astrocytoma, dysembryoplastic neuroepithelial tumors, and ganglioglioma in this population (Fig 3). 16,17nterpretation of diffusivity parameters in heterogeneous tumoral tissue and in the presence of hemorrhage was also challenging in both groups.Additional challenges encountered, particularly in posterior fossa tumors, were differentiation of cortical involvement from cortical distortion and obscuration from mass effect, and tumors crossing midline."Crossing the midline," by VASARI definition, included tumors that crossed over to the contralateral side via WM/commissural pathways.No clear definition existed, however, for lesions involving the brain stem.Another challenge we faced was ascertaining pial and/or ependymal invasion in instances in which the tumor  reached the pial and or ependymal surface.Clearer definitions (such as pial enhancement contiguous with the tumor) may help improve interrater agreement in pediatric brain tumors.It would, however, still be challenging to assess nonenhancing components of the tumor that reach the pial and ependymal surfaces.
Slight agreement for calvarial remodeling in both groups may be explained by the absence of a VASARI definition for skull base remodeling and the higher variability of the individual skull base morphology in children, because posterior fossa tumors causing secondary osseous changes were rated inconsistently (Fig 4A).In addition, the ability of the calvaria of very young children to expand in response to raised intracranial pressure (due to open fontanelles and sutures) 18 necessitates the need to update this definition for pediatric tumors (Fig 4B ).
There are several limitations to our study.First, the inclusion criteria were all consecutive tumors regardless of tumor pathology.These criteria also included a small number of cases that did not undergo biopsy.Because there is no semantic-based feature set in pediatric brain tumors, and given that in certain situations biopsy is not warranted, we decided to include all consecutive patients with brain tumors diagnosed on imaging.This choice led to a heterogeneous group of tumors with differing tumor biology but also highlighted the need for pediatric-specific and perhaps tumor-or location-specific terminology.We also performed a subgroup analysis only for pediatric gliomas.Second, the sample size was relatively limited but corresponded to our a priori calculations of sample size.

CONCLUSIONS
Although many VASARI features are reproducible in pediatric brain tumors with acceptable interrater agreement, the differing landscape and heterogeneity of pediatric tumors necessitates the use of tailored features, depending on tumor type and location.
Disclosure forms provided by the authors are available with the full text and PDF of this article at www.ajnr.org.

FIG 1 .
FIG 1. Depiction of cysts versus necrosis.Axial T2-weighted (A) and contrast-enhanced axial T1weighted (B) images in a 57-week-old child with atypical teratoid/rhabdoid tumor demonstrate peripherally located "fluid-filled" structures (arrow in A) that do not parallel CSF intensity on T1weighted imaging and do not show irregular rim enhancement (arrow in B), leading to difficulty in characterizing these as cysts or small necrotic pockets as per the VASARI definition.

FIG 2 .
FIG 2. Enhancement characteristics.Axial contrast-enhanced T1-weighted image in a 6-year-old girl with group 4 medulloblastoma (A) shows focal nodular enhancement along the periphery (arrow).Axial contrast-enhanced T1-weighted image in a 13-year-old girl with pilocytic astrocytoma (B) demonstrates spotty central enhancement (arrows) with faint rim enhancement (arrowhead).

FIG 3 .
FIG 3. Cysts in pediatric intracranial tumors.A and B, BRAF V600E-positive astrocytoma.Axial T2-weighted and contrast-enhanced axial T1-weighted images show a dominant cyst with an enhancing mural nodule.The cyst wall does not enhance (arrow).C and D, Pilocytic astrocytoma.Axial T2-weighted and contrast-enhanced axial T1-weighted images show a dominant cyst with an enhancing mural nodule.The cyst wall enhances (arrow).

FIG 4 .
FIG 4. Osseous remodeling.Axial and sagittal T2-weighted images in a 4-month-old infant with a dysembryoplastic infantile ganglioglioma show asymmetric expansion of the calvaria (right .left, arrows in A), which occurs due to open sutures and fontanelles in this age group.Note the bulging anterior fontanelle (arrows in B).