Original ArticleEmpirical Bayes estimates generated in a hierarchical summary ROC analysis agreed closely with those of a full Bayesian analysis
Introduction
The systematic review of the results of primary studies is becoming increasingly important for assessing and summarizing evidence about the accuracy of diagnostic tests. Guidelines for the conduct of such systematic reviews [1], [2] include defining the objective of the review, retrieval of the relevant literature, data extraction, meta-analytic methods for obtaining summary estimates of test accuracy, and investigating reasons for variation in test accuracy across studies.
The aim of the present report is to (i) provide a brief outline of commonly used approaches for the meta-analysis of diagnostic studies and their limitations, (ii) outline the hierarchical summary receiver operating characteristic (HSROC) model and its advantages, and (iii) describe how maximum likelihood estimation can be used to fit the HSROC model using the NLMIXED procedure in SAS [3]. An example is used to illustrate the method and a sample program is provided. The results are compared with the estimates published for the same data using Bayesian Markov chain Monte Carlo (MCMC) methods [4].
Section snippets
Commonly used methods for meta-analysis of diagnostic tests
Most methods used for the meta-analysis of diagnostic studies use a single estimate of sensitivity and specificity derived from each study. Commonly used approaches are here outlined briefly.
HSROC: hierarchical SROC model
An alternative approach for fitting SROC curves has been proposed by Rutter and Gatsonis [19]. It is based on the ordinal logistic regression model of McCullagh [20], [21] that has been used by Tosteson and Begg [22] to fit an ROC curve when data are available at multiple thresholds in a single study. The model allows for asymmetry in the ROC through inclusion of a scale parameter that determines the shape of the ROC. Rutter and Gatsonis [4], [19] have applied this model to the estimation of a
Example: methods for fitting SROC curves applied to comparison of imaging methods for the detection of lymph node metastases in women with invasive cancer of the cervix
The data are taken from a meta-analysis conducted by Scheidler et al. [31] to compare the global accuracy of three types of diagnostic imaging—lymphangiography (LAG), computed tomography (CT), and magnetic resonance (MR)—to detect lymph node metastases in women diagnosed with invasive cancer of the cervix. A total of 36 studies are included in the analysis and provide 46 estimates of test sensitivity and specificity: 17 for LAG, 19 for CT, and 10 for MR. The observed sensitivity and specificity
Conclusions
The HSROC model provides a general framework for the meta-analysis of diagnostic studies. It allows the meta-analyst to investigate heterogeneity between studies while taking into account both within- and between-study variability. The example considered here compares the accuracy of different imaging methods; however, the same approach can also be applied to assess whether the SROC, and the expected operating point on the SROC, differs according to patient or study characteristics (or both).
References (35)
- et al.
Meta-analytic methods for diagnostic test accuracy
J Clin Epidemiol
(1995) Validation of the summary ROC for diagnostic test meta-analysis: a Monte Carlo simulation
Acad Radiol
(2003)- et al.
Meta-analysis of diagnostic tests for acute sinusitis
J Clin Epidemiol
(2000) Meta-analyses of studies of the diagnostic accuracy of laboratory tests: a review of the concepts and methods
Arch Pathol Lab Med
(1998)- et al.
Guidelines for conducting systematic reviews of studies evaluating the accuracy of diagnostic tests
SAS/STAT user's guide. Version 8
(1999)- et al.
A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations
Stat Med
(2001) Systematic reviews of evaluations of diagnostic and screening tests
- et al.
Designing studies to ensure that estimates of test accuracy are transferable
BMJ
(2002) - et al.
Comparative diagnostic performance of three radiological procedures for the detection of lumbar disk herniation
Methods Inf Med
(1990)
Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations
Stat Med
Estimating diagnostic accuracy from multiple conflicting reports: a new meta-analytic method
Med Decis Making
Meta-analysis of Pap test accuracy
Am J Epidemiol
Problems of spectrum and bias in evaluating the efficacy of diagnostic tests
N Engl J Med
Exploring sources of heterogeneity in systematic reviews of diagnostic tests
Stat Med
Meta-analysis for 2 × 2 tables: a Bayesian approach
Stat Med
A meta-analytic method for summarizing diagnostic test performances: receiver-operating-characteristic-summary point estimates
Med Decis Making
Cited by (165)
A new method for synthesizing test accuracy data outperformed the bivariate method
2021, Journal of Clinical EpidemiologyDouble-Negative Results Matter: A Reevaluation of Sensitivities for Detecting SARS-CoV-2 Infection Using Saliva Versus Nasopharyngeal Swabs
2024, American Journal of Epidemiology