Elsevier

Academic Radiology

Volume 7, Issue 6, June 2000, Pages 413-419
Academic Radiology

Original Investigation
Bootstrap estimation of diagnostic accuracy with patient-clustered data

https://doi.org/10.1016/S1076-6332(00)80381-5Get rights and content

Rationale and Objectives.

The purpose of this study was to describe a simple bootstrap approach for estimating sensitivity, specificity, and the area under the receiver operating characteristic curve for multisite test outcome data.

Materials and Methods.

The performance of bootstrap estimates was evaluated and compared with that of analytic estimates by using a simulation study. Bootstrapping was demonstrated by using data from a previous study comparing two angiographic methods.

Results.

Analytic and bootstrap estimates had similar coverage rates for 95% confidence intervals. With many sites per patient, bootstrap estimates had slightly better coverage than analytic estimates. Bootstrap percentile intervals had better coverage than asymptotic normal bootstrap intervals.

Conclusion.

Bootstrapping is a useful method of estimating confidence intervals for the area under the receiver operating characteristic curve, sensitivity, and specificity when data are correlated.

References (12)

  • ZerhouniEA et al.

    CT and MRI imaging in the staging of colorectal carcinoma: report of the Radiologic Diagnostic Oncology Group II

    Radiology

    (1996)
  • BaumRA et al.

    Multi-center trial to evaluate peripheral vascular magnetic resonance angiography

    JAMA

    (1995)
  • SongHH

    Analysis of correlated ROC areas in diagnostic testing

    Biometrics

    (1997)
  • LeisenringW et al.

    A marginal regression modeling framework for evaluating medical diagnostic tests

    Stat Med

    (1997)
  • LipsitzLR et al.

    Analyzing correlated binary data using SAS

    Comput Biomed Res

    (1990)
  • ObuchowskiNA

    Nonparametric analysis of clustered ROC curve data

    Biometrics

    (1997)
There are more references available in the full text version of this article.

Cited by (81)

  • Outcome of LR-3 and LR-4 observations without arterial phase hyperenhancement at Gd-EOB-DTPA-enhanced MRI follow-up

    2020, Clinical Imaging
    Citation Excerpt :

    However, there were not correlated lesions because all observations within the same patient had the same category at follow-up. Therefore, exact Clopper-Pearson confidence intervals were used to calculate sensitivity and specificity, while standard logit confidence intervals were used to calculate PPV and NPV [17]. In order to assess if major or ancillary features were associated with outcome of LR-3 and LR-4 observations, the Chi-square test and Fischer's exact test were used.

  • Automatic classification of pulmonary peri-fissural nodules in computed tomography using an ensemble of 2D views and a convolutional neural network out-of-the-box

    2015, Medical Image Analysis
    Citation Excerpt :

    Furthermore, the ensemble with SVM performs better than BoF in the region of the curve with specificity > 0.9. In order to evaluate the statistical significance of the difference in performance for the considered approaches, we applied bootstrap to the ROC curves and tested the statistical significance considering 95% of confidence interval, as described by Rutter (2000). As a result, the three considered approaches did not show any significant difference in performance: BoF vs SVM (p-Value = 0.255), BoF vs RF (p-Value = 0.132), SVM vs RF (p-Value = 0.258).

View all citing articles on Scopus
View full text