Objective: Administrative databases of hospital admissions are increasingly being used, mostly without validation, for epidemiological and clinical outcomes studies. Although it has been difficult to assess the true accuracy of administrative databases, we have identified an opportunity to directly compare the State of Maryland administrative database against a prospectively maintained departmental database at The Johns Hopkins Hospital.
Methods: Data for patients with the diagnosis of an intracranial aneurysm treated at The Johns Hopkins Hospital over a 17-year period were compared in the State of Maryland administrative database and the neurosurgery departmental database. Discrepancies were clarified by review of the original medical records. The sensitivity, specificity, and positive predictive value of each database were calculated.
Results: The administrative database missed 16% of all cases and was significantly inaccurate in 10 of 12 categories. It had particularly low values in the specificity regarding surgical treatment (67%), the sensitivity regarding endovascular treatment (48%), and the positive predictive value regarding endovascular treatment (30%). By contrast, the lowest score of the departmental database in any category was 97%.
Conclusion: We show that this representative administrative database is significantly flawed. Given the exponentially increasing number of research studies based on administrative databases, the pitfalls of research based solely on these need to be recognized. Strong criteria requiring accurate data validation are critical to justify the conclusions of these studies, regardless of their large numbers and complex statistics.