Early Adoption of a Certainty Scale to Improve Diagnostic Certainty Communication

Objective: Assess the early voluntary adoption of a certainty scale to improve communicating diagnostic certainty in radiology reports.

Methods: This institutional review board-approved study was part of a multifaceted initiative to improve radiology report quality at a tertiary academic hospital. A committee comprised of radiology subspecialty division representatives worked to develop recommendations for communicating varying degrees of diagnostic certainty in radiology reports in the form of a certainty scale, made publicly available online, which specified the terms recommended and the terms to be avoided in radiology reports. Twelve radiologists voluntarily piloted the scale; use was not mandatory. We assessed proportion of recommended terms among all diagnostic certainty terms in the Impression section (primary outcome) of all reports generated by the radiologists. Certainty terms were extracted via natural language processing over a 22-week postintervention period (31,399 reports) and compared with the same 22 calendar weeks 1 year pre-intervention (24,244 reports) using Fisher's exact test and statistical process control charts.

Results: Overall, the proportion of recommended terms significantly increased from 8,498 of 10,650 (80.0%) pre-intervention to 9,646 of 11,239 (85.8%) postintervention (P < .0001 and by statistical process control). The proportion of recommended terms significantly increased for 8 of 12 radiologists (P < .0005 each), increased insignificantly for 3 radiologists (P > .05), and decreased without significance for 1 radiologist.

Conclusion: Designing and implementing a certainty scale was associated with increased voluntary use of recommended certainty terms in a small radiologist cohort. Larger-scale interventions will be needed for adoption of the scale across a broad range of radiologists.