Skip to main content
Log in

QRSE: a Novel Metric for the Evaluation of Trainee Radiologist Reporting Skills

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Diagnostic radiology training programs must produce highly skilled diagnostic radiologists capable of interpreting radiological examinations and communicating results to clinicians. Established training performance tools evaluate interpretive skills, but trainees' competency in reporting skills is also essential. Our semi-automated passive electronic tool entitled the Quantitative Reporting Skills Evaluation (QRSE) allows radiology training programs to evaluate the quantity of edits made to trainee preliminary reports by attending physicians as a metric to evaluate trainee reporting performance. Consecutive report pairs and metadata extracted from the radiology information system were anonymized and exported to a MySQL database. To perform the QRSE, for each report pair, open source software was first utilized to calculate the Levenshtein Percent (LP), the percent of character changes required to convert each preliminary report to its corresponding final report. The average LP (ALP), ALP for each trainee, and standard deviations were calculated. Eighty-four trainees and 56 attending radiologists interpreted 228,543 radiological examinations during the study period. The overall ALP was 6.38 %. Trainee-specific ALPs ranged from 1.1 to 15.3 %. Among trainee-specific ALPs, the standard deviation was 3.7 %. Our analysis identified five trainees with trainee-specific ALPs above 2 standard deviations from the mean and 14 trainees with trainee-specific ALPs less than 1 standard deviation below the mean. The QRSE methodology allows for the passive, quantitative, and longitudinal evaluation of the reporting skills of trainees during diagnostic radiology residency training. The QRSE identifies trainees with high and low levels of edits to their preliminary reports, as a marker for trainee overall reporting skills, and thus represents a novel performance metric for radiology training programs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. American College of Radiology: ACR standard for communication: diagnostic radiology. In: Standards. Reston: American College of Radiology, 1999–2000, pp. 1–3

  2. Goodman NW, Edwards MB: Medical Writing: A Prescription for Clarity. Cambridge Univ. Press, New York, 1991

    Google Scholar 

  3. Hall FM: Language of the radiology report: primer for residents and wayward radiologists. Am J Roentgenol 175:1239–1242, 2000

    Article  CAS  Google Scholar 

  4. Hunter TB: Radiographic reports: structure and review. (letter). Am J Roentgenol 142:647–648, 1984

    Article  CAS  Google Scholar 

  5. American College of Radiology: The American College of Radiology DXIT™ & TXIT™ in-training examinations. Downloaded from http://www.acr.org/Education/Exams-Certifications/DXIT-TXIT on July 22, 2012

  6. American Board of Radiology: The American Board of Radiology initial certification process. Downloaded from http://www.theabr.org/ic-landing on July 22, 2012

  7. Google-diff-match-patch. Downloaded from http://code.google.com/p/google-diff-match-patch/ on June 1, 2011

  8. Carney E, Kempf J, DeCarvalho V, Yudd A, Nosher J: Preliminary interpretations of after-hours CT and sonography by radiology residents versus final interpretations by body imaging radiologists at a level 1 trauma center. AJR Am J Roentgenol 181:367–373, 2003

    Article  PubMed  Google Scholar 

  9. Wysoki MG, Nassar CJ, Koenigsberg RA, Novelline RA, Faro SH, Faerber EN: Head trauma: CT scan interpretation by radiology residents versus staff radiologists. Radiology 208:125–128, 1998

    PubMed  CAS  Google Scholar 

  10. Tieng N, Grinberg D, Li SF: Discrepancies in interpretation of ED body computed tomographic scans by radiology residents. Am J Emerg Med 25:45–48, 2007

    Article  PubMed  Google Scholar 

  11. Meyer RE, Nickerson JP, Burbank HN, Alsofrom GF, Linnell GJ, Filippi CG: Discrepancy rates of on-call radiology residents interpretations of CT angiography studies of the neck and circle of Willis. AJR Am J Roentgenol 193:527–532, 2009

    Article  PubMed  Google Scholar 

  12. Itri JN, Kim W, Scanlon MH: Orion: a web-based application designed to monitor resident and fellow performance on-call. J Digit Imaging 24(5):897–907, 2011

    Article  PubMed  Google Scholar 

  13. Itri JN, Redfern RO, Scanlon MH: Using a web-based application to enhance resident trianing and improve performance on-call. Acad Radiol 17(7):917–920, 2009

    Article  Google Scholar 

  14. Sharpe Jr, RE, Surrey D, Gorniak RJ, Nazarian L, Rao VM, Flanders AE: Radiology report comparator: a novel method to augment resident education. J Digit Imaging 25(3):330–336, 2012

    Article  PubMed  Google Scholar 

  15. Baghdanian A, Sharpe RE, Gorniak RJ, Nazarian LN, Rao VA, Flanders A: Quantitative metric to evaluate the clinical significance of changes made to trainee radiology reports. 2012 Annual Meeting of the Association of University Radiologists. March 21, 2012 (SS01-01). Downloaded from http://www.aur.org/Annual_Meeting/upload/Abstracts.pdf on July 22, 2012

  16. ACGME: ACGME core competencies. Downloaded from http://www.acgme.org/acwebsite/RRC_280/280_corecomp.asp on June 1, 2012

  17. Collins J, Rosado de Christenson M, Gray L, Hyde C, Koeller KK, Laine F, et al: General competencies in radiology training: definitions, skills, education and assessment. Academic Radiology 9(6):L721–L726, 2002

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Richard E. Sharpe Jr..

Rights and permissions

Reprints and permissions

About this article

Cite this article

Surrey, D., Sharpe, R.E., Gorniak, R.J.T. et al. QRSE: a Novel Metric for the Evaluation of Trainee Radiologist Reporting Skills. J Digit Imaging 26, 678–682 (2013). https://doi.org/10.1007/s10278-013-9574-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-013-9574-y

Keywords

Navigation