Qualifying Certainty in Radiology Reports through Deep Learning–Based Natural Language Processing

References

1. 1.↵
   American College of Radiology. ACR practice guidelines for communication of diagnostic imaging findings. 2014. Revised 2020. https://www.acr.org/-/media/acr/files/practice-parameters/communicationdiag.pdf. Accessed January 7, 2019
2. 2.↵
   1. Swensen SJ,
   2. Johnson CD

   . Radiologic quality and safety: mapping value into radiology. J Am Coll Radiol 2005;2:992–1000 doi:10.1016/j.jacr.2005.08.003 pmid:17411982

   CrossRefPubMed
3. 3.↵
   1. Lafortune M,
   2. Breton G,
   3. Baudouin JL

   . The radiological report: what is useful for the referring physician? Can Assoc Radiology J 1988;39:140–43 pmid:2967838

   PubMed
4. 4.↵
   1. Lee B,
   2. Whitehead MT

   . Radiology reports: What YOU Think You’re Saying and What THEY Think You’re Saying. Curr Probl Diagn Radiol 2017;46:186–95 doi:10.1067/j.cpradiol.2016.11.005 pmid:28069356

   CrossRefPubMed
5. 5.↵
   1. Gunn AJ,
   2. Tuttle MC,
   3. Flores EJ, et al

   . Differing interpretations of report terminology between primary care physicians and radiologists. J Am Coll Radiol 2016;13:1525–29.e1 doi:10.1016/j.jacr.2016.07.016 pmid:27595196

   CrossRefPubMed
6. 6.↵
   1. Rosenkrantz AB

   . Differences in perceptions among radiologists, referring physicians, and patients regarding language for incidental findings reporting. AJR Am J Roentgenol 2017;208:140–43 doi:10.2214/AJR.16.16633 pmid:27657356

   CrossRefPubMed
7. 7.↵
   1. Panicek DM,
   2. Hricak H

   . How sure are you, doctor? A standardized lexicon to describe the radiologist’s level of certainty. AJR Am J Roentgenol 2016;207:2–3 doi:10.2214/AJR.15.15895 pmid:27065212

   CrossRefPubMed
8. 8.↵
   1. Hoang JK

   . Do not hedge when there is certainty. J Am Coll Radiol 2017;14:5 doi:10.1016/j.jacr.2016.08.027 pmid:27889193

   CrossRefPubMed
9. 9.↵
   1. Pons E,
   2. Braun LM,
   3. Hunink MG, et al

   . Natural language processing in radiology: a systematic review. Radiology 2016;279:329–43 doi:10.1148/radiol.16142770 pmid:27089187

   CrossRefPubMed
10. 10.↵
    1. Peng Y,
    2. Wang X,
    3. Lu L, et al

    . NegBio: a high-performance tool for negation and uncertainty detection in radiology reports. AMIA Jt Summits Transl Sci Proc 2018;2017:188–96 pmid:29888070

    PubMed
11. 11.↵
    1. Agarwal S,
    2. Yu H

    . Detecting hedge cues and their scope in biomedical text with conditional random fields. J Biomed Inform 2010;43:953–61 doi:10.1016/j.jbi.2010.08.003 pmid:20709188

    CrossRefPubMed
12. 12.↵
    1. Light M,
    2. Qiu XY,
    3. Srinivasan P

    . The Language of Bioscience: Facts, Speculations, and Statements In Between. HLT-NAACL 2004 Workshop: Linking Biological Literature, Ontologies and Databases. Association for Computational Linguistics 2004:17–24.
13. 13.↵
    1. Shatkay H,
    2. Pan F,
    3. Rzhetsky A, et al

    . Multi-dimensional classification of biomedical text: toward automated, practical provision of high-utility text to diverse users. Bioinformatics 2008;24:2086–93 doi:10.1093/bioinformatics/btn381 pmid:18718948

    CrossRefPubMedWeb of Science
14. 14.↵
    1. Vincze V,
    2. Szarvas G,
    3. Móra G, et al

    . Linguistic scope-based and biological event-based speculation and negation annotations in the BioScope and Genia Event corpora. J Biomed Semantics 2011;2(Suppl 5):S8 doi:10.1186/2041-1480-2-S5-S8 pmid:22166355

    CrossRefPubMed
15. 15.↵
    1. Sorin V,
    2. Barash Y,
    3. Konen E, et al

    . Deep learning for natural language processing in radiology: fundamentals and a systematic review. J Am Coll Radiol 2020;17:639–48 doi:10.1016/j.jacr.2019.12.026 pmid:32004480

    CrossRefPubMed
16. 16.↵
    1. South BR,
    2. Shen S,
    3. Leng J, et al

    . A prototype tool set to support machine-assisted annotation. In: Proceedings of the 2012 Workshop on Biomedical Natural Language Processing, Montreal, Canada. June 8, 2012:130–39
17. 17.↵
    1. Uzuner Ö,
    2. South BR,
    3. Shen S, et al

    . 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text. J Am Med Inform Assoc 2011;18:552–56 doi:10.1136/amiajnl-2011-000203 pmid:21685143

    CrossRefPubMed
18. 18.↵
    1. South BR,
    2. Mowery D,
    3. Suo Y, et al

    . Evaluating the effects of machine pre-annotation and an interactive annotation interface on manual de-identification of clinical text. J Biomed Inform 2014;50:162–72 doi:10.1016/j.jbi.2014.05.002 pmid:24859155

    CrossRefPubMed
19. 19.↵
    Medical diagnosis. Wikipedia. 2019. https://en.wikipedia.org/w/index.php?title=Medical_diagnosis&oldid=900598243. Accessed June 15, 2019
20. 20.↵
    1. Landis JR,
    2. Koch GG

    . The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74 pmid:843571

    CrossRefPubMedWeb of Science
21. 21.↵
    1. Young T,
    2. Hazarika D,
    3. Poria S, et al

    . Recent trends in deep learning based natural language processing. IEEE Comput Intell Mag 2018:13:55–75 doi:10.1109/MCI.2018.2840738

    CrossRef
22. 22.↵
    1. Kim Y

    . Convolutional Neural Networks for Sentence Classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar. October 25–29, 2014:1746–51 doi:10.3115/v1/D14-1181

    CrossRef
23. 23.↵
    1. Liu P,
    2. Qiu X,
    3. Huang X

    . Recurrent neural network for text classification with multi-task learning. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, New York, New York. July 9–15, 2016:2873–79
24. 24.↵
    1. Tan C,
    2. Sun F,
    3. Kong T, et al

    . A survey on deep transfer learning. arXiv 2018 http://arxiv.org/abs/1808.01974. Accessed May 24, 2019
25. 25.↵
    1. Devlin J,
    2. Chang MW,
    3. Lee K, et al

    . BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Minnesota. June 2–7, 2019:4171–86 doi:10.18653/v1/N19-1423

    CrossRef
26. 26.↵
    1. Vaswani A,
    2. Shazeer N,
    3. Parmar N, et al

    . Attention is all you need. In: Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, California. December 4–9, 2017:5998–6008
27. 27.↵
    1. Bird S,
    2. Klein E,
    3. Loper E

    . Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit. O’Reilly Media, Inc; July 21, 2009
28. 28.↵
    1. Wu Y,
    2. Schuster M,
    3. Chen Z, et al

    . Google’s Neural Machine Translation system: bridging the gap between human and machine translation. arXiv 2016. http://arxiv.org/abs/1609.08144. Accessed April 16, 2019
29. 29.↵
    1. Srivastava N,
    2. Hinton G,
    3. Krizhevsky A, et al

    . Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 2014;15:1929–58

    CrossRef
30. 30.↵
    1. Lee J,
    2. Yoon W,
    3. Kim S, et al

    . BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 2020;36:1234–40 doi:10.1093/bioinformatics/btz682 pmid:31501885

    CrossRefPubMed
31. 31.↵
    1. Alsentzer E,
    2. Murphy J,
    3. Boag W, et al

    . Publicly available clinical BERT embeddings. Proceedings of the 2nd Clinical Natural Language Processing Workshop, Minneapolis, Minnesota. 2019;72–78 doi:10.18653/v1/W19-1909

    CrossRef
32. 32.↵
    1. Johnson AE,
    2. Pollard TJ,
    3. Shen L, et al

    . MIMIC-III: a freely accessible critical care database. Sci Data 2016;3:160035 doi:10.1038/sdata.2016.35 pmid:27219127

    CrossRefPubMed
33. 33.↵
    1. Shinagare AB,
    2. Alper DP,
    3. Hashemi R, et al

    . Early adoption of a certainty scale to improve diagnostic certainty communication. J Am Coll Radiol 2020;17:1276–84 doi:10.1016/j.jacr.2020.03.033 pmid:32387371

    CrossRefPubMed
34. 34.↵
    1. Li R,
    2. Hu B,
    3. Liu F, et al

    . Detection of bleeding events in electronic health record notes using convolutional neural network models enhanced with recurrent neural network autoencoders: deep learning approach. JMIR Med Inform 2019;7:e10788 doi:10.2196/10788 pmid:30735140

    CrossRefPubMed
35. 35.↵
    1. Tang YP,
    2. Li GX,
    3. Huang SJ

    . ALiPy: active learning in Python. arXiv 2019. http://arxiv.org/abs/1901.03802. Accessed June 17, 2019
36. 36.↵
    1. Tzeng E,
    2. Hoffman J,
    3. Saenko K, et al

    . Adversarial discriminative domain adaptation. arXiv 2017. https://arxiv.org/abs/1702.05464. Accessed May 20, 2019