The Updated Neuroradiology Milestones: Synapsing from 1.0 to 2.0

SUMMARY: The Accreditation Council for Graduate Medical Education is currently in the process of specialty-by-specialty revision of the Milestones. As a result, the Neuroradiology Milestones 2.0 Workgroup was created to refine a system of competency-based assessments for fellow educational growth and development. Strengths of the new Milestones include decreased complexity and uniformity within a subcompetency as it relates to a specific educational development trajectory. The Supplemental Guide serves to decrease clutter in the Milestones 2.0 document and provides a more practical resource for guidance. This article serves to review the history of the Neuroradiology Milestones, followed by a summary of the timeline of events and discussions of the workgroup for development of Neuroradiology Milestones 2.0 and a synopsis of major changes. The plan is for the updated Neuroradiology Milestones to take effect in 2021 or 2022 based on public commentary.

F irst introduced in 1999, the Accreditation Council for Graduate Medical Education (ACGME) and American Board of Medical Specialties jointly approved 6 core competencies to begin an outcomes-based framework to enhance medical education. The goal was to certify physicians based on training outcomes rather than duration spent in training, thus improving the quality of graduate medical education. 1 The 6 core competencies included: patient care and procedural skills (PC), medical knowledge (MK), systems-based practice (SBP), practice-based learning and improvement (PBLI), professionalism (PROF), and interpersonal and communication skills (ICS). 2 These were launched as the Outcomes Project in 2001. 3 The aim was to avoid overspecialization and provide key developmental areas common to all specialties. 2 Unfortunately, residency and fellowship programs struggled with implementation of the competencies due to uncertainty about how to integrate the competencies into evaluation models already in place, as well as differences in expectations for each specialty. As a result, in 2009, the ACGME introduced the Milestones, which organized subcompetencies under the 6 core competencies, providing a continuum of outcome guidelines to effectively assess the competency level of a trainee and follow growth and progress through a residency or fellowship training program. The Dreyfus model of skill development was utilized as a framework to assess the level of development of competencies and skills of trainees learning new skills, ideas, and concepts. 2 Subcompetencies were selected to address either a discipline-specific element or disease process to translate core competencies into specialty-or subspecialty-specific competencies. 4 It was expected that individual programs have a set of tools to assess a trainee's progress in achieving each milestone. Under this system, the program's clinical competency committee is tasked with determining each trainee's level of performance by using evaluations and other assessment tools twice a year and reporting results to the ACGME. 5 It is important to understand that the Milestones represent the essence of a discipline, but it is up to the individual program to develop a robust curriculum, as well as assessment tools to evaluate each trainee's progress. The Milestones are not intended to be used as evaluation forms, but rather a guide for trainee selfassessment by providing explicit expectations for learners, as well as for curriculum development, and assessment of training methods. 6

Harmonized Milestones
The Milestones were not intended to remain a document set in stone, but rather to be reviewed and revised as adequate feedback is obtained, graduate medical education evolves, and the specialty advances and changes. Experts from each specialty or subspecialty creating subcompetencies resulted in marked variability between specialties and subspecialties with respect to content and how educational progression was operationalized across Milestone levels. 6 In 2016 at the 2nd ACGME Milestones Summit, stakeholders from all ACGME core specialties and subspecialties expressed dissatisfaction with inconsistencies in the Milestones and subcompetencies. For example, the subcompetencies tended to detail a variety of differing skills across milestone levels, rather than describing the expected spectrum of growth from beginning to advanced resident or fellow of one particular skill. Additional problems that needed to be addressed were that there were too many subcompetencies for some specialties, the language at times was too complex, some of the guidelines were not practical, and there was too much material in a milestone set. It was reported that these differences had "complicated efforts to share assessment tools across programs and provide comprehensive faculty development across specialties." 7 In other words, the Milestones needed to have more national stakeholder input and overall simplification.
To evaluate and address these concerns, the ACGME Department of Research, Milestone Development, and Evaluation reviewed data collected from the field, data submitted for the biannual milestone reporting, as well as learning from quantitative and qualitative research. Analysis of non-PC and non-MK domains showed marked variation between 26 core specialties and the transitional year; there were more than 230 different ways of describing PROF, 171 for PBLI, 176 for ICS, and 122 for SBP. 8 As a result, the ACGME created 4 stakeholder groups consisting of content experts, directors, interprofessional team members, and other faculty; each group was tasked to develop 2 to 3 subcompetencies that applied to all specialties and subspecialties, thus creating Harmonized Milestones. Terminology in these Harmonized Milestones could be modified if needed to pertain to a particular specialty or subspecialty during the Milestones 2.0 workgroup meetings.

Original Neuroradiology Milestones
The Neuroradiology Fellowship Milestones were initially published in February 2014, followed by a minor edit to the introductory pages in July 2015; however, the content remained the same. 9 The original task force, chaired by Jim Anderson, MD, consisted of a panel of experts from the ACGME Radiology Review Committee, the American Board of Radiology, the American Society of Neuroradiology, and additional volunteers. The group met over 2 days at the ACGME headquarters in Chicago. There were 14 separate subcompetencies included in the initial document that were based in the 6 core competencies (Table). The workgroup focused on those milestones that were specific to neuroradiology training under the PC and MK domains. Those milestones within SBP, PBLI, PROF, and ICS were modelled after the Diagnostic Radiology Residency Milestones, which had been published in December 2012. Following the initial draft and edits, the document was posted for public commentary, approved, and published approximately 1 year later.
The main goal of the original task force was to create a list of subcompetencies that were applicable to all neuroradiology fellowships and representative of the expected abilities of graduating fellows. There was a general interest to keep the number to a minimum, which ultimately led to grouping of subcompetencies under larger headings.
The major criticism of the Neuroradiology Milestones 1.0 is the challenge in assessing a learner in a Milestone when more than one element is included. For example, "Contrast Agents," "Radiation Safety," and "Magnetic Resonance Safety" are all included as separate elements under Patient Care 3 in Milestones 1.0. The more elements represented within a single Milestone has made it challenging for programs to accurately assess its learners. The other major shortcoming of the process is inherent in applying a 5-point scale to show Milestone progression in a 1-year fellowship with midpoint and end of year assessment time points.

Development of Neuroradiology Milestones 2.0
The Neuroradiology Milestones 2.0 workgroup was created with the intention of involving neuroradiologists from various backgrounds; as such, representatives were appointed by the ACGME Review Committee and American Board of Radiology, and there was a call for volunteers in late 2019; a total of 9 neuroradiologists and the ACGME Vice President of Milestones Development were part of the working group. The original plan was to have 3 meetings in Chicago, with a target implementation date of the Milestones 2.0 in either 2021 or 2022, based on public commentary. However, due to the COVID-19 pandemic, the ACGME converted from in-person to virtual meetings. The group work was extended to 4 virtual meetings with individual or small group work between meetings. As with in-person meetings, the virtual meetings included development and thorough analysis of the Milestones and Supplemental Guide and individual off-line reviews for accuracy and agreement on the draft.
The first meeting was August 17, 2020, followed by August 18, October 20, and November 5. Before the first meeting, reading materials were provided to serve as background and a basis for discussion in revising the Milestones. Meetings 1 and 2 focused on creating a shared mental model of the task, then selecting topics important to the graduate of 2025 and deciding if they were at the level of a subcompetency, a theme within a subcompetency, something to be used as an example, or if it was not something to be evaluated regularly. The group also started developing subcompetencies. Before meeting 3, the group reviewed the changes discussed and made edits as necessary. Meeting 3 was utilized to complete development of the Milestones and work on the Supplemental Guide. At the end of meeting 3, the team was divided into 4 groups (3 groups of 2 and 1 group of 3) to work on refining or developing examples within each subcompetency, describing the expected growth from beginning to advanced fellow of a particular skill. These smaller groups also reviewed or updated suggested reading material and references included in the Supplemental Guide. The Neuroradiology Milestones 2.0 workgroup was tasked with several areas of improvement to focus upon. First, since many evaluators utilize both the Radiology Milestones and the Neuroradiology Milestones, the workgroup aimed to revise the Neuroradiology Milestones 2.0 such that they align with the Radiology Milestones 2.0 put into effect several months before the Neuroradiology workgroup convened. Radiology Milestones 2.0 was used as a starting point in writing the Neuroradiology Milestones 2.0, which results in commonalities in language and organizational structure between the documents, thus harmonizing and simplifying the radiology specialty and neuroradiology subspecialty Milestones documents. An effort was made to keep the overall number of milestones manageable while still capturing the core assessable components of neuroradiology practice and fellowship training. The Radiology Milestones 2.0 contains 24 milestones, and the Neuroradiology Milestones 2.0 was expanded from the original 14 milestones to 21 milestones. The differences between Radiology Milestones 2.0 and Neuroradiology Milestones 2.0 are primarily under the MK competency. MK2, MK3, and MK4 of the Radiology Milestones 2.0, which assess application of physics to imaging, appropriate selection of protocol and contrast, and knowledge of imaging technology, respectively, were combined to form MK2 of the Neuroradiology Milestones 2.0, assessing knowledge of all 3 topics in a format appropriate for an advanced subspecialty trainee.
The workgroup assessed the current and projected future practice of neuroradiology and training to ensure that the Milestones remain relevant. Using the first Neuroradiology Milestones as a starting point, we identified the most important areas concerning the MK and PC competencies that needed to be included in the Neuroradiology 2.0 Milestones. The increasing adoption of lexicons and structured reporting, skills reflecting the importance of the role of the neuroradiologist as a consultant to subspecialized clinicians, and the need to apply concepts regarding imaging physics, image processing, and image acquisition to optimize imaging were incorporated into Neuroradiology Milestones 2.0 as changes from the original version. The non-PC, non-MK milestones are derived from the harmonized milestones across all medical specialties in the Neuroradiology Milestones 2.0 document, with additional milestones specific to medical imaging added to the Radiology and Neuroradiology SBP domain (SBP5, SBP6, SBP7).
The workgroup simplified the Milestones by reducing the number of rows within each milestone set and the complexity of the milestone wording. Subcompetencies were limited to no more than 3 rows, with most only containing 1 or 2. Also, within each subcompetency, there are no longer any random milestones (evaluation or introduction of a different skill at a different level within a particular subcompetency), meaning that each subcompetency follows expected growth of 1 particular skill from level 1 to level 5 (Figure). While limiting the number of subcompetencies led to an increase in overall number of Milestones, the new Milestones create concise benchmarks. Often with the original milestones, the separate subcompetencies may have been attained at different levels of training for individual trainees. This led to difficulties in assigning a single Milestone level to any given trainee. With the more granular milestones, while there are overall more milestones to score by the Clinical Competency Committee, the time spent deciding between Milestone levels will be decreased due to the uniformity of each Milestone.
The Supplemental Guide for Neuroradiology Milestones 2.0 is a separate document available through the ACGME website that replaces the footnotes at the bottom of the original Neuroradiology Milestones. The Supplemental Guide was created to provide insights into the intent of the subcompetencies, with examples for each level, sample assessment methods, and other resources. The Supplemental Guide for the Radiology Milestones 2.0 was also used as a starting point for the Neuroradiology Milestone 2.0 Supplemental Guide, with general radiology clinical examples replaced with neuroradiology subspecialty-specific content.

CONCLUSIONS
The ACGME is currently in the process of specialty-by-specialty revision of the Milestones. The Neuroradiology Milestones 2.0 Workgroup is part of this project and has taken a step forward in refining a system of competency-based assessments for fellow educational growth and development. Strengths of the new Milestones include improved usability, decreased complexity, less redundancy, and uniformity within a subcompetency as it relates to a specific educational development trajectory. The Supplemental Guide serves to decrease clutter in the Milestones 2.0 document and provides a more practical resource for guidance and examples of expected progress within a subcompetency. These milestones will be reviewed and updated as necessary with a plan for Milestones 3.0 in an appropriate time period after national experience and stakeholder feedback and comments have been obtained.