Abstract
BACKGROUND AND PURPOSE: There is no standardized curriculum currently available at most institutions for establishing procedural competency in trainees performing cervicocerebral angiography. The purpose of this study was to evaluate a simple learning program to supplement the teaching of basic cervicocerebral angiography.
MATERIALS AND METHODS: An 11-session interactive curriculum was implemented covering anatomic, clinical, and radiographic topics for the novice cervicocerebral angiographer. The target learner was the neuroradiology fellow. Data were gathered regarding fellow comfort level on topics relating to cervicocerebral angiography by using a 5-point Likert scale. Improvement in scores on knowledge-based questions after completion of the curriculum was calculated (McNemar test). Trainee-perceived utility of the program was also recorded by using a 5-point Likert scale. Focus sessions were held at the completion of the curriculum to gather feedback regarding the strengths and weaknesses of the program from participants.
RESULTS: Ten subjects were enrolled in this pilot study for 3 years. Topics where participants reported a poor initial comfort level (4 or higher) included selection of injection rates and volumes and reformation of reverse-curve catheters. Trainees demonstrated a statistically significant change in the distribution of scores of 29.3% (49.4%–78.7% correct response rate, P < .0001). The average perceived utility was 1.5 (1 = most useful, 5 = least useful).
CONCLUSIONS: This simple learning program was a useful adjunct to the training of fellows in diagnostic cervicocerebral angiography, resulting in quantitative improvements in knowledge.
ABBREVIATIONS:
- ACGME
- Accreditation Council for Graduate Medical Education
- ASNR
- American Society of Neuroradiology
- SNIS
- Society of NeuroInterventional Surgery
Diagnostic cervicocerebral conventional angiography is a minimally invasive procedure used to delineate intracranial vascular anatomy for a variety of clinical indications. Despite many recent advances in noninvasive neurovascular imaging by using CT and MR imaging techniques, cervicocerebral conventional angiography remains an important tool in the evaluation of neurovascular disease.1⇓⇓⇓–5 This technique is particularly important in evaluating and characterizing vascular lesions before endovascular and/or open operative treatment.3 It is also useful in examining small vessels of the brain and spinal cord—for example, in the setting of central nervous system vasculitis.5
Teaching trainees the knowledge they need to then gain competence in the procedure and to perform well in clinical practice form the Miller pyramid of learning.6 Training in neuroendovascular procedures is quite variable across fellowship programs. Recent work by Bech et al7 showed that the design and implementation of comprehensive endovascular curricula are critical to training yet have been lacking in current training programs. There are currently ongoing efforts by the ASNR to establish both a global rating scale and a comprehensive curriculum for diagnostic cervicocerebral angiography (Hetts S, Mukherji S, Zimmerman RD, personal communication, June 2010). Multiple societies including ACGME, ASNR, SNIS, and the Society of Cardiovascular and Interventional Radiology agree that technical training in the procedure and cognitive training in neurologic and neurovascular anatomy, imaging, and disease are prerequisites for performing and interpreting diagnostic cervicocerebral angiography.8⇓–10
A few studies have shown that the use of interactive training models improves resident performance.11⇓–13 A 2009 study by Yamagami et al14 is the only report on the use of simple plastic cast models in the instruction of visceral angiography. They demonstrated improvement in trainee performance during 5 days of training and positive results transferable to in vivo arteriography.14
Incorporating the use of interactive models into a comprehensive curriculum on diagnostic cervicocerebral angiography addresses both cognitive and technical aspects of training that may enhance trainee education as fellows perform procedures on patients. The purpose of this study was to implement and assess an interactive learning program on diagnostic cervicocerebral conventional angiography targeted at neuroradiology fellows in their sixth or seventh postgraduate year.
Materials and Methods
The study has approval by the internal review board our institution for exemption and is Health Insurance Portability and Accountability Act compliant.
Curriculum Development
An 11-session curriculum on diagnostic cervicocerebral angiography was developed as a short introductory course for neuroradiology fellows. The curriculum covered clinical, anatomic, and radiographic topics relating to cervical and cerebral vascular anatomy and neuroangiography. Each session was designed to last approximately 1 hour, focused on 1 or 2 main learning objectives, and included a summary handout PDF and a didactic lecture. Seven of the 11 sessions also included an interactive component (Table 1).
Session components and topics included in the learning program
Summary handouts were bound into a syllabus and distributed to participants at the beginning of the program. The didactic components consisted of short lectures by using a slide-presentation format (Fig 1). The interactive component involved using angiography materials and supplies as examples (eg, unused catheters and arterial access kits) and simple models to demonstrate anatomy and develop proper hand-positioning and catheter technique. The models used in this curriculum included a commercially available clear acrylic vascular model [Angiogram Sam Plus (with extended carotids) Anatomical Training Model (Lake Forest Anatomicals, Lake Forest, Illinois)] (Fig 2), a clear skull model [Transparent Skull Model (Medical Supplies and Equipment Company, Houston, Texas)] for demonstrating cranial anatomy and fluoroscopic view selection, and simple foam sponges with rubber tubing for practicing arteriotomy, which were developed for the curriculum (Fig 3).
Examples of didactic presentation slides covering an array of neuroangiographic topics.
Clear acrylic vascular model (Lake Forest Anatomicals, Lake Forest, Illinois) at the level of the aortic arch with a wire [Aniogram Sam Plus (with extended carotids) Anatomical Training Model] and catheter placed intraluminally, demonstrating techniques for selection of the major cervical arteries.
Micropuncture common femoral arteriotomy demonstrated using a simple foam sponge model with embedded rubber tubing.
Implementation
The course is offered at the start of the academic medical year with sessions generally given once or twice a week. Neuroradiology fellows enrolled in the neuroradiology fellowship program at our institution participated between 2008 and 2011. The first year of the 2-year fellowship is ACGME-accredited. Data are collected from all fellows enrolled in the course for the first time. The faculty comprised 2 instructors: both attending neuroradiologists with American Board of Radiology Certificate of Added Qualification; 1, a practicing neurointerventionalist.
Trainee Reported Experience before the Course
To assess baseline trainee experience level before the course, trainees subjectively graded their comfort level in performing various parts of a diagnostic cervicocerebral angiogram, including obtaining informed consent, wire/catheter manipulation, and achieving groin hemostasis. Subjects reported their comfort level by using a 5-point Likert scale (1 = most comfortable; 5 = least comfortable).
Assessment of Learning Program
A five-answer multiple-choice question quiz with 59 questions covering subjects from all 11 sessions was administered to trainees before and after completion of the curriculum. The assessment included approximately 5–6 questions pertaining to key concepts from each session to gauge overall comprehension of material. The questions were developed by 3 neuroradiologists, 2 with subspecialty training in neurointerventional radiology. Consensus agreement on clarity and correctness of the questions was reached before incorporation into the curriculum. Both the order of questions and the order of answers were presented in a randomized way on the precourse and postcourse tests. Answers were anonymously recorded by using an audience response system. Participants were encouraged to answer to the best of their ability. Subjects were informed that the purpose of the questions was to assess the learning program, and the results were not used to evaluate individual participants. Participant scores on knowledge-based questions were compared before and after completing the course. Scores before and after training were reported for each session as well as by year of training. Overall percentage improvement was calculated, and the McNemar test was used to assess significant differences between the proportion of correct responses before and after completion of the course.
In addition, for each session, subjects were asked to anonymously rate their perceived utility of each component of the session (handout, didactic lecture, and interactive component if applicable) again by using a 5-point Likert scale (1 = most useful, 5 = least useful). Average perceived utility was then calculated for each session, including all components, and for each component, including all sessions. Average perceived utility <3.0 was considered positive.
At the end of the course, focus sessions were conducted by the instructors in which subjects were asked to comment on the strengths and weaknesses of the learning program and provide feedback. Subjects were aware that the purpose of the focus session was not to assess their performance. All results were also stratified by year of training.
Results
Ten fellows (6 first-year and 4 second-year) participated in the learning program. All participants completed all 11 training sessions.
Trainee-Reported Experience before the Course
Trainees reported an average comfort level of 2.1 (1 = most comfortable, 5 = least comfortable) at performing various portions of the procedure. The reported level of comfort was 2.2 for first-year fellows and 1.9 for second-year fellows. The median and mode of the reported comfort level was 1 for first- and second-year fellows. Topics for which individual fellows reported poor comfort levels (≥4) included selection of injection rates and volumes as well as reformation of a reverse-curve catheter.
First-year fellows reported the least comfort with material presented in sessions 2 and 6 (Gaining Vascular Access, Understanding and Using Fluoroscopic Views) with scores of 2.5 and 2.8, respectively. First-year fellows reported the highest level of comfort with materials presented in sessions 7 and 8 (Obtaining Groin Hemostasis and Clinical Considerations). Second-year fellows reported the least comfort with material presented in session 5 (Wire Catheter Manipulation Part II: Dealing with Difficult Vessels and Complications) and the highest comfort level for information presented in session 9 (A Review of Basic Cerebrovascular Anatomy).
Assessment of Learning Program
After completing the curriculum, subjects demonstrated a significant 29.3% improvement in scores on knowledge-based questions (49.4%–78.7% correct response rate) (McNemar Test, P < .0001). There were no differences when stratified by level of training (Fig 4).
Percentage of correct responses on knowledge-based questions before and after the course, stratified by level of training (A) and session number (B).
On average, all 3 components of the learning program (handout, didactic lecture, interactive task) were found to be useful (<3.0 on a 5-point Likert scale; 1 = most useful, 5 = least useful). Average perceived utility of each component of the learning program was 1.5, 1.5, and 1.4 for the handout, didactic, and interactive components, respectively (Table 2). Overall, fellows found material in sessions 1, 4, 7, and 11 (Evaluating the Patient, Groin Hemostasis, Wire Catheterization, and Spinal Angiography) to be the most useful (Table 3). The median and mode of the reported utility was 1 for first and second-year fellows.
Average fellow-perceived utility of the learning program stratified by type of instruction and level of training
Average fellow-perceived utility of the learning program stratified by subject matter/session and level of training
The results of the focus sessions were overwhelmingly positive, with all participants finding the course well-suited to fellowship-level training. Fellows agreed that the time allotted for covering the material was appropriate. Topics that fellows found particularly useful varied among individuals. Handouts were deemed valuable as both a course syllabus as well as a summary document for future reference. Some fellows found the didactic component the most valuable, citing the value of an organized and comprehensive review. Others found the interactive component the most valuable component of the course, citing the reduced anxiety of first being able to practice hand-positioning and catheter technique in a model. Specific recommendations included reorganizing the curriculum into a more concentrated time period in the first few weeks of fellowship. A suggestion was made to tailor the curriculum on the basis of the trainees' level of general angiography experience from residency. Fellows thought that many parts of the curriculum could also be useful for radiology resident training, including introductory topics, anatomy review, and vascular access.
Discussion
We present a combined didactic and interactive curriculum for diagnostic cervicocerebral conventional angiography. Pilot data show improvement in trainee knowledge across many topics in cerebrovascular imaging, not only technical aspects of cervicocerebral angiography but also cognitive topics, including neurovascular anatomy and basic principles of fluoroscopy. Both first- and second-year fellows reported a high perceived utility of the program.
A 2005 consensus statement from collaborating neuroscience societies including the American Academy of Neurology, American Association of Neurologic Surgeons, SNIS, American Society of Neuroimaging, Congress of Neurologic Surgeons, and Society of Interventional Radiology outlined proposed standards of cognitive and technical training for diagnostic and therapeutic cervicocerebral endovascular procedures. In addition to direct experience performing neuroendovascular procedures, these guidelines recommend a minimum of 6 months of cognitive training in clinical neuroscience specifically, including indications, limitations, and complications for these procedures.9,15 Current ACGME guidelines require 50 catheter-based angiographic procedures, with at least 5 intracranial microcatheter procedures highly recommended.
Providing an organized curriculum to supplement learning under graded supervision in cervicocerebral angiography is an important and useful adjunct to training.7 This is supported by research in radiology residency training,16,17 including training in endovascular procedures.18 In addition, standardization of curricula across institutions would be useful19 because trainees experiences vary widely. There is a recent push to expand global rating scales for procedural competency assessment to include nontechnical skills such as preprocedure planning, understanding and dealing with complications, knowledge of radiation physics and radiation biology, generation of high-quality diagnostic imaging, and interpretation of such images.7 Attention to house staff education positively impacts patient care and is, therefore, an important contribution of academic medicine to health care.7,20
In designing the current learning program, our goal was to create a tool to supplement and not supplant current education of fellows. The program presented here is a simple, inexpensive, and easy-to-implement curriculum that incorporates a combination of didactic lectures and simple models to teach both cognitive and technical skills. Such a tool is easy and feasible to include and implement in an ACGME-accredited neuroradiology fellowship program and may prove useful in helping neuroangiography trainees achieve and document procedural competency. Implementation of such a learning program fits the goal of the ACGME to measure outcomes in trainee education.
Most interesting, there was no difference between improvement in scores on knowledge-based questions for first-year and second-year fellows. At our institution, first-year fellows rotate for 12 weeks concentrating on neuroendovascular procedures, for which they assume primary assistant responsibilities. Despite having, by current standards, a robust first-year fellow experience in neuroangiography, data show benefit to trainees beyond the first year of dedicated neuroradiology training. For the second-year fellows, having had some experience in neuroangiographic procedures may provide both a perspective of relevance and an appreciation of challenges. Material presented within an existing framework of knowledge and experience may have more relevance.
Limitations of this pilot study include the small sample size. Despite this, the differences between scores on knowledge-based questions were highly statistically significant before and after completion of the course. An additional limitation is the lack of a control group because the course was offered to all fellows. Also, evaluation of the benefit from the curriculum was through improvement in knowledge-based questions, and no quantitative predictors of successful procedure performance were measured. Thus far, no long-term follow-up has been accrued. Future directions include making improvements in the curriculum based on feedback from previous participants—for example, highlighting areas of low comfort and high utility. Evaluation of longer term retention of knowledge and validation of procedure performance following training would be helpful to assess the continuing effectiveness of the program.
Conclusions
This learning program on cervicocerebral conventional angiography is easily incorporated into an ACGME-accredited neuroradiology fellowship program, improves fellow scores on knowledge-based questions pertinent to the topic, and is perceived as having high utility by the fellows themselves. The development of learning tools, which include both technical and cognitive aspects of learning such as those presented here, is important to continue to ensure a high level of education for radiology specialty and subspecialty trainees as well as the highest levels of quality and safety for patients.
Acknowledgments
We thank E. Stephen Amis, Jr., MD, for supporting this project from its conception to implementation.
Footnotes
This work had grant support from the Radiological Society of North America Research and Education Foundation through an educational seed grant, ESD0803.
Paper previously presented in part at: Annual Meeting of the Association of University Radiologists, May 12–15, 2009; Arlington, Virginia.
References
- Received July 17, 2011.
- Accepted after revision September 13, 2011.
- © 2012 by American Journal of Neuroradiology
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.