Original articleSelf-Assessment for Maintenance of Certification
Section snippets
Why self-assessment?
The literature has a number of concise descriptions of self-assessment. Here are 2 of the best, eloquently stated in each authors’ own words:
A key attribute of an effective learner is the ability to critically analyze one’s achievements and progress … in the full cycle of action, reflection, evaluation, and further planning for continued improvement [1].
Good [practitioners] have always assessed or evaluated their own performances. They observe the consequences of their actions, listen to
ABR’S approach to self-assessment
In the ABR’s Maintenance of Certification program, self-assessment is part of the second of 4 components; this component is called lifelong learning and periodic self-assessment. The intended approach to this component is the development of a personal educational plan to guide and coordinate one’s learning and maximize the impact of learning. Societies with educational missions are ideal resources for diplomates needing help in devising their plans and in knowing when and how to adjust them as
The nature of self-assessment
Dixon [2] described a self-assessment cycle that can readily be related to the creation of SAMs and to participating radiologists reevaluating their personal MOC educational plans by doing the following:
- 1
Decide what quality means (agreed-on standards and improvement targets).
- 2
Develop coherent quality assurance systems (monitor actual performance against standards).
- 3
Self-assess (make judgments on the basis of the evidence).
- 4
Write self-assessment reports (summarize the analysis, evaluation, and
Format
The medical literature features several meta-analytic studies, as well as numerous individual studies, on the effectiveness of continuing medical education that are instructive in terms of SAM design. Davis et al [7] stated,
Variables over which the CME provider has control and appear to have a positive effect [on the practice of physicians and the health of their patients] are the degree of active learning opportunities, learning delivered in a longitudinal or sequenced manner, and the
The importance of feedback
Learners without some form of feedback are “in the dark” about the effectiveness of their learning efforts. The best feedback consists of rich and detailed descriptions of what a learner did and did not do relative to shared, appropriate, and rigorous standards. To be a self-directed adult learner, one’s assessment process must provide opportunities to receive and use feedback to self-adjust [13]. Wiggins [13] regarded feedback that occurs during the assessment activity as being more valuable
Assessment questions
Green [9] observed that often, as instructional designers, we want learners to achieve higher levels of learning than we evaluate. We must not succumb to the temptation to “make the measurable important” but rather should strive to “make the important measurable” (in the words of Robert McNamara, while president of the World Bank).
To correct the discrepancy between the level of learning aimed for and the level reflected in assessments, adjustments in assessment activities should be considered.
Expected outcomes of self-assessment
Ideally, assessment is expected, in one way or another, to facilitate and increase the amount of learning that occurs. Gibbs [16] illustrated how small changes in assessment strategy can dramatically transform the learning that occurs, or does not occur. An assessment system tends to maximize learning when it
- •
requires learners to put in an adequate amount of time,
- •
encourages them to apply themselves to appropriate learning activities (not just problem solving but also applying and internalizing
Summary of self-assessment principles
- •
Self-assessment promotes enhanced learning effectiveness through a cycle of reflection, evaluation, planning, and change.
- •
Key elements for effective self-assessments are dedicated time, a supportive and collegial environment, and the availability of clear standards and criteria.
- •
An effective self-assessment experience will use some or all of these techniques:
- ∘
active learning methods (not only didactic methods),
- ∘
assignments for practice,
- ∘
instruction in self-assessment skills and attitudes,
- ∘
the
- ∘
Experience with SAMs in the ABR maintenance of certification process
In the ABR’s planning for its Maintenance of Certification program, SAMs represent a vital part of the lifelong learning strategy and practice that the program seeks to foster. Questions from SAMs may be used directly or indirectly (reworked) in the cognitive expertise examination (component 3 of the program) each diplomate takes once in each MOC cycle.
As of June 1, 2006, 97 SAMs had been submitted to the ABR for qualification by 12 societies. Table 2 presents the number of SAMs submitted by
Current limitations of SAMs
Several of the present set of SAMs were written early in the process, when neither societies nor the ABR was experienced with creating the best possible SAMs. Over the past year, both have gained experience and learned from reviewer feedback and SAM data, so it is anticipated that future SAMs will steadily improve in quality.
Some categories are clearly underrepresented in the current set of SAMs. As a consequence, a diplomate with a well-developed educational plan may find that there are no
SAMs for competencies
The ABR has identified an additional contribution that SAMs will potentially make to the MOC process. Self-assessment modules that offer instruction and self-assessment in each of the 6 competencies would be extremely valuable to diplomates, because all diplomates must address all 6 competencies in the course of each Maintenance of Certification program cycle. For example, SAMs that address the competencies of systems-based practice and practice-based learning and improvement could aid
Implications for societies
The effectiveness of SAMs will be greatest when
- •
societies encourage and guide radiologists in seriously preparing, updating, and fulfilling their personal lifelong learning (educational) plans;
- •
societies help radiologists select SAMs that are most congruent with their educational plans and practice profiles; and
- •
societies make every effort to ensure that assessment questions are prepared as an integral part of SAMs more than an afterthought, that the assessment questions are important and not just
References (16)
Student self-assessment: developing learner agency
Learning Q
(1999)Self-assessment and self-evaluation in new forms of training near the workplace
Assessment and learning
Learning Q
(1999)Breaking new ground with the Learning Assessment Network
Learning Q
(1999)Conducting research on practice
Educational Researcher
(1994)- et al.
Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes?
JAMA
(1999) The “hands on” approach to humanities and social science delivery: case studies in applied education
Learning Q
(1999)
Cited by (8)
Experience with a practice quality improvement system in a university radiology department
2012, JACR Journal of the American College of RadiologyCitation Excerpt :Once faculty members, fellows, and residents are made aware of the PQI requirements, it is helpful to educate them on the basics of PQI. PQI can be simply defined as any measurable activity that results in improved patient safety and outcomes or promotes efficiency in daily practice [10-15]. Five categories for PQI have been delineated by the ABR: patient safety, accuracy of interpretation, report turnaround time, practice guidelines and technical standards, and referring physician surveys.
Editorial: Section editor's notebook: Using Self-assessment modules to document milestone achievement in radiology residency training
2013, American Journal of RoentgenologyA case tracking system with electronic medical record integration to automate outcome tracking for radiologists
2010, Journal of Digital ImagingMaintenance of certification: Update on attitudes of members of the American Society of Neuroradiology
2010, American Journal of NeuroradiologyI'll never play professional football and other fallacies of self-assessment
2008, Journal of Continuing Education in the Health Professions