Original article
Self-Assessment for Maintenance of Certification

https://doi.org/10.1016/j.jacr.2006.08.014Get rights and content

Self-assessment is paired with continuing medical education in the “lifelong learning ” component of the American Board of Radiology’s (ABR) Maintenance of Certification program. The literature on the use of self-assessment in higher education and the professions is instructive with regard to the distinctiveness of this form of adult learning. This article reviews pertinent literature, summarizes the findings, and applies the principles to the creation and use of “self-assessment modules” (SAMs) for the maintenance of certification. Data on the use of ABR-qualified SAMs are reported. On completion of this article, the reader should be able to describe the nature, role, and best practices of self-assessment; apply these concepts to the creation or use of a SAM; and identify future opportunities for SAM development and use that will enhance the lifelong learning of radiologists.

Section snippets

Why self-assessment?

The literature has a number of concise descriptions of self-assessment. Here are 2 of the best, eloquently stated in each authors’ own words:

A key attribute of an effective learner is the ability to critically analyze one’s achievements and progress … in the full cycle of action, reflection, evaluation, and further planning for continued improvement [1].

Good [practitioners] have always assessed or evaluated their own performances. They observe the consequences of their actions, listen to

ABR’S approach to self-assessment

In the ABR’s Maintenance of Certification program, self-assessment is part of the second of 4 components; this component is called lifelong learning and periodic self-assessment. The intended approach to this component is the development of a personal educational plan to guide and coordinate one’s learning and maximize the impact of learning. Societies with educational missions are ideal resources for diplomates needing help in devising their plans and in knowing when and how to adjust them as

The nature of self-assessment

Dixon [2] described a self-assessment cycle that can readily be related to the creation of SAMs and to participating radiologists reevaluating their personal MOC educational plans by doing the following:

  • 1

    Decide what quality means (agreed-on standards and improvement targets).

  • 2

    Develop coherent quality assurance systems (monitor actual performance against standards).

  • 3

    Self-assess (make judgments on the basis of the evidence).

  • 4

    Write self-assessment reports (summarize the analysis, evaluation, and

Format

The medical literature features several meta-analytic studies, as well as numerous individual studies, on the effectiveness of continuing medical education that are instructive in terms of SAM design. Davis et al [7] stated,

Variables over which the CME provider has control and appear to have a positive effect [on the practice of physicians and the health of their patients] are the degree of active learning opportunities, learning delivered in a longitudinal or sequenced manner, and the

The importance of feedback

Learners without some form of feedback are “in the dark” about the effectiveness of their learning efforts. The best feedback consists of rich and detailed descriptions of what a learner did and did not do relative to shared, appropriate, and rigorous standards. To be a self-directed adult learner, one’s assessment process must provide opportunities to receive and use feedback to self-adjust [13]. Wiggins [13] regarded feedback that occurs during the assessment activity as being more valuable

Assessment questions

Green [9] observed that often, as instructional designers, we want learners to achieve higher levels of learning than we evaluate. We must not succumb to the temptation to “make the measurable important” but rather should strive to “make the important measurable” (in the words of Robert McNamara, while president of the World Bank).

To correct the discrepancy between the level of learning aimed for and the level reflected in assessments, adjustments in assessment activities should be considered.

Expected outcomes of self-assessment

Ideally, assessment is expected, in one way or another, to facilitate and increase the amount of learning that occurs. Gibbs [16] illustrated how small changes in assessment strategy can dramatically transform the learning that occurs, or does not occur. An assessment system tends to maximize learning when it

  • requires learners to put in an adequate amount of time,

  • encourages them to apply themselves to appropriate learning activities (not just problem solving but also applying and internalizing

Summary of self-assessment principles

  • Self-assessment promotes enhanced learning effectiveness through a cycle of reflection, evaluation, planning, and change.

  • Key elements for effective self-assessments are dedicated time, a supportive and collegial environment, and the availability of clear standards and criteria.

  • An effective self-assessment experience will use some or all of these techniques:

    • active learning methods (not only didactic methods),

    • assignments for practice,

    • instruction in self-assessment skills and attitudes,

    • the

Experience with SAMs in the ABR maintenance of certification process

In the ABR’s planning for its Maintenance of Certification program, SAMs represent a vital part of the lifelong learning strategy and practice that the program seeks to foster. Questions from SAMs may be used directly or indirectly (reworked) in the cognitive expertise examination (component 3 of the program) each diplomate takes once in each MOC cycle.

As of June 1, 2006, 97 SAMs had been submitted to the ABR for qualification by 12 societies. Table 2 presents the number of SAMs submitted by

Current limitations of SAMs

Several of the present set of SAMs were written early in the process, when neither societies nor the ABR was experienced with creating the best possible SAMs. Over the past year, both have gained experience and learned from reviewer feedback and SAM data, so it is anticipated that future SAMs will steadily improve in quality.

Some categories are clearly underrepresented in the current set of SAMs. As a consequence, a diplomate with a well-developed educational plan may find that there are no

SAMs for competencies

The ABR has identified an additional contribution that SAMs will potentially make to the MOC process. Self-assessment modules that offer instruction and self-assessment in each of the 6 competencies would be extremely valuable to diplomates, because all diplomates must address all 6 competencies in the course of each Maintenance of Certification program cycle. For example, SAMs that address the competencies of systems-based practice and practice-based learning and improvement could aid

Implications for societies

The effectiveness of SAMs will be greatest when

  • societies encourage and guide radiologists in seriously preparing, updating, and fulfilling their personal lifelong learning (educational) plans;

  • societies help radiologists select SAMs that are most congruent with their educational plans and practice profiles; and

  • societies make every effort to ensure that assessment questions are prepared as an integral part of SAMs more than an afterthought, that the assessment questions are important and not just

References (16)

  • N. Randall

    Student self-assessment: developing learner agency

    Learning Q

    (1999)
  • S. Dixon
  • B. Gelderman

    Self-assessment and self-evaluation in new forms of training near the workplace

  • M. Battersby

    Assessment and learning

    Learning Q

    (1999)
  • C. Dunlop

    Breaking new ground with the Learning Assessment Network

    Learning Q

    (1999)
  • V. Richardson

    Conducting research on practice

    Educational Researcher

    (1994)
  • D.A. Davis et al.

    Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes?

    JAMA

    (1999)
  • R. Taylor

    The “hands on” approach to humanities and social science delivery: case studies in applied education

    Learning Q

    (1999)
There are more references available in the full text version of this article.

Cited by (8)

View all citing articles on Scopus
View full text