Toward Practice-Based Continuing Education Protocols: Using Testing to Help Physicians Update Their Knowledge.

INTRODUCTION: Using assessment to facilitate learning is a well-established priority in education but has been associated with variable effectiveness for continuing professional development. What factors modulate the impact of testing in practitioners are unclear. We aimed to improve capacity to support maintenance of competence by exploring variables that influence the value of web-based pretesting. METHODS: Family physicians belonging to a practice-based learning program studied two educational modules independently or in small groups. Before learning sessions they completed a needs assessment and were assigned to either sit a pretest intervention or read a relevant review article. After the learning session, they completed an outcome test, indicated plans to change practice, and subsequently documented changes made. RESULTS: One hundred twelve physicians completed the study, 92 in small groups. The average lag between tests was 6.3 weeks. Relative to those given a review article, physicians given a pretest intervention: (1) reported spending less time completing the assigned task (16.7 versus 25.7 minutes); (2) performed better on outcome test questions that were repeated from the pretest (65.9% versus 58.7%); and (3) when the learning module was completed independently, reported making a greater proportion of practice changes to which they committed (80.0% versus 45.0%). Knowledge gain was unrelated to physicians' stated needs. DISCUSSION: Low-stakes formative quizzes, delivered with feedback, can influence the amount of material practicing physicians remember from an educational intervention independent of perceptions regarding the need to engage in continuing professional development on the particular topic.

Assessing Unperceived Learning Needs in Continuing Medical Education for Primary Care Physicians: A Scoping Review.

INTRODUCTION: Assessing needs before developing continuing medical education/continuing professional development (CME/CPD) programs is a crucial step in the education process. A previous systematic literature review described a lack of objective evaluation for learning needs assessments in primary care physicians. This scoping review updates the literature on uses of objective evaluations to assess physicians' unperceived learning needs in CME/CPD. Identifying and understanding these approaches can inform the development of educational programs that are relevant to clinical practice and patient care. The study objectives were to (1) scope the literature since the last systematic review published in 1999; (2) conduct a comprehensive search for studies and reports that explore innovative tools and approaches to identify physicians' unperceived learning needs; (3) summarize, compare, and classify the identified approaches; and (4) map any gaps in the literature to identify future areas of research. METHODS: A scoping review was used to "map" the literature on current knowledge regarding approaches to unperceived needs assessment using conceptual frameworks for planning and assessing CME/CPD activities. RESULTS: Two prominent gaps were identified: (1) performance-based assessment strategies are highly recommended in nonresearch articles yet have low levels of implementation in published studies and (2) analysis of secondary data through patient input or environmental scanning is emphasized in grey literature implementation strategies more so than in peer-reviewed theoretical and research articles. DISCUSSION: Future evaluations should continue to incorporate multiple strategies and focus on making unperceived needs assessments actionable by describing strategies for resource management.

Is the Cognitive Complexity of Commitment-to-Change Statements Associated With Change in Clinical Practice? An Application of Bloom's Taxonomy.

INTRODUCTION: This study categorizes 4 practice change options, including commitment-to-change (CTC) statements using Bloom's taxonomy to explore the relationship between a hierarchy of CTC statements and implementation of changes in practice. Our hypothesis was that deeper learning would be positively associated with implementation of planned practice changes. METHODS: Thirty-five family physicians were recruited from existing practice-based small learning groups. They were asked to use their usual small-group process while exploring an educational module on peripheral neuropathy. Part of this process included the completion of a practice reflection tool (PRT) that incorporates CTC statements containing a broader set of practice change options-considering change, confirmation of practice, and not convinced a change is needed ("enhanced" CTC). The statements were categorized using Bloom's taxonomy and then compared to reported practice implementation after 3 months. RESULTS: Nearly all participants made a CTC statement and successful practice implementation at 3 months. By using the "enhanced" CTC options, additional components that contribute to practice change were captured. Unanticipated changes accounted for one-third of all successful changes. Categorizing statements on the PRT using Bloom's taxonomy highlighted the progression from knowledge/comprehension to application/analysis to synthesis/evaluation. All PRT statements were classified in the upper 2 levels of the taxonomy, and these higher-level (deep learning) statements were related to higher levels of practice implementation. CONCLUSION: The "enhanced" CTC options captured changes that would not otherwise be identified and may be worthy of further exploration in other CME activities. Using Bloom's taxonomy to code the PRT statements proved useful in highlighting the progression through increasing levels of cognitive complexity-reflecting deep learning.

Encouraging Reflection and Change in Clinical Practice: Evolution of a Tool.

This article describes the systematic development and gradual transformation of a tool to guide participants in a continuing medical education program to reflect on their current practices and to make commitments to change. The continuous improvement of this tool was influenced by evolving needs of the program, reviews of relevant educational literature, feedback from periodic program surveys, interviews with group facilitators, and results from educational research studies. As an integral component of the educational process used in the Practice Based Small Group Learning Program, the current tool is designed to help family physicians think about what has been learned during each educational session and examine issues related to the implementation of evidence-based changes into their clinical practice. Lessons learned will be highlighted. Both the developmental processes employed and the practice reflection tool itself have applicability to other educational environments that focus on continuing professional development.

Which factors, personal or external, most influence students' generation of learning goals?

BACKGROUND: While concern has been expressed about the validity of self-assessments, external feedback is likely filtered through self-assessment. This paper explores the relationship between self-assessments and feedback uptake. METHOD: During an objective structured clinical examination, students were asked to evaluate their performance and rate the quality of feedback provided by observers. Afterward, they were asked to list learning goals they generated, to indicate what activities they would undertake to fulfill those goals, and to identify which station(s) led them to generate each response. Regression analyses were used to determine which variables predicted the generation of goals/activities. RESULTS: Students' perceptions of their own performance were more likely to result in the generation of goals/strategies than was observer feedback or student perceptions of observer feedback quality. Later stations were more likely to result in goal/strategy generation than earlier stations. CONCLUSIONS: While self-assessments may not validly indicate ability, it is still critical to determine how students perceive their ability because their opinions drive their learning goals.

Using a novel small-group approach to enhance feedback skills for community-based teachers.

BACKGROUND: As medical education expands into distant settings, challenges in providing faculty development to busy clinical teachers increase-especially for those who have difficulty accessing sessions offered at academic centers. DESCRIPTION: Sixty-five clinical teachers participated in six small-group workshops, using a printed module on the topic of delivering feedback. The modules included teaching-learning "cases," tools, and a summary of medical literature. The group facilitator did not require expertise in delivering feedback. Surveys inquired about impact immediately after the session and at 3 months. EVALUATION: Analysis confirmed that participants found the workshop format valuable, and the majority committed to making changes in their approaches to providing feedback. At follow-up, most participants reported that planned changes had been implemented. CONCLUSIONS: A low-tech approach to faculty development, using facilitated small-group discussion of a specially prepared educational module, is feasible for any site and can enhance teaching approaches in both urban and rural practice settings.

Translating learning into practice: lessons from the practice-based small group learning program.

UNLABELLED: PROBLEM ADDRESSED The need for effective and accessible educational approaches by which family physicians can maintain practice competence in the face of an overwhelming amount of medical information. OBJECTIVE OF PROGRAM: The practice-based small group (PBSG) learning program encourages practice changes through a process of small-group peer discussion-identifying practice gaps and reviewing clinical approaches in light of evidence. PROGRAM DESCRIPTION: The PBSG uses an interactive educational approach to continuing professional development. In small, self-formed groups within their local communities, family physicians discuss clinical topics using prepared modules that provide sample patient cases and accompanying information that distils the best evidence. Participants are guided by peer facilitators to reflect on the discussion and commit to appropriate practice changes. CONCLUSION: The PBSG has evolved over the past 15 years in response to feedback from members and reflections of the developers. The success of the program is evidenced in effect on clinical practice, a large and increasing number of members, and the growth of interest internationally.

Better Prescribing Project: a randomized controlled trial of the impact of case-based educational modules and personal prescribing feedback on prescribing for hypertension in primary care.

OBJECTIVE: The purpose of this study was to assess the impacts of individualized prescribing feedback and interactive small group education aimed at encouraging evidence-based prescribing in family/general practice. METHODS: A two-by-two factorial randomized controlled trial was carried out involving 200 family physicians in British Columbia, Canada. The physicians met monthly in 28 peer learning groups within the Practice-Based Small Group (PBSG) learning programme. Personalized prescribing feedback related to hypertension was provided through 'prescribing portraits' which graphically displayed comparative rates of individual and peer group prescribing, together with a concise guide for evidence-based prescribing. A case-based educational module, containing the same evidence-based message, was discussed in small groups. Groups were matched and randomized into four arms of seven groups each: control (n = 56), prescribing portrait only (n = 48), educational module only (n = 47), both portrait and module (n = 49). The main outcome measure was changes in 'prescribing preferences' to new patients among those medications used to treat hypertension (i.e. probability that a patient would receive the evidence-based medication as first-line therapy). RESULTS: Using data from the provincial pharmacy registry (PharmaNet), prescribing preferences for antihypertensive agents were determined for all groups for a 6 month period before and after the interventions, based on 4394 patients receiving a first-line antihypertensive. Significant absolute increases in prescribing preference for thiazides were documented for both the module +0.068 (confidence interval [CI] 0.022-0.115) and the portrait +0.065 (CI 0.018-0.111). Preference in the group receiving both module and portrait increased by +0.115 (CI 0.040-0.189). CONCLUSIONS: Evidence-based educational interventions combining personalized prescribing feedback with interactive group discussion can lead to modest but meaningful changes in physician prescribing. Clear messages, proper trial design and sensitive outcomes are necessary to demonstrate these changes.

Commitment to change: exploring its role in changing physician behavior through continuing education.

Statements of commitment to change are advocated both to promote physician change and to assess interventions designed to promote change. Although commitment to change is only one part of a complex process of change, recent progress has established a solid theoretical and research base to support this approach. Studies have demonstrated that it can be used effectively with many different types of educational activities and that statements of "plans to change " practice can predict actual changes. The importance of follow-up as part of the commitment to change model is becoming clearer, although questions remain about the most effective process to accomplish this and the optimal timing. Further research is needed to establish the effectiveness of the commitment-to-change approach itself as well as to better understand the functions (and thus the forms) of the different components of the commitment-to-change model.

Commitment to change statements can predict actual change in practice.

INTRODUCTION: Statements of commitment to change are advocated both to promote and to assess continuing education interventions. However, most studies of commitment to change have used self-reported outcomes, and self-reports may significantly overestimate actual performance. As part of an educational randomized controlled trial, this study documented changes that family physicians committed to make in their prescribing and then used third-party data to examine actual changes. METHOD: Following participation in a continuing medical education program using interactive small groups, physicians were asked to identify changes that they planned to make in their practices. For prescribing changes related to four conditions, data from a provincial pharmacy registry were analyzed for 6-month periods before and after the educational intervention. RESULTS: A total of 207 physicians participated in the project, which involved monthly meetings of 30 peer learning groups. Ninety-nine physicians received experimental case-based educational modules +/- personal prescribing feedback, and 91 of these indicated that they planned to make at least one change in practice. Of the 209 intended changes, 71% were directly related to the prescribing messages in the materials. DISCUSSION: In three of four indicator conditions, physicians who expressed a commitment to change were significantly more likely to change their actual prescribing for the target medications in the following 6 months. The percentage of physicians who did change their prescribing varied significantly by condition. Further study of the process of translating commitment to change into real practice change is needed.