Effects of continuing professional development on clinical performance: results of a study involving family practitioners in Quebec.

OBJECTIVE: To evaluate the link between the quantity and quality of continuing professional development (CPD) activities completed by family physicians in Quebec and the quality of their practice. DESIGN: Retrospective analysis of data collected during professional inspection visits (PIVs). SETTING: Quebec. PARTICIPANTS: Three groups were created from among Quebec family physicians who had been subject to PIVs (peer evaluation) by the Collège des médecins du Québec between 1998 and 2005. Group 1 was composed of physicians who were members of the College of Family Physicians of Canada, which requires participation in 250 hours of CPD in every 5-year cycle. Group 2 was composed of family physicians who were not members of the College of Family Physicians of Canada but who had declared at least 50 hours a year of CPD on their Collège des médecins du Québec annual notice of assessment for the same period. Group 3 was composed of family physicians who had declared fewer than 10 hours of CPD a year. MAIN OUTCOME MEASURES: During the PIV, the following characteristics were examined: record keeping, quality and number of hours of CPD activities, and quality of professional practice based on 3 components- clinical investigation, accuracy of diagnosis, and appropriateness of treatment plan and follow-up. RESULTS: The factors associated with a high quality of practice were privileges in a hospital or local community health centre (institution) and a substantial number of accredited CPD hours (Mainpro-M1, Credit I, or Mainpro-C). The factors associated with a poor quality of practice were advanced age of the physician, absence of privileges in an institution (hospital or local community health centre), and participation in CPD activities that were more informal, such as reading and non-accredited activities (Mainpro-M2). CONCLUSION: This study supports earlier research showing that CPD activities of sufficient quality and quantity are correlated with a high quality of professional practice by family physicians.

Practice features associated with patient-reported accessibility, continuity, and coordination of primary health care.

PURPOSE: On the eve of major primary health care reforms, we conducted a multilevel survey of primary health care clinics to identify attributes of clinic organization and physician practice that predict accessibility, continuity, and coordination of care as experienced by patients. METHODS: Primary health care clinics were selected by stratified random sampling in urban, suburban, rural, and remote locations in Quebec, Canada. Up to 4 family or general physicians were selected in each clinic, and 20 patients seeing each physician used the Primary Care Assessment Tool to report on first-contact accessibility (being able to obtain care promptly for sudden illness), relational continuity (having an ongoing relationship with a physician who knew their particulars), and coordination continuity (having coordination between their physician and specialists). Physicians reported on aspects of their practice, and secretaries and directors reported on organizational features of the clinic. We used hierarchical regression modeling on the subsample of regular patients at the clinic. RESULTS: One hundred clinics participated (61% response rate), for a total of 221 physicians and 2,725 regular patients (87% response and completion rate). First-contact accessibility was most problematic. Such accessibility was better in clinics with 10 or fewer physicians, a nurse, telephone access 24 hours a day and 7 days a week, operational agreements to facilitate care with other health care establishments, and evening walk-in services. Operational agreements and evening care also positively affected relational continuity. Physicians who valued continuity and felt attached to the community fostered better relational continuity, whereas an accessibility-oriented style (as indicated by a high proportion of walk-in care and high patient volume) hindered it. Coordination continuity was also associated with more operational agreements and continuous telephone access, and was better when physicians practiced part time in hospitals and performed a larger range of medical procedures in their office. CONCLUSIONS: The way a clinic is organized allows physicians to achieve both accessibility and continuity rather than one or the other. Features that achieve both are offering care in the evenings and access to telephone advice, and having operational agreements with other health care establishments.

Influence of remedial professional development programs for poorly performing physicians.

INTRODUCTION: The Collège des Médecins du Québec (CMQ) offers an individualized remedial professional development program to help physicians overcome selected clinical shortcomings. To measure the influence of the remedial professional development program, physicians who completed the program between 1993 and 2004 and who were assessed by peer review during a 2-year period preceding or following the remedial activities were tracked. METHODS: For each physician, 30 to 50 patient records were selected randomly for review. Ratings were assigned for the quality of record keeping and for 3 elements pertaining to the quality of care: the clinical investigation plan, diagnostic accuracy, and patient treatment and follow-up. The impact of the program was measured by comparing the proportion of physicians with satisfactory ratings assigned by peer review before and after the remedial professional development program. RESULTS: Statistically significant improvements (p < .05) were observed for a proportion of physicians (n = 51) with satisfactory ratings with regard to record keeping (20% before and 54% after remediation), the clinical investigation plan (13% before and 59% after remediation), diagnostic accuracy (32% before and 61% after remediation), and patient treatment and follow-up (31% before and 67% after remediation). DISCUSSION: Participation in a CMQ remedial professional development program can result in improved clinical performance, as assessed through peer review.

Assessment of family physicians' performance using patient charts: interrater reliability and concordance with chart-stimulated recall interview.

Peer-assessment processes with chart review have been used for many years to assess the clinical performance of physicians. The Quebec medical licensing authority has been required by provincial law to assess the practicing Quebec physicians on a nonvoluntary basis. During the period from January 2001 to November 2004, 25 family physicians in active practice were randomly selected from a pool of about 300. For each physician, 25 to 40 patients' medical charts were randomly selected to evaluate the interrater reliability of peer-review assessment of medical charts and to compare ratings based on chart review with a chart-stimulated recall interview to those based on chart review alone. The concordance between chart review alone and that of chart review with chart-stimulated recall interview was 75% for chart keeping, 69% for clinical investigation, 81% for diagnostic accuracy, and 74% for treatment plan. Ratings based on chart review alone achieve moderate levels of reliability (Kappa = 0.44 to 0.56). It appears that some important information about quality of care is missed when only chart review is used.

Room for improvement: patients' experiences of primary care in Quebec before major reforms.

OBJECTIVE: To investigate variations in accessibility, continuity of care, and coordination of services as experienced by patients in Quebec on the eve of major reforms, and to provide baseline information against which reforms could be measured. DESIGN: Multilevel cross-sectional survey of practice. SETTING: One hundred primary health care settings were randomly selected in urban, suburban, rural, and remote locations in 5 health regions in Quebec. PARTICIPANTS: In each clinic, we chose up to 4 physicians and 20 consecutive patients consulting each physician. MAIN OUTCOME MEASURES: Patients' responses to a self-administered questionnaire, the Primary Care Assessment Tool, that assessed patient-provider affiliation, accessibility, relational continuity, coordination of primary and specialty care, and whether patients received health promotion and preventive services. RESULTS: A total of 3441 patients participated (87% acceptance rate) in 100 clinics (64% response rate). Timely access was difficult; only 10% expressed confidence they could be seen by their regular doctors within a day if they became suddenly ill. Average waiting time for a doctor's appointment was 24 days. Coordination of care with specialists was at minimally acceptable levels. Patients with family physicians recalled them addressing only 56% of the health promotion and preventive issues appropriate for their age and sex, and patients without family physicians recalled physicians addressing substantially fewer (38%). Most patients reported they were highly confident that their physicians knew them well and would manage their care beyond clinical encounters (relational continuity). The exception was the 16% of patients overall who did not have family physicians (34% of patients at walk-in clinics). CONCLUSION: This survey highlights serious problems with accessibility. Improvement is needed urgently to avoid deterioration of patients' confidence in the health system even though patients rate their relationships with their physician highly. Health promotion, preventive services, and coordination with specialists also needed to be improved, and careful thought must be given to the plight of those without family physicians.

CME workshop in recognizing boundary limits during a medical visit.

BACKGROUND AND OBJECTIVES: Professional boundaries help to establish and maintain the foundation of the doctor-patient relationship. Our objective was to measure physicians' level of awareness in recognizing a boundary limit, their intention to change their behavior regarding boundaries, and their self-reported changes in behavior after participation in an educational workshop, "The Right Distance." METHODS: This longitudinal study used questionnaires to measure the level of awareness of the participants in recognizing boundary limits before and after the workshop and to measure their intention to change their behavior and self-reported behavior changes immediately after the workshop and 6 months later. RESULTS: Before the workshop, the level of awareness in recognizing the boundary limits was 3.9 on a 5.0 point scale in which 5.0 represented the highest level of awareness, and immediately after the workshop it was 4.2/5.0. Female physicians were more aware of these issues. Following the workshop, 51% of participants indicated an intention to change their behavior. On the 6-month follow-up questionnaire, 138 physicians (60%) answered that they had not changed their behavior, and 92 physicians (40%) answered that they had modified their behavior by changing their attitude toward their personal and professional life and toward gifts and gratuities. When the post-workshop and the 6-month follow-up questionnaires were matched, 34 of 57 (60%) participants who stated their intention to change did in fact change their behavior regarding the "right distance." CONCLUSIONS: Boundary issues and level of awareness can be effectively taught in a continuing medical education workshop.

An innovative approach to remedial continuing medical education, 1992-2002.

The authors describe the process of remedial retraining programs organized and planned for Quebec physicians by the College des medecins du Quebec (CMQ) and report the outcomes of these efforts from April 1992 to March 2002. The CMQ (the Quebec medical licensing authority) developed a process to identify physicians who had shortcomings in their clinical performance, determine their educational needs, propose, in collaboration with the four medical schools in the province, personalized retraining programs (clinical training programs, tutorials, focused readings, workshops, and refresher courses), and subsequently evaluate the impact of these retraining programs. During the ten-year period reported, 305 physicians (216 family physicians and 89 specialists) were referred to the Practice Enhancement Division of the CMQ for personalized remedial retraining. The vast majority of these physicians were men (81%). The following difficulties were identified: therapeutic knowledge (37%), diagnostic knowledge (32%), record-keeping (14%), technical skills (10%), clinical judgment (5%), and communication skills (2%). A total of 329 personalized retraining programs were completed: 273 clinical training programs, 41 tutorials, and 15 focused readings. A reevaluation of all these physicians showed that 70% of the retraining programs had succeeded, 15% were partially successful and only 13% had failed. The remaining 2% involved missing data or withdrawal of physicians. The authors conclude that the collaborative CME process described has important and effective original features.

Breaking bad news: impact of a continuing medical education workshop.

OBJECTIVE: To evaluate the impact of an interactive continuing medical education workshop designed to help physicians in breaking bad news to their patients. METHODOLOGY: Analysis of post-workshop questionnaires from 539 physicians assessing the retention of the key concepts and the perception of the potential impact of the workshop on their practice immediately after the workshop and six months later. RESULTS: The most significant concepts retained by the respondents are: the need to take into consideration the whole patient (42.7% post-workshop and 45.6% of follow-up responses), the need to be prepared for the consultation (11.6% and 15%), the importance of better guiding the interview (18.8% and 13.6%), and the value of taking more time during the consultation (5.8% and 8.3%). Analysis of paired responses on the post-workshop and the follow-up questionnaires shows that 35% of the concepts retained are identical. CONCLUSION: The majority of physicians retained the key concepts, both immediately following the workshop and in the longer term.

Determining the number of patient charts necessary for a reliable assessment of practicing family physicians' performance.

In many countries, peer assessment programs based on the examination of patient charts are becoming a standard to assess physician's clinical performance. Although data on validity of the process are acceptable, reliability issues need some improvement. This article addresses the rarely studied aspect of optimal number of patient charts for an acceptable reliable assessment. Fifteen patient charts for each of a group of 20 practicing physicians were independently reviewed by 4 professional peer assessors. Generalizability (G) and decision (D) studies were applied to the data. It appears that as few as 10 patient charts are sufficient for any assessor to obtain a G coefficient of 0.80. Results of the current study suggest the possibility of getting generalizable assessments by peer reviewer with minimal information. These results are not in accordance with the concept of case specificity in which it is claimed that performance on a case is a poor predictor of performance on a different case.

Poorly performing physicians: does the Script Concordance Test detect bad clinical reasoning?

INTRODUCTION: Evaluation of poorly performing physicians is a worldwide concern for licensing bodies. The Collège des Médecins du Québec currently assesses the clinical competence of physicians previously identified with potential clinical competence difficulties through a day-long procedure called the Structured Oral Interview (SOI). Two peer physicians produce a qualitative report. In view of remediation activities and the potential for legal consequences, more information on the clinical reasoning process (CRP) and quantitative data on the quality of that process is needed. This study examines the Script Concordance Test (SCT), a tool that provides a standardized and objective measure of a specific dimension of CRP, clinical data interpretation (CDI), to determine whether it could be useful in that endeavor. METHODS: Over a 2-year period, 20 family physicians took, in addition to the SOI, a 1-hour paper-and-pencil SCT. Three evaluators, blind as to the purpose of the experiment, retrospectively reviewed SOI reports and were asked to estimate clinical reasoning quality. Subjects were classified into 2 groups (below and above median of the score distribution) for the 2 assessment methods. Agreement between classifications is estimated with the use of the Kappa coefficient. RESULTS: Intraclass correlation for SOI was 0.89. Cronbach alpha coefficient for the SCT was 0.90. Agreement between methods was found for 13 participants (Kappa: 0.30, P = 0.18), but 7 out of 20 participants were classified differently in both methods. All participants but 1 had SCT scores below 2 SD of panel mean, thus indicating serious deficiencies in CDI. DISCUSSION: The finding that the majority of the referred group did so poorly on CDI tasks has great interest for assessment as well as for remediation. In remediation of prescribing skills, adding SCT to SOI is useful for assessment of cognitive reasoning in poorly performing physicians. The structured oral interview should be improved with more precise reporting by those who assess the clinical reasoning process of examinees, and caution is recommended in interpreting SCT scores; they reflect only a part of the reasoning process.

Performance assessment. Family physicians in Montreal meet the mark!

OBJECTIVE: To assess the clinical performance of a representative non-volunteer sample of family physicians in metropolitan Montreal, Que. DESIGN: Assessment of clinical performance was based on inspection visits to offices, peer review of medical records, and chart-stimulated recall interviews. The procedure was the one usually followed by the Professional Inspection Committee of the Collège des médecins du Québec. SETTING: Family physicians' practices in metropolitan Montreal. PARTICIPANTS: One hundred randomly selected family physicians. INTERVENTIONS: For each physician, 30 randomly chosen patient charts with data on three to five previous visits were reviewed using explicit criteria and a standard scale using global scores from 1 to 5 (unacceptable to excellent). MAIN OUTCOME MEASURES: Scores were assigned for office practices; record keeping; number of continuing medical education (CME) activities; and quality of clinical performance assessed in terms of investigation plan, diagnostic accuracy, treatment plan, and relevance of care. RESULTS: Overall performance was judged to be good to excellent for 98% of physicians in their private practices; for 90% of physicians concerning CME activities; for 94% of physicians concerning their clinical performance in terms of quality of care; and for 75% of physicians as to record keeping. There was a link between record keeping and quality of care as well as between the number of CME activities and quality of care. CONCLUSION: The overall clinical performance of family physicians in the greater Montreal region is excellent.

Participants' perception of impact of a workshop on their preventive practices.

A 90-minute interactive workshop, offered to small groups on request, was developed to help physicians include evidence-based preventive interventions in their practices. Between 25 September 1996 and 10 December 1997, 593 family physicians throughout the Province of Quebec (Canada) participated in one of the 40 workshops presented in all the regions of Quebec. Almost all participants (98%) completed the self-administered questionnaire. Their opinion of the achievement of three workshop objectives were evaluated using a seven-point Likert scale (-3 to +3)as their perception of the direct impact of the workshop on their practice. The workshop objectives were reached to a high degree: 2.1 (sd 0.90) for prescribing a proper check-up for adults; 1.83(sd 1.02) for explaining to the patient the reasons motivating his/her choice to include or exclude certain tests; 2.09 (sd 0.93)for using concrete and useful tool facilitating the integration of preventive measures in his/her professional practice. Female physicians and those under 40 perceived that the objectives were reached to a greater degree. Participants indicate their intention to modify their practice according to the clinical practice guidelines presented in the workshop.