Defining competency-based evaluation objectives in family medicine: procedure skills.

OBJECTIVE: To develop evaluation objectives for assessing competence in procedure skills using a key-features approach. This was part of a multiyear project to develop competency-based evaluation objectives for Certification in Family Medicine. DESIGN: Nominal group technique. SETTING: The College of Family Physicians of Canada in Mississauga, Ont. PARTICIPANTS: An expert group of 7 family physicians and 1 educational consultant, all of whom had experience in assessing competence in family medicine. Group members represented the Canadian context with respect to region, sex, language, community type, and experience. METHODS: Using a nominal group technique, the expert group developed the general key features for procedure skills. The expert group also linked the key features to already established skill dimensions in the domain of competence, to the 4 principles of family medicine, and to the CanMEDS roles. MAIN FINDINGS: The general key features were developed after 5 iterations. Ten key features were outlined and were shown to reflect all the essential skill dimensions in the domain of competence for family medicine. The key features were linked to 2 of the 4 principles of family medicine and to 4 of the CanMEDS roles. CONCLUSION: The general key features for procedure skills were developed to assess competence in procedure skills in family medicine.

Defining competency-based evaluation objectives in family medicine: professionalism.

OBJECTIVE: To develop and describe observable evaluation objectives for assessing competence in professionalism, which are grounded in the experience of practising physicians. DESIGN: Modified nominal group technique. SETTING: The College of Family Physicians of Canada in Mississauga, Ont. PARTICIPANTS: An expert group of 7 family physicians and 1 educational consultant, all of whom had experience in assessing competence in family medicine. Group members represented the Canadian context with respect to region, sex, language, community type, and experience. METHODS: Using an iterative process, the expert group defined a list of observable behaviours that are indicative of professionalism, or not, in the family medicine setting. Themes relate to professional behaviour in family medicine; specific observable behaviours are those that family physicians believe are indicative of professionalism for each theme. MAIN FINDINGS: The expert group identified 12 themes and 140 specific observable behaviours to assist in the observation and discussion of professional behaviour in family medicine workplace settings. CONCLUSION: Competency-based education literature emphasizes the importance of formative evaluation and feedback. Such feedback is particularly challenging in the domain of professionalism because of its personal nature and the potential for emotional reactions. Effective dialogue between learners and teachers begins with clear expectations and reference to descriptions of relevant, specific behaviour. This research has generated a competency-based resource to assist the assessment of professional behaviour in family medicine educational programs.

Defining competency-based evaluation objectives in family medicine: communication skills.

OBJECTIVE: To provide a pragmatic approach to the evaluation of communication skills using observable behaviours, as part of a multiyear project to develop competency-based evaluation objectives for Certification in family medicine. DESIGN: A nominal group technique was used to develop themes and subthemes and to identify positive and negative observable behaviours that demonstrate competence in communication in family medicine. SETTING: The College of Family Physicians of Canada in Mississauga, Ont. PARTICIPANTS: An expert group of 7 family physicians and 1 educational consultant, all of whom had experience in assessing competence in family medicine. Group members represented the Canadian context with respect to region, sex, language, community type, and experience. METHODS: The group used the nominal group technique to derive a list of observable behaviours that would constitute a detailed operational definition of competence in communication skills; multiple iterations were used until saturation was achieved. The group met several times a year, and membership remained unchanged during the 4 years in which the work was conducted. The iterative process was undertaken twice--once for communication with patients and once for communication with colleagues. MAIN FINDINGS: Five themes, 5 subthemes, and 106 positive and negative observable behaviours were generated. The subtheme of charting skills was defined using a key-features analysis. CONCLUSION: Communication skills were defined in terms of themes and observable behaviours. These definitions were intended to help assess family physicians' competence at the start of independent practice.

Defining competency-based evaluation objectives in family medicine: dimensions of competence and priority topics for assessment.

OBJECTIVE: To develop a definition of competence in family medicine sufficient to guide a review of Certification examinations by the Board of Examiners of the College of Family Physicians of Canada. DESIGN: Delphi analysis of responses to a 4-question postal survey. SETTING: Canadian family practice. PARTICIPANTS: A total of 302 family physicians who have served as examiners for the College of Family Physicians of Canada's Certification examination. METHODS: A survey comprising 4 short-answer questions was mailed to the 302 participating family physicians asking them to list elements that define competence in family medicine among newly certified family physicians beginning independent practice. Two expert groups used a modified Delphi consensus process to analyze responses and generate 2 basic components of this definition of competence: first, the problems that a newly practising family physician should be competent to handle; second, the qualities, behaviour, and skills that characterize competence at the start of independent practice. MAIN FINDINGS: Response rate was 54%; total number of elements among all responses was 5077, for an average 31 per respondent. Of the elements, 2676 were topics or clinical situations to be dealt with; the other 2401 were skills, behaviour patterns, or qualities, without reference to a specific clinical problem. The expert groups identified 6 essential skills, the phases of the clinical encounter, and 99 priority topics as the descriptors used by the respondents. More than 20% of respondents cited 30 of the topics. CONCLUSION: Family physicians define the domain of competence in family medicine in terms of 6 essential skills, the phases of the clinical encounter, and priority topics. This survey represents the first level of definition of evaluation objectives in family medicine. Definition of the interactions among these elements will permit these objectives to become detailed enough to effectively guide assessment.

Defining competency-based evaluation objectives in family medicine: key-feature approach.

OBJECTIVE: To develop key features for priority topics previously identified by the College of Family Physicians of Canada that, together with skill dimensions and phases of the clinical encounter, broadly describe competence in family medicine. DESIGN: Modified nominal group methodology, which was used to develop key features for each priority topic through an iterative process. SETTING: The College of Family Physicians of Canada. PARTICIPANTS: An expert group of 7 family physicians and 1 educational consultant, all of whom had experience in assessing competence in family medicine. Group members represented the Canadian family medicine context with respect to region, sex, language, community type, and experience. METHODS: The group used a modified Delphi process to derive a detailed operational definition of competence, using multiple iterations until consensus was achieved for the items under discussion. The group met 3 to 4 times a year from 2000 to 2007. MAIN FINDINGS: The group analyzed 99 topics and generated 773 key features. There were 2 to 20 (average 7.8) key features per topic; 63% of the key features focused on the diagnostic phase of the clinical encounter. CONCLUSION: This project expands previous descriptions of the process of generating key features for assessment, and removes this process from the context of written examinations. A key-features analysis of topics focuses on higher-order cognitive processes of clinical competence. The project did not define all the skill dimensions of competence to the same degree, but it clearly identified those requiring further definition. This work generates part of a discipline-specific, competency-based definition of family medicine for assessment purposes. It limits the domain for assessment purposes, which is an advantage for the teaching and assessment of learners. A validation study on the content of this work would ensure that it truly reflects competence in family medicine.

Big cities and bright lights: rural- and northern-trained physicians in urban practice.

INTRODUCTION: Rural medical education is increasing in popularity in Canada. This study examines why some family physicians who completed their residency training in northern Ontario decided to practise in urban centres. METHODS: We used a qualitative research method. We interviewed 14 graduates of the Family Medicine North program and the Northeastern Ontario Family Medicine program. The interview transcripts were content-analyzed. RESULTS: There were different pathways leading to urban practice. While some pathways were straightforward, others were more complicated. Most participants offered multiple reasons for choosing to work in urban areas, suggesting that the decision-making processes could be quite complex. Family and personal factors were most frequently mentioned as reasons for choosing the urban option. The needs of the spouse and the children were especially important. Most of the participants had no plans to return to rural medical practice, but even these physicians retained some vestiges of rural practice. CONCLUSION: Most Canadian medical schools now offer some rural medical training opportunities. The findings of this study provide some useful insights that could help medical educators and decision-makers know what to expect and understand how practice location decisions are made by doctors.

Graduates of northern Ontario family medicine residency programs practise where they train.

OBJECTIVE: To examine where graduates of the Northeastern Ontario Family Medicine (NOFM) residency program in Sudbury and the Family Medicine North (FMN) program in Thunder Bay practise after graduation, using cross-sectional and longitudinal analyses. METHODS: Data from the Scott's Medical Database were examined. All physicians who graduated from NOFM and FMN between 1993 and 2002 were included in this analysis. Differences in the location of first practice between NOFM and FMN graduates were tested using chi-squared tests. Logistic regression analyses were used to examine the impact of the training program on a physician's first, as well as continuing, practice location. RESULTS: Between 1993 and 2002, FMN graduates were 4.56 times more likely (95% confidence interval [CI] 2.34-8.90) to practise in rural areas, compared with NOFM graduates, but NOFM graduates were 2.50 times more likely than FMN graduates (95% CI 1.35-4.76) to practise in northern Ontario. There was no statistically significant difference between the graduates of the 2 programs in the likelihood of working in either northern Ontario or a rural area. About two-thirds (67.5%) of all person-years of medical practice provided by NOFM and FMN graduates took place in northern Ontario or rural areas outside the north. CONCLUSION: NOFM and FMN have been successful in producing family physicians to work in northern Ontario and rural areas. Results from this study add to the growing evidence from Canada and abroad that rural or northern medical education and training increases the likelihood that the graduates will practise in rural or northern communities.

Duration of rural training during residency: rural family physicians prefer 6 months.

OBJECTIVE: To determine whether rural family physicians thought they had received enough months of rural exposure during family medicine residency, how many months of rural exposure those who were satisfied with their training had had, and how many months of rural exposure those who were not satisfied with their training wanted. DESIGN: Mailed survey. SETTING: Rural Canada. PARTICIPANTS: Rural family physicians who had graduated between 1991 and 2000 from a Canadian medical school. MAIN OUTCOME MEASURES: Respondents' opinions about whether their exposure to rural medicine during training had been adequate. RESULTS: Response rate was 59% (382/651). After excluding physicians who had not had Canadian family medicine residency training, 348 physicians remained, and of those, 58% thought they had had adequate rural exposure during residency. Median duration of rural training among those who thought they had had enough rural exposure was 6 months; median duration of rural exposure among those who thought they had not had enough was 2 months. Median duration of rural exposure desired by those who thought they had not had enough rural training was 6 months. Some physicians wanted much more than 6 months of rural training; for example, one quarter of those satisfied with their rural training had had 10 or more months of rural rotations. Fewer than 1% of respondents thought they had received too much rural training. There was no significant difference in number of months of rural training preferred by men and women (P = .94). One third of respondents had graduated from rural-focused family practice residency programs. Rural program graduates were more likely than non-rural program graduates to report that the duration of their rural training was adequate (84% vs 46%, P < .0001) and to report more mean months of rural exposure (8.9 vs 3.4; P < .0001). CONCLUSION: Typical rural family physicians prefer to have 6 months of rural exposure during residency. This finding is consistent with the recommendation of a College of Family Physicians of Canada committee that rural family medicine training programs offer at least 6 months of rural rotations. Almost half of rural family physicians wished they had had more rural training. Both rural-focused and non-rural-focused programs should consider providing opportunities for pursuing elective rotations in rural areas in addition to mandatory rotations if they want to respond to these preferences for training.

Factors influencing family physicians to enter rural practice: does rural or urban background make a difference?

OBJECTIVE: To examine where rural physicians grew up, when during their training they became interested in rural medicine, factors influencing their decision to practise rural medicine, and differences in these measures according to rural or urban upbringing. DESIGN: Mailed survey. SETTING: Rural Canada. PARTICIPANTS: Rural family physicians who graduated between 1991 and 2000 from a Canadian medical school. MAIN OUTCOME MEASURES: Backgrounds of recently graduated rural physicians, when physicians first became interested in rural practice during training, and most influential factors in decisions to practise rural medicine. RESULTS: Response rate was 59% (382/651). About 33% of rural physicians grew up in communities of less than 10 000 people, 44% in cities of 10 000 to 499 999 people, and 23% in cities of more than 500 000 people. Physicians raised in rural areas were more likely than those raised in urban areas to have some interest in rural family practice at the start and end of medical school (90% vs 67% at the start, 98% vs 91% at the end, respectively, P < .0001). Physicians raised in urban areas were more likely to state that rural medical training was the most influential factor in their choice of rural practice (19% vs 9%, P = .015). Other factors cited as influential were the challenge of rural practice (24% for both urban and rural upbringing), rural lifestyle (14% for urban and 18% for rural upbringing) and, for physicians raised in rural areas, having grown up or spent time in a rural area (27% for rural and 4.1% for urban upbringing, P < .001). Financial incentives were least frequently cited as the most influential factor (7.5% for urban and 4.9% for rural upbringing, P = .35). CONCLUSION: Although other studies have suggested that physicians with a rural upbringing are more likely to practise rural medicine and policy makers might still wish to target students raised in rural areas as candidates for rural medicine, this study shows that physicians raised in urban areas remain the main source of human resources for rural communities. They account for two thirds of new physicians in rural areas. Education in rural medicine during medical training has a stronger influence on physicians raised in urban areas than on physicians raised in rural areas. Undergraduate and postgraduate training periods, therefore, offer an important opportunity for recruiting physicians raised in urban areas to rural practice.