Patients' perceptions of medical students in a longitudinal family medicine clerkship.

BACKGROUND: Although educational characteristics of ambulatory clinical environments are becoming clearer, less is known concerning patient opinions about participating in medical student instruction in ambulatory settings. Such perceptions may have an important influence on recruitment and retention of community faculty. METHODS: Surveys were administered to 121 patients seen by medical students during a longitudinal family medicine clerkship. The survey explored patients' opinions regarding the extent of direct student involvement in their care, students'competence, and patient feelings about participating in medical student instruction. RESULTS: Patients felt that students were highly involved in providing care and that they performed competently and professionally. Patients found participation in medical education enjoyable, not excessively time-consuming or disruptive, and believed that students' participation improved the quality of care they received. CONCLUSIONS: Patients in our family medicine clerkship do not have negative perceptions about their participation in medical student education. In fact, this study suggests that such participation may actually enhance patient satisfaction.

Assessing the acquisition of core clinical skills through the use of serial standardized patient assessments.

PURPOSE: Students in many medical schools now undergo multiple standardized-patient-based assessments. In this study, the authors examine the ability of such serial assessments to detect interval learning. METHOD: Twenty-one students from the University of California, Irvine, College of Medicine, class of 1999, underwent a clinical skills appraisal after three months of their third-year instruction. After nine months, all 89 members of that class completed an OSCE. Subsequently, all 87 students in the class of 2000 also completed clinical skills assessments after their third and ninth months of third-year instruction. All of these exercises included identical or similar stations measuring history, physical examination, and communication skills. Communication skills were measured somewhat differently during some of the exercises, using checklists that were either "content-" or "process-"oriented. The authors compared the performances for all groups. RESULTS: Both classes demonstrated significant improvement in physical examination performance, while their history performances remained unchanged. According to the assessments, their communication skills deteriorated over the course of their third-year instruction. Repeated exposures to similar or identical cases on the serial assessments did not impact the students' performances. Both content- and process-oriented measures of communication skills yielded highly similar results. CONCLUSIONS: Serial assessments using standardized patients can detect interval changes in performance that are independent of repeated exposures to similar or identical cases. Changes detected using this approach may have important curricular implications.

Initial experience with a multi-station objective structured teaching skills evaluation.

PURPOSE: To explore the applicability of using an objective structured evaluation to assess community preceptors' teaching performances. METHOD: The authors developed and, in 1996, administered an objective structured teaching evaluation (OSTE) at the University of California, Irvine, College of Medicine. They compared assessments of teaching skills made by faculty observers and standardized students, and examined instrument reliability, realism, and utility. RESULTS: Reliabilities of the OSTE varied with individual stations but, in several instances, approached acceptable standards. Faculty observers' and standardized students' evaluations of preceptors' performances were similar. Participants' feedback indicated that the greatest OSTE-related benefit was derived by the faculty observers. CONCLUSIONS: An OSTE that reflects realistic teaching situations can be successfully developed. Objective structured evaluation can be successfully applied to assessing faculty teaching performance. However, it may be no more discriminating than are student evaluations. An OSTE's utility may be greatest in guiding faculty development initiatives.

What students say about learning and teaching in longitudinal ambulatory primary care clerkships: a multi-institutional study.

PURPOSE: Ambulatory primary care clerkships have become crucial elements in medical education. Although most such clerkships employ a block-rotation format, an alternative longitudinal approach has been developed. This study examines students' perceptions of learning and instruction occurring during longitudinal ambulatory clerkships. METHOD: Characteristics of longitudinal ambulatory primary care clerkships at five medical schools are described. Responses of 429 medical students to a standardized survey administered at these institutions are analyzed to ascertain perceptions of learning and teaching occurring during longitudinal ambulatory clerkship experiences. RESULTS: Enhancements of interpersonal communication and clinical skills were perceived to be the most positive learning attributes of the longitudinal ambulatory clerkships. No advantage was discerned with respect to disease-pattern recognition or generation of differential diagnoses. While significant inter-institutional variation was present, particularly with respect to instructional format, there was notable agreement regarding several aspects of clerkship-related learning and the adequacy of faculty supervision. CONCLUSION: Students perceived that learning during longitudinal ambulatory clerkships had greater impact on skill enhancement than on attainment of knowledge-related objectives. Sources of variation in student opinion, perceptions of learning as a function of career preference, and correlation of students' perceptions of learning to demonstrable changes in their competence require further investigation.

Use of an objective structured clinical examination in evaluating student performance.

BACKGROUND AND OBJECTIVES: The objective structured clinical exam (OSCE) is increasingly being used to evaluate student clinical performance. However, scant literature exists pertinent to this approach in evaluating family medicine clerkship performance. In this study, we assess 8 years' experience with a family medicine clerkship OSCE. METHODS: Eight annual clerkship OSCEs and the performance of 696 students are described. Comparisons of faculty evaluation, written exam, and OSCE performance are made for 335 students. Post-OSCE student and faculty feedback regarding OSCE validity and utility is also presented. RESULTS: Student performance is highest in medical history taking and physical examination and lowest in information-sharing stations. OSCE results appear to be relatively consistent on a year-to-year basis. OSCE, faculty evaluation, and written exam results have low overall levels of correlation, particularly in assessing performance that differs substantially from the mean. Students and faculty agree that the OSCE experience reflects skills that students should possess, but there is less agreement that the OSCE reflects clerkship-related learning and actual student performance. Both students and faculty derive insight from the OSCE regarding the definition of specific learning needs. CONCLUSIONS: The family medicine clerkship OSCE we describe appears to provide consistent measures of student performance. Although content validity is high, further assessment is needed to assure construct validity. The OSCE experience provides students with a rich resource for defining clerkship-related learning needs. Study results strongly suggest that OSCEs, faculty evaluations, and written exams provide differing measures of student performance. The reasons for these differences merit further exploration.

A comparison of family medicine clerkship student performance across multiple teaching sites.

BACKGROUND AND OBJECTIVES: Family medicine clerkships are frequently conducted in decentralized settings. Concerns have been raised regarding the educational consistency of such rotations. In this study, we assess learning across multiple training sites, as reflected by measures of student performance. METHODS: The study population was 77 students assigned to six clusters of family medicine clerkship training sites during the 1993-1994 academic year. Learning, as measured by faculty evaluations, objective structured clinical examination (OSCE) performance, and Society of Teachers of Family Medicine (STFM) predoctoral examination performance, was compared for student cohorts assigned to each of the teaching site clusters using analysis of variance. RESULTS: No differences in OSCE or STFM examination performance were found across the teaching site clusters. Greater variation in faculty evaluation across sites was apparent, and when the academic and private practice teaching sites were compared with the HMO and community clinics teaching sites, this variation achieved statistical significance. CONCLUSIONS: As measured by OSCE and STFM examination performance, student learning did not vary across multiple training sites. However, greater variability was found in faculty evaluations of student performance. Sources of this variation might include differing levels of student-faculty interactions across sites or subjectivity inherent to the faculty evaluation process.

Family practice residency behavioral science training: influence on graduate practice activity.

BACKGROUND AND OBJECTIVES: The educational efficacy of family practice residency behavioral science training and how various educational approaches might influence graduate practice activity are poorly understood. In this study, we compare a traditional didactic and clinical block rotation approach to a problem-based learning (PBL) and clinical, experiential behavioral science curriculum. METHODS: Surveys of pre- and post-intervention cohorts were used to assess graduates' perceptions of their understanding of broad behavioral science concepts, their competence to manage specific behavioral conditions, and their behavioral science practice activity. The two cohorts were University of California, Irvine family practice residency program graduates from 1984-1988 (58) and residency graduates from 1993-1995 (27). American Board of Family Practice (ABFP) In-service Training Examination scores were also compared. RESULTS: No significant differences were detected in self-perceived competence and ABFP examination performance. Residency graduates in the post-intervention cohort more often included depression, marital counseling, and eating disorders in their practice and reported more frequent practice activity for situational stress and sexual dysfunction. The post-intervention group reported less involvement with alcohol and substance abuse problems. This group also reported practice activity that exceeded perceived levels of competence for attention deficit disorder, learning disorders, and eating disorders. CONCLUSIONS: Participants in a PBL-clinical experiential curriculum reported higher levels of practice activity for several common behavioral problems. It seems unlikely that these differences were due to curriculum changes. Further investigation of the influence of educational and other factors on residency graduate practice activity is needed.

On-site colposcopy services in a family practice residency clinic: impact on physician test-ordering behavior, patient compliance, and practice revenue generation.

BACKGROUND: Using colposcopy as a model, we examined the impact of introducing a new diagnostic technology into the ambulatory primary care setting. METHODS: Records of patients with abnormal findings on Papanicolaou smears were reviewed from three study periods: 1 year before, 1 year after, and 5 years after initiation of on-site colposcopy services. Data analyzed include physician management decisions, site of colposcopic service, and patient compliance. Practice revenue estimates were based upon patterns of physician management and patient compliance found during each study period. RESULTS: Management of low-grade squamous intraepithelial lesions varied during each study period. By period 3, however, most patients were undergoing colposcopy (P = 0.03). High-grade squamous intraepithelial lesions were uniformly managed with colposcopy during all study periods (P < 0.001). Introduction of on-site colposcopic services resulted in a rapid shift to the on-site location for evaluation of low-grade squamous intraepithelial lesions and a more gradual shift to the on-site location for evaluation of high-grade squamous intraepithelial lesions. Patient compliance was not affected by the introduction of on-site services. On-site colposcopy resulted in a nearly 100 percent transfer of revenue to the practice, but the economic benefit was quite modest. CONCLUSIONS: Although offering on-site colposcopy services might have had some impact on physician management of low-grade squamous intraepithelial lesions, the lack of benefit regarding patient compliance, the relatively small patient volume for this procedure, and its modest impact on practice revenue cause us to question the value of including colposcopy in everyday practice.

Practice-based preceptoral residency training in family practice.

BACKGROUND AND OBJECTIVES: Highly specific accreditation requirements have led to a uniform approach in US family practice continuity training. In this paper, we describe an experiment in practice-based preceptoral continuity training. METHODS: We assessed the impact of assigning a resident to a private faculty practice for the second- and third-year levels of family practice continuity training. Data analysis included comparisons of practice demographics, clinical content, training costs, and resident assessment of training quality. RESULTS: The preceptoral resident practice better approximated community demographic patterns than did the established residency practice. However, the preceptoral resident practice had relatively fewer visits for hypertension and diabetes mellitus than did the established residency, preceptoral site faculty, and National Ambulatory Medical Care Survey practices. Resident training at the preceptoral site resulted in a positive training expense differential of $69,300 calculated on an annual per third-year resident basis. The preceptoral resident expressed high levels of satisfaction with the quality of family practice center training. CONCLUSIONS: Practice-based preceptoral continuity training in family practice appears feasible in terms of clinical content exposure and may offer substantial financial advantages. Important questions remain, particularly about the quality of teaching and supervision in a preceptoral training model.

The community health center and family practice residency training.

Community health centers (CHCs), sponsored by US Public Health Service (USPHS) Section 330, represent successful models of the application of community-oriented primary care principles. Recently, increased interest has been shown in conducting medical education programs in CHCs. The USPHS has tried to facilitate this interest, particularly through its support of establishing linkages between CHCs and family practice residency programs. In this paper, we describe an integrated CHC-family practice residency continuity training clinic program. We discuss the challenges inherent to conducting family practice residency training in the CHC, including the educational content of clinical experiences, the impact of provider productivity expectations, the academic and operational governance of the program, and the financial considerations pertinent to the integrated function of the program. We conclude that while the clinical experiences available in the CHC differ somewhat from mainstream family practice, successful adaptations can be made, and a CHC offers a rich educational environment. We also conclude that the challenges inherent to integrated CHC-family practice residency programs can be successfully addressed. Of great concern, however, are financial considerations relevant to the operation of such integrated programs. These considerations underscore the urgent need for a reassessment of the funding of ambulatory clinical medical education.

Ambulatory primary care medical education in managed care and non-managed care settings.

BACKGROUND AND OBJECTIVES: The transformation of health care delivery to a managed care approach has implications for community-based clinical education in family medicine. In this study, we compare students' clerkship experiences in managed and non-managed care environments. METHODS: Student clerkship encounter logs were used to compare clinical content, level of student-patient interactions, and types of preceptor supervision and teaching activities in five clusters of community-based practice sites. Three of the groups represented managed care settings, and two represented non-managed care settings. RESULTS: The clinical content of students' experiences at the five types of teaching sites was not significantly different. However, teaching methods and responsibility levels were different. Observational student-patient interactions occurred more frequently, and comprehensive, direct student-patient interactions occurred less frequently in both managed and non-managed care private practices. Preceptor observation of students occurred less frequently in two managed care site clusters. A transition of variable degree and timing from observational to direct student-patient interactions occurred in all but one teaching cluster. Intra-site variation existed in both student-patient interactions and preceptor teaching and supervision. CONCLUSIONS: Clinical education can be successfully conducted in managed care settings. Differences in practice settings influence student-patient interactions and preceptor teaching. These differences have implications for faculty development and student education.

Family practice obstetric ultrasound in an urban community health center. Birth outcomes and examination accuracy of the initial 227 cases.

The initial 227 consecutive prenatal ultrasound examinations by two family physicians in an urban community health center were compared with actual birth outcomes. Of 186 examinations for which follow-up information was available, 162 infants were represented. The sampling rate was 81%. No serious anomalies were noted by ultrasound. One placenta previa, one fetal death, and two unsuspected cases of twins were detected by ultrasound. These data represent one of the first detailed reports of outcomes reflecting family physicians' psychomotor and cognitive skill in the use of obstetric ultrasound. The high accuracy (92% to 96%) of correct ultrasound dating suggests that a short postgraduate continuing medical education course was effective for these two family physicians. The accuracy rate compares favorably to more rigorous training. This structured format utilizing the average of four direct measurements for ultrasound-estimated gestational age and three anatomy ratios for assessing proper imaging relationships or growth symmetry may be useful as other family physicians develop educational methods and quality-assurance protocols in this area.

Working with the resident in difficulty.

The resident in difficulty is an omnipresent and seemingly intractable problem. Some definitions of this concept are explored, as are means and methods of problem identification. Principles of successful intervention are discussed, as well as some obstacles to successful intervention. Utilization of interpersonal process recall, behavioral techniques, and a more insight-oriented approach in the remediation process are emphasized. The role of the faculty in responding to the resident in difficulty is explored, with special consideration to pitfalls and possibilities. The article concludes with a series of questions and directions for future exploration.

The impact of a health screening flow sheet on the performance and documentation of health screening procedures.

We monitored physician performance of stool occult blood testing and breast exams during health maintenance visits of patients aged 50 or greater, to assess the impact of a physician education program and the use of a health screening flow sheet on performance of these health screening procedures. During the baseline study period, use of these procedures was documented for only about 40% of patients. A physician education program (conferences reviewing appropriate health maintenance screening procedures, and information on the use of a specific health screening flow sheet) did not significantly change the rate of documented physician performance of these procedures. After a health screening flow sheet was introduced into the medical record, the rate of documented performance of these screening tests increased. However, despite the presence of the screening flow sheet in the patient record, most screening evaluations were documented only in the progress notes. These results suggest that the screening flow sheet serves primarily as a memory prompting device. The usefulness of the flow sheet as a patient data storage device appears questionable.