Ensuring the clinical competence of medical school graduates through standardized patients.

There are substantial problems with the clinical training provided to medical students and with the assessment procedure used by medical schools to ensure that students have acquired the clinical skills necessary for graduate medical education. These skills are not evaluated carefully nor systematically at any point in training or licensure. This article describes the use of standardized patients to help resolve some of these shortcomings. Standardized patients are non-physicians highly trained to function in the multiple roles of patient, teacher, and evaluator while realistically replicating a patient encounter. They are effective teachers of interviewing and physical examination skills. They can help to provide a controlled exposure to common ambulatory and difficult patient communication situations. Initial studies indicate the promise of this approach for ensuring the competence of medical school graduates.

A diagnostic fourth-year performance assessment.

The University of Massachusetts Medical Center, Worcester, attempted to develop a standardized, performance-based test battery aimed directly at assessing the critical aspects of clinical competence required for graduation from medical school. The battery used a blend of standardized patient-based and written test materials and was designed to yield a profile of scores, providing a "diagnosis" of student strengths and weaknesses on a skill-by-skill basis. Results indicate that a stable, reproducible assessment of clinical skills can be achieved in a one- to two-day test battery, depending on the specific skills measured. The resulting score profile provides faculty with important information about the clinical competence of students that is not readily available from other sources, thus improving the breadth and accuracy of student assessment. A long-term goal is that performance-based testing techniques will be incorporated into the licensure process to evaluate clinical skills and ensure the competence of graduating physicians.

Improved student learning of infant growth and development.

All physicians who care for children require a knowledge of normal growth and development in order to detect and interpret abnormalities. Yet traditional instruction tends to focus on the ill child and rarely provides students the opportunity to follow up the cases of healthy children over a prolonged period of time. A new course was developed that enabled students to observe the longitudinal developmemt of a normal infant from birth to age 16 months. At the completion of the course, a multiple-choice examination covering developmental concepts from birth to age 6 years was administered to the students who participated in this course (experimental group) and to two comparison groups. The experimental group scored substantially higher than both comparison groups on items covering birth to age 18 months. The program offers an opportunity for medical students to observe the longitudinal development of a normal infant and appears complementary to other instructional methods.

The use of paraprofessionals to teach interviewing skills.

Two nonphysician mothers were each trained to give a consistent and authentic history of a child's common medical problem. At the beginning of a Pediatric Clerkship, one half the students were randomly assigned to interview one of the trained mothers. The interviews were videotaped and then replayed in an evaluative feedback session with each student. At the end of the clerkship these students were rated significantly higher on the process of interviewing than those students who did not have the initial evaluative feedback session.

Use of trained mothers to teach interviewing skills to first-year medical students: a follow-up study.

This report describes an attempt to evaluate the effectiveness of "trained mother" interviews early in the medical school curriculum. As an adjunct to a first-year course that teaches interviewing techniques, half of the students were exposed to an interview with one of three trained mothers early in the course. This treatment interview was immediately followed by a feedback session which concentrated on the content and process of interviewing. At the end of the course, all students had an evaluative interview. Those students who had an initial interview and feedback session with a trained mother scored significantly higher on both the content and process of their interviews than the control group. This technique is an effective and efficient way to teach interviewing skills to medical students prior to entering any of their clinical clerkships. A follow-up assessment conducted one year later indicated that one interview with a trained mother is sufficient for optimal learning and that the skills learned are retained over at least that period of time.

Technical issues: logistics. AAMC.

The author states that she became interested in standardized patients (SPs) around 20 years ago as a means of developing a more uniform and effective way to provide instruction and evaluation of basic clinical skills. She reflects upon in detail: (1) the logistics of using SPs in teaching; (2) how SPs are used in assessment; (3) what aspects of performance SPs can be trained to record and evaluate; (4) issues concerning checklists; (5) evaluation of interviewing skills; (6) evaluation of written communication skills; (7) importance of defining what is being tested; (8) various kinds and uses of inter-station exercises and problems of scoring them; (9) case development and the various sources for case material; (10) ways to generate scores; (11) selecting and training SPs; (12) role of the faculty and primary importance of bedside training with real patients; and (13) pros and cons of national versus single-school efforts to use SPs. She concludes by cautioning that further research must be done before SPs can be used for high-stakes certifying and licensing examinations.

A new program to enhance the teaching and assessment of clinical skills in the People's Republic of China.

The authors describe the history and current state of development of an ongoing and still-evolving collaboration--between three Chinese medical schools, the China Medical Board of New York, Inc. (CMB), and one of the authors (PLS) and her staff at the University of Massachusetts Medical School--to enhance the teaching and assessment of clinical skills in the schools. The program grew out of a request by the three schools to the CMB for assistance in improving their process of evaluating students. Its goal is to develop a uniform and comprehensive course for teaching and assessing basic clinical skills. Standardized patients are also being used as instructors and evaluators. The program is co-sponsored by the three schools and the CMB. Background is given about the CMB, its involvement in Chinese medical education, and its American perspective on the nature of education in Chinese institutions of modern medicine and the factors that can assist and impede educational change. The authors present the mechanisms of the program, the benefits that have occurred so far, and the problems encountered in creating the model curriculum to teach clinical skills that is now being implemented by the three schools. Future plans are outlined, including the hope that the three schools will serve as a resource and disseminate some or all aspects of the new approach to similar medical schools in China, and that the program can serve as a model for implementation of similar programs in other countries.

Results of a survey on the use of standardized patients to teach and evaluate clinical skills.

In 1989, a survey was sent to each U.S. and Canadian medical school requesting information about how standardized patients are used for teaching and evaluating clinical skills, and 95% of the schools responded. Although there was widespread use of standardized patients throughout the curricula, the role and training of these patients varied markedly within a given school as well as across schools. One outcome of this survey is the development of a network to share resources, protocols, and training material to enhance the development of this educational strategy.

The association between students' research involvement in medical school and their postgraduate medical activities.

The authors examined the impact of students' research involvement during medical school on their postresidency medical activities. The three medical schools involved--The Pennsylvania State University College of Medicine (PSU), The University of Connecticut School of Medicine (UCONN), and The University of Massachusetts Medical School (UMASS)--have nearly indistinguishable applicant, matriculant, and curriculum profiles. However, at PSU a research project is a curriculum requirement for students who did not do medical research prior to entering medical school. Questionnaires were sent to all graduates from the classes of 1980, 1981, and 1982. A total of 567 graduates completed the questionnaires, an overall response rate of approximately 76%. Medical school research experience was reported by 83% (183) of the PSU graduates, 34% (52) of the UCONN graduates, and 28% (54) of the UMASS graduates. When compared on a school-by-school basis, the graduates from the three schools did not differ with respect to residency specialty training, fellowship training, academic appointments, career practice choices, or postgraduate research involvement. However, when all the graduates studied were examined as a single group, medical school research experience was found to be strongly associated with postgraduate research involvement.

Positive effects of a clinical performance assessment program.

Since 1986, there has been a clinical performance assessment program for fourth-year students at the University of Massachusetts Medical School. Students interact with several standardized patients (SPs) and complete other tasks such as interpretation of electrocardiograms and interpretation of X-rays. Scores are generated both by checklists and rating forms completed by the SPs and by paperwork completed by the students at the end of each encounter. Since 1986, students have been asked how frequently they have been observed by faculty and residents as they interacted with actual patients; the students report that such observations have markedly increased. Since 1989, there has been increased feedback to students by the attending faculty during and following clinical rotations. Although it is difficult to claim cause and effect, it is clear that since the inception of this exercise, the faculty have made a conscious effort to improve students' clinical skills by providing increased observation and feedback.

An assessment of the clinical skills of fourth-year students at four New England medical schools.

This paper describes a collaborative effort among five New England medical schools to assess important clinical skills of fourth-year medical students graduating in the class of 1988; results are presented from the four schools that provided sufficient data. Faculty from each school developed 36 patient cases representing a variety of common ambulatory-care problems. Over the course of a day, each student, on average, interacted with 16 different standardized patients, who were nonphysicians trained to accurately and consistently portray a patient in a simulated clinical setting. The students obtained focused histories, performed relevant physical examinations, and provided the patients with education and counseling. At each school, the performance of a small number of the students fell below standards set by the faculty. These deficiencies were not detected with the evaluation strategies currently being used. Although the use of standardized patients should not substitute for the process of faculty observing students as they interact with real patients, it appears that standardized patients can provide faculty with important information, not readily available from other sources, about students' performances of essential clinical activities and the levels of their clinical skills.

The Medical Education Evaluation Program of the state of Ohio.

In October 1988, seven foreign medical graduates participated in the first administration of the examination devised by the Medical Education Evaluation Program (MEEP) mandated by the State Medical Board of Ohio. The MEEP was established to provide an objective evaluation of an applicant's clinical competencies; passing the MEEP examination was intended to certify that the applicant's clinical skills were comparable to those of a medical student graduating from a school accredited by the Liaison Committee on Medical Education. An applicant who successfully passed the MEEP examination and fulfilled the other Ohio licensure requirements would be eligible to take the Federation Licensing Examination (FLEX) and apply for an unrestricted license to practice medicine in Ohio. The paper describes the origin and development of the MEEP examination and the testing modalities selected (multiple-choice examinations and the use of standardized patients). Four fundamental areas were tested; these are named and described, along with the method for calculating scores for each area and the criteria for passing the different components of the examination. Although the small sample size prohibited meaningful data analysis for the performance of the first group of MEEP candidates, the MEEP examination appears to meet psychometric standards of certifying and licensing examinations, based on data from comparable tests taken by beginning fourth-year medical students in New England and NBME Part III examination examinees. Some potential pitfalls of the MEEP examination are mentioned, as well as the fact that it presents a challenge to boards of medical examiners of other states to implement performance-based assessments of physicians who graduate from non-accredited medical schools.

Pilot study of the use of the ECFMG clinical competence assessment to provide profiles of clinical competencies of graduates of foreign medical schools for residency directors. Educational Commission for Foreign Medical Graduates.

PURPOSE: To conduct the first of a series of pilot projects of the clinical competence assessment (CCA) of the Educational Commission for Foreign Medical Graduates (ECFMG) in order to provide profiles of clinical competencies of graduates of foreign medical schools for residency directors in the United States and for governments and institutions in other countries. METHOD AND RESULTS: In September 1992 the first pilot project of the ECFMG CCA was conducted for a program director who wanted to evaluate ten first-year residents in a midwestern U.S. program. The CCA consists of integrated clinical encounters with ten standardized patients, 60 laser videodisc pictorials, and analysis of test items of previously completed ECFMG certification examinations. Profiles of the following clinical competencies were provided to the program director: data gathering (history and physical examination), interviewing and interpersonal skills, diagnosis and management skills, interpretation of diagnostic and laboratory procedures, written communication of information to the health care team, and spoken-English proficiency. The profiles were provided as individual scores compared with mean scores of a reference group of 525 first-year residents who took the CCA at four U.S. assessment centers, and as percentile scores with a range of one standard error of measurement. CONCLUSION: The individual performance data in this first pilot project were valuable to the program director, who used them to supplement scores on a written examination during the first residency year. The pilot project has shown the ECFMG CCA to be a useful tool for program directors to evaluate applicants and residents who are graduates of foreign medical schools.

The use of patient instructors to teach interviewing skills.

A program which utilizes patient instructors (PI) to teach and evaluate interviewing skills of pharmacy students is described. The PIs were programmed with a history of either hypertension, chronic pulmonary disease, or congestive heart failure. Content areas within each history included past and present drug therapy, adverse drug reactions, drug interactions, as well as socioeconomic and other factors affecting therapy. The interviews were done with small groups of students and followed by PI evaluation of interview content and process. The majority of students felt the experience helped them assess and improve their interviewing skills and agreed the program should continue. The advantages of utilizing PIs to teach interviewing skills are discussed.