Formative assessment promotes learning in undergraduate clinical clerkships.

INTRODUCTION: Clinical clerkships, typically situated in environments lacking educational structure, form the backbone of undergraduate medical training. The imperative to develop strategies that enhance learning in this context is apparent. This study explored the impact of longitudinal bedside formative assessment on student learning in a medical clerkship. METHODS: We studied a class of 4th-year students completing a 14-week medical clerkship at the University of Cape Town in South Africa. Clinician educators assessed student performance during weekly bedside teaching sessions using blinded patient encounters (in which students had no prior knowledge of the patient's diagnosis or access to the clinical records). Student feedback was standardised using performance rating scales. The impact of formative assessment on student learning was determined from questionnaire responses. RESULTS: A total of 575 formative assessments took place during the study period. Students perceived blinded patient encounters to be a valuable learning activity that improved their clinical reasoning skills and assessed progress fairly. They reported that feedback helped inform them of their level of competence and learning needs, motivated them to read more, and significantly improved their participation in patient-centred learning activities. Participating clinicians agreed that this formative assessment strategy enhanced the learning potential of bedside teaching sessions. CONCLUSIONS: Longitudinal formative assessment, using blinded patient encounters, was successfully integrated into undergraduate clerkship bedside teaching. According to both students and staff this assessment strategy enhanced bedside learning and improved student participation in patient-centred learning activities during the clerkship.

Formative Assessment Promotes Learning in Undergraduate Clinical Clerkship

INTRODUCTION: Clinical clerkships; typically situated in environments lacking educational structure; form the backbone of undergraduate medical training. The imperative to develop strategies that enhance learning in this context is apparent. This study explored the impact of longitudinal bedside formative assessment on student learning in a medical clerkship.METHODS: We studied a class of 4th-year students completing a 14-week medical clerkship at the University of Cape Town in South Africa. Clinician educators assessed student performance during weekly bedside teaching sessions using blinded patient encounters (in which students had no prior knowledge of the patient's diagnosis or access to the clinical records). Student feedback was standardised using performance rating scales. The impact of formative assessment on student learning was determined from questionnaire responses. RESULTS: A total of 575 formative assessments took place during the study period. Students perceived blinded patient encounters to be a valuable learning activity that improved their clinical reasoning skills and assessed progress fairly. They reported that feedback helped inform them of their level of competence and learning needs; motivated them to read more; and significantly improved their participation in patient-centred learning activities. Participating clinicians agreed that this formative assessment strategy enhanced the learning potential of bedside teaching sessions. CONCLUSIONS: Longitudinal formative assessment; using blinded patient encounters; was successfully integrated into undergraduate clerkship bedside teaching. According to both students and staff this assessment strategy enhanced bedside learning and improved student participation in patient-centred learning activities during the clerkship

Evaluating the spoken English proficiency of graduates of foreign medical schools.

PURPOSE: The purpose of this study was to gather additional evidence for the validity and reliability of spoken English proficiency ratings provided by trained standardized patients (SPs) in high-stakes clinical skills examination. METHOD: Over 2500 candidates who took the Educational Commission for Foreign Medical Graduates' (ECFMG) Clinical Skills Assessment (CSA) were studied. The CSA consists of 10 or 11 timed clinical encounters. Standardized patients evaluate spoken English proficiency and interpersonal skills in every encounter. Generalizability theory was used to estimate the consistency of spoken English ratings. Validity coefficients were calculated by correlating summary English ratings with CSA scores and other external criterion measures. Mean spoken English ratings were also compared by various candidate background variables. RESULTS: The reliability of the spoken English ratings, based on 10 independent evaluations, was high. The magnitudes of the associated variance components indicated that the evaluation of a candidate's spoken English proficiency is unlikely to be affected by the choice of cases or SPs used in a given assessment. Proficiency in spoken English was related to native language (English versus other) and scores from the Test of English as a Foreign Language (TOEFL). DISCUSSION: The pattern of the relationships, both within assessment components and with external criterion measures, suggests that valid measures of spoken English proficiency are obtained. This result, combined with the high reproducibility of the ratings over encounters and SPs, supports the use of trained SPs to measure spoken English skills in a simulated medical environment.

The management of patient encounter time in a high-stakes assessment using standardized patients.

OBJECTIVES: The purpose of this study was to gather information regarding the appropriateness of the length of time allotted for candidates to complete the history taking and physical examination tasks in a high-stakes standardized patient (SP) assessment. DESIGN: Data were collected on actual time used by 1548 examinees for each of their 10 standardized patient encounters, for which a maximum of 15 minutes was allotted, but not required. SETTING: The Clinical Skills Assessment Center of the Educational Commission for Foreign Medical Graduates (ECFMG), Philadelphia, Pennsylvania, USA. SUBJECTS: Graduates of foreign medical schools who are seeking ECFMG certification. RESULTS: The average time spent with the standardized patient was 13.3 minutes, suggesting that the 15-minute time limit was sufficient. A positive correlation was found between data-gathering scores and patient interview times. Candidates did tend to spend more time with SPs presenting with cases involving complex histories, as well as with cases of chronic conditions. CONCLUSIONS: Candidate time use varied as a function of type of clinical encounter, providing additional evidence of the content validity of the Clinical Skills Assessment.

The performance of foreign medical graduates on the National Board of Medical Examiners (NBME) standardized patient examination prototype: a collaborative study of the NBME and the Educational Commission for Foreign Medical Graduates (ECFMG).

OBJECTIVES: The purpose of the study was to explore foreign medical graduates' (FMGs) performance on a clinical skills (SPX) examination. The National Board of Medical Examiners (NBME) is in the process of developing an SPX for potential use in the United States Medical Licensing Examination (USMLE). The Educational Commission for Foreign Medical Graduates (ECFMG) is developing the Clinical Skills Assessment (CSA) as an additional requirement for FMGs who wish to be certified by ECFMG. DESIGN: Thirty-three FMGs and 151 United States medical students (USMSs) took the SPX during the winter of 1996 as part of the ongoing pilot studies conducted by the NBME. Four clinical skill areas were assessed: history-taking, physical examination, communication and interpersonal skills. The examination used in this research consisted of 12 cases. The examination utilizes standardized patients (SPs) who are trained to document examinee behaviours and evaluate the communication component of the test. The SPs were also trained to evaluate the English proficiency of the candidates. Candidates were also administered the Test of Spoken English developed by the Educational Testing Services (ETS). SETTING: The examination was conducted in one medical school which served as an SPX centre for NBME pilot studies. SUBJECTS: Thirty-three foreign medical students and 151 US medical students. RESULTS: The indications were that the majority of candidates in both groups felt the examination was moderately fair but 78% of FMGs felt moderately pressed for time, vs. 80% of the USMSs who did not feel pressed for time. Reliabilities obtained for the various SPX components were somewhat higher for the FMGs reflecting the heterogeneity of this group. CONCLUSIONS: The NBME-ECFMG collaborative study yielded important information regarding the NBME SPX prototype as a performance measure for FMGs.

Internists' and surgeons' attitudes toward guns and firearm injury prevention.

BACKGROUND: The high rates of death, injury, and long-term disability related to firearms in the United States have led to growing concern in the health care community. Medical organizations and journals are devoting increasing attention to firearm violence as a public health problem; however, few reports discuss physician attitudes toward guns and prevention of firearm-related injury. OBJECTIVE: To determine internists' and surgeons' attitudes toward guns and firearm injury prevention. DESIGN: Analysis of results of a structured telephone interview. SETTING: Internal medicine and surgical offices. PARTICIPANTS: 457 internists and 458 surgeons. MEASURES: 55 questions that covered six domains: experience with firearms, knowledge about clinical sequelae of firearm injury, knowledge about public policies on firearm violence, attitudes toward public policies on firearm violence, clinical practice behavior, and education and training. RESULTS: The interview response rate was 45.3%, with a compliance rate of 82.5% and a 95% probability (error rate, +/- 5%). Ninety-four percent of internists and 87% of surgeons believe firearm violence is a major public health issue. A majority of internists and surgeons also support community efforts to enact legislation to restrict the possession or sale of handguns (84% and 64%, respectively). Furthermore, although 84% of internists and 72% of surgeons believe that physicians should be involved with firearm injury prevention, less than 20% of respondents usually engage in some form of firearm injury prevention practice in patient care. CONCLUSION: Many internists and surgeons think that firearm injuries are a public health issue of growing importance, that physicians should incorporate firearm safety screening and counseling into their practice, that physicians should join community efforts to regulate handguns, and that specific gun regulation measures should be adopted as public policy.

Lessons learned from six years of international administrations of the ECFMG's SP-based clinical skills assessment.

PURPOSE: The Educational Commission on Foreign Medical Graduates (ECFMG) conducted international clinical skills assessments (CSAs) to evaluate the readiness of foreign medical graduates to enter U.S. residency programs, to validate national medical examinations in other countries, and to introduce other countries to new methods of evaluating medical students. METHOD: The ECFMG conducted CSA studies in the United States, Israel, Spain (Madrid and Barcelona), Ukraine, and Brazil between 1989 and 1995. ECFMG staff worked with local teams in following a seven-phase implementation process. The CSAs were conducted in each country's native language, and clinical cases were translated from English and culturally adapted. A total of 636 examinees took a ten-station standardized patient-based examination. RESULTS: A comparison of test results indicated stable psychometric properties and similar patterns of relationships among test components across all six countries. In every country, physical-examination and patient-note mean scores were lower than were history-taking scores, indicating the possibility of common skill deficiencies. CONCLUSION: The successful completion of international CSA projects in six countries suggests that high-quality standardized CSA projects are feasible and can be implemented from long distances.

Retraining physicians for primary care. A study of physician perspectives and program development.

OBJECTIVE: To determine the number and kinds of programs that medical schools and managed care organizations offer or plan to offer to retrain physician specialists to practice primary care medicine and to discover physicians' attitudes toward such retraining. DESIGN: A survey was mailed in 1994 to all 126 medical schools and the 19 largest US managed care organizations to collect detailed information about existing and potential retraining programs. Physicians' attitudes toward retraining were elicited from participants in 3 geographically diverse focus groups. Selected specialists were polled through the national survey of the American Medical Association's Socioeconomic Monitoring System to ascertain the demand for retraining. RESULTS: The majority of institutions contacted perceived a need for retraining, but few programs had been established. Programs being "considered" varied widely in duration, class size, target audience, accreditation, and projected training settings. Although unenthusiastic about retraining, physicians preferred programs that would expand their patient base, maintain the practice population, be inexpensive and close to home, and provide hands-on training in the eventual practice environment. Physicians also preferred a goal-oriented, part-time retraining program in a large group practice or managed care setting that would allow them to practice their specialty while retraining. Few planned or existing programs incorporate many of these features. The most likely candidates for retraining are subspecialty physicians who currently provide some primary care and are employed by a medical plan. CONCLUSIONS: Despite efforts by those who perceive that a need for more generalist physicians is stimulating interest in retraining specialists and subspecialists to provide primary medical care, physician interest and program availability remain low, and programs under development are not being designed to attract those who may seek retraining. This situation is probably fortuitous, because changed perceptions about the adequacy of the generalist physician workforce since the beginning of this study have diminished the call for retraining.

Graduates of foreign medical schools: progression to certification by the Educational Commission for Foreign Medical Graduates.

The application process leading to certification by the Education Commission for Foreign Medical Graduates (ECFMG) was studied using the group of 9,491 graduates of foreign medical schools who initiated their applications in 1988. Using the ECFMG's database, these applicants' countries of citizenship, examination histories, certification status, and exchange visitor status were determined for a period of seven years and 9 months, ending in September 1995. Within that time, 45% of these applicants became ECFMG-certified, and 26% of that group entered residency programs accredited by the Accreditation Council for Graduate Medical Education. Of the total number of non-U.S. citizens (2,243) who entered such programs, 61% did so as exchange visitor physicians. The remainder had other kinds of visa status, such as permanent resident, or had become U.S. citizens. Although U.S. citizens and non-U.S. citizens achieved certification at the same rate (45%), the U.S. citizens had greater success in obtaining positions in graduate medical education (GME). Seventy-four percent of the ECFMG-certified certified U.S. citizens entered GME, versus 57% of the ECFMG-certified non-U.S. citizens. The status of the 55% of the 9,491 applicants who did not obtain certification is discussed; a portion of this group of this group continues to pursue certification. Details and requirements of the ECFMG certification process are also described.

Nancy E. Gary, MD: providing HCFA with a new perspective on the 'hassle factor':. Interview by Janice C. Simmons.

As senior medical adviser to Health Care Financing Administration head, Gail Wilensky, Dr. Gary provides the agency with a physician's view on issues and has become a liaison with the medical community--with an open ear to the regulatory hassles that beset practicing physicians.

Changes in admissions, lengths of stay, and discharge diagnoses at a major university-affiliated teaching hospital: implications for medical education.

The implications for medical education of a declining average length of stay (ALOS) and changes in numbers of hospital admissions have been the subject of considerable discussion. The changing hospital reimbursement schedules and the rise of managed-care systems have altered patterns of hospital use and consequently posed new problems for the education of medical students and residents. Between 1980 and 1986 in one university-affiliated hospital, the ALOS for patients with any of the most frequent discharge diagnoses for either of those years had declined 34% since 1980, despite a 59% increase in the number of patients with those diagnoses. Patients with any one of 21 of the 88 most frequent discharge diagnoses for 1986 were hospitalized for less than one day. Those patients with another eight of the disorders had an ALOSs of between one and two days; some of these patients had serious and common gastrointestinal disorders. Discharge diagnoses were categorized by the hospital service in which students and residents would have rotations and were analyzed for changes in case mix. This study showed that for the hospital's medical students and residents to be exposed to the same case mix of clinical disorders that were seen in-hospital in 1980, it is now necessary for them to have experience in the ambulatory setting.