Key challenges in simulated patient programs: an international comparative case study.

BACKGROUND: The literature on simulated or standardized patient (SP) methodology is expanding. However, at the level of the program, there are several gaps in the literature. We seek to fill this gap through documenting experiences from four programs in Australia, Canada, Switzerland and the United Kingdom. We focused on challenges in SP methodology, faculty, organisational structure and quality assurance. METHODS: We used a multiple case study method with cross-case synthesis. Over eighteen months during a series of informal and formal interactions (focused meetings and conference presentations) we documented key characteristics of programs and drew on secondary document sources. RESULTS: Although programs shared challenges in SP methodology they also experienced differences. Key challenges common to programs included systematic quality assurance and the opportunity for research. There were differences in the terminology used to describe SPs, in their recruitment and training. Other differences reflected local conditions and demands in organisational structure, funding relationships with the host institution and national trends, especially in assessments. CONCLUSION: This international case study reveals similarities and differences in SP methodology. Programs were highly contextualised and have emerged in response to local, institutional, profession/discipline and national conditions. Broader trends in healthcare education have also influenced development. Each of the programs experienced challenges in the same themes but the nature of the challenges often varied widely.

Using an objective structured clinical examination (OSCE) to assess multiple physician competencies in postgraduate training.

BACKGROUND: Competency-based models of medical education require reliable and valid assessment of multiple physician roles. AIMS: To develop and evaluate an objective structured clinical examination (OSCE) designed to assess 7 physician competencies (CanMEDS Roles). METHODS: Twenty four candidates from 4 neonatal-perinatal medicine training programs participated in a 10-station OSCE. Ten 5-point rating scales were developed and used to assess the CanMEDS Roles of Medical Expert, Communicator, Collaborator, Manager, Health Advocate, Scholar and Professional. Three descriptors of performance anchored the ratings. For each station, examiners completed appropriate CanMEDS ratings, a station-specific binary checklist and an overall process-related global rating. Trained standardized patients (SP) and standardized health professionals (SHP) completed rating scales that assessed verbal and non-verbal expression, empathy and coherence as well as the overall global rating. RESULTS: Each station incorporated 3-5 physician Roles. Interstation alpha was 0.80 for checklist scores and 0.88 for examiners' overall global rating. Median interstation alpha for individual CanMEDS ratings was 0.72 (range 0.08-0.91). There were significant correlations between examiner Medical Expert scores and SP/SHP overall global scores and between examiner Communicator scores and 4 SP/SHP assessments of communication skills. Second year trainees' CanMEDS scores for each competency were significantly higher than those of first year trainees (p < 0.05). CONCLUSIONS: The OSCE may be useful as a reliable and valid method of simultaneously assessing multiple physician competencies.

Teaching Primary Care Genetics: A Randomized Controlled Trial Comparison.

BACKGROUND AND OBJECTIVE: Given the increasing discussions of the impact of genetic medicine within family medicine, it is important to determine the most effective way of teaching this material to family medicine residents (FMRs). The objective of this study was to evaluate and compare the impact of three methods of delivering primary care genetic content to FMRs. METHODS: Curriculum materials and assessment tools were created to teach and evaluate knowledge, skills, and attitudes around four core competencies in primary care genetics, with a focus on hereditary colorectal cancer (CRC). Participants were randomly allocated to four learning conditions: (1) no intervention (control), (2) web-based module outlining genetic concepts applied to CRC, (3) live presentation of the web-based material, (4) live presentation and subsequent standardized patient (SP) encounter. Three months later, all participants completed a written knowledge test, attitude survey, and a standardized patient-based performance assessment. RESULTS: Sixty FMRs completed the study. All three educational interventions resulted in significantly improved outcome measures in knowledge and skills but not attitudes, compared to control. There was no significant difference in outcomes between intervention groups. CONCLUSION: FMRs acquired knowledge and improved skills in genetic medicine with three educational methods. Resources such as faculty expertise in genetic medicine and cost should guide decisions on curricular development for this rapidly expanding field. This may be especially relevant for programs with distributed teaching sites.

The 'gut bucket': a novel training tool for standardised patients.

BACKGROUND: Standardised patients (SPs) are often asked to portray complex physical roles in which authenticity is paramount; however, SPs come from a variety of backgrounds, and may lack knowledge of basic anatomy and physiology that can hamper their portrayal. This lack of knowledge can lead to gaps in accuracy and credibility. CONTEXT: In our efforts to bridge the gap and create training that would lead to authentic and confident portrayals, we developed the 'gut bucket'. This three-dimensional learning tool is tactile, portable, simple and affordable. INNOVATION: The 'gut bucket' has life-size abdominal organs made from fabric resting in a standard plastic washbasin. The organs can be easily removed and manipulated to simulate different disease states (for example: appendicitis, kidney stones or cholecystitis). The tool is visually engaging and durable, encouraging SPs to handle the organs and become familiar with their size and placement within the basin and on themselves. IMPLICATIONS: The SPs in our programme found that the 'gut bucket' provided a deeper level of knowledge, made them more confident in their portrayal and enhanced their experience, and they generally preferred the 'gut bucket' over traditional methods of training. This was demonstrated using 'gut bucket' training evaluation forms. The 'gut bucket' can easily be incorporated into training sessions to provide a kinaesthetic approach as well as increased engagement for the SPs. We hope others may benefit by sharing our experience with the 'gut bucket'.

In urban and rural India, a standardized patient study showed low levels of provider training and huge quality gaps.

This article reports on the quality of care delivered by private and public providers of primary health care services in rural and urban India. To measure quality, the study used standardized patients recruited from the local community and trained to present consistent cases of illness to providers. We found low overall levels of medical training among health care providers; in rural Madhya Pradesh, for example, 67 percent of health care providers who were sampled reported no medical qualifications at all. What's more, we found only small differences between trained and untrained doctors in such areas as adherence to clinical checklists. Correct diagnoses were rare, incorrect treatments were widely prescribed, and adherence to clinical checklists was higher in private than in public clinics. Our results suggest an urgent need to measure the quality of health care services systematically and to improve the quality of medical education and continuing education programs, among other policy changes.

Defining responsibilities of simulated patients in medical education.

BACKGROUND: Simulated patients (SPs) play a critical role in medical education. The development of SP methodology has resulted in wide ranging responsibilities. For SPs to work effectively, we believed it was important to clearly articulate their responsibilities, and that this would be best achieved by consultation with all stakeholders-SPs, students, tutors, and administrators. METHODS: As part of a quality assurance initiative, we designed a questionnaire and focus group study to explore stakeholders' perceptions of the responsibilities of SPs in teaching. Convenience and purposive sampling was used to recruit participants to questionnaires and focus groups, respectively. Data were analyzed thematically. RESULTS: Eighty-six questionnaires were collected, and six focus groups were conducted. Five sets of guidelines on responsibilities were produced. In addition, guidelines were established for feedback that SPs and tutors could use to maximize impact. DISCUSSION: The results highlight the complexity of SP-based teaching. Clarification of all stakeholders' responsibilities demonstrates the importance of a team approach to SP-based teaching. Focusing attention on just one set of stakeholder's responsibilities is unlikely to improve perception of quality. The process for developing the guidelines may be valuable for those who work with SPs. Stakeholder engagement is likely to ensure greater commitment than those developed by faculty.

Psychometric properties of an integrated assessment of technical and communication skills.

PURPOSE: The Integrated Procedural Performance Instrument (IPPI) consists of clinical scenarios in which bench-top models are positioned to simulated patients. Trainees are required to perform technical skills while engaging with the patient. The purpose of this study was to determine whether an IPPI format examination could discriminate between different levels of trainees. METHODS: Sixteen fourth-year medical students and 16 first-year surgery residents participated in 4 IPPI scenarios. Videotaped performances were scored by 2 blinded independent clinician raters on previously validated instruments: checklist of technical skills, Global Rating Scale of technical skills, and communication scale. We conducted separate mixed design analyses of variance (level x cases) on the 3 scales. RESULTS: Residents performed better than medical students on the checklist (74% vs 60%, P < .05), the Global Rating Scale of technical skills (75% vs 56%, P < .01), and the coherence communication subscale (79% vs 69%, P < .05). CONCLUSIONS: An IPPI examination discriminated between students' and residents' technical skills and coherence in communication skills. It also highlighted a potential gap in the training of residents' communication skills.

Teaching communication skills using the integrated procedural performance instrument (IPPI): a randomized controlled trial.

BACKGROUND: The Integrated Procedural Performance Instrument (IPPI) uses various bench-top models positioned to standardized patients (SP) to recreate realistic clinical encounters. This study assessed the effectiveness of using an IPPI format as a teaching tool for communication skills. METHOD: Thirty-two participants underwent 2 videotaped IPPI scenarios before randomization into 2 groups--experimental (SP-led feedback) or control (no feedback). Participants then completed 2 further IPPI format scenarios. Videotapes were scored by 2 blinded independent raters using validated assessment scales (communication and technical). RESULTS: The experimental group performed significantly better on the communication scores following feedback compared with the control group (mean 77% vs 66%, P < .05). No difference in scores for technical skills post-intervention were demonstrated (checklist: experimental mean = 64% vs control = 59%, P = .40; global ratings: experimental mean = 66% vs no control = 62%, P = .37). CONCLUSIONS: The IPPI is an effective tool for teaching communication skills in residents and medical students and should be considered for incorporation into undergraduate and surgical curricula.

Simulated patients vs. standardized patients in objective structured clinical examinations.

OBJECTIVES: To describe the use of patient-actors as educators in a senior-level pharmacy practice course, and to contrast the value and application of "standardized patient" and "simulated patient" educational methodologies. DESIGN: The objective structured clinical examination (OSCE) of the licensing examination were utilized during and at the end of the course along with external assessment to determine the impact of this educational methodology. Interviews with a randomly selected cohort of 14 students were undertaken 3 years after graduation and licensure to evaluate long-term impact of this course. ASSESSMENT: Overall, students responded positively to the shift from "standardized" patients to "simulated" patients, recognizing their value in teaching clinical and pharmaceutical care skills. Concerns were expressed regarding objectivity in assessment and individual grading. Over 98% of students successfully passed the OSCE component of the licensing examination. Long-term follow-up suggests students valued this approach to education and that it provided them with a foundation for better understanding of the psychosocial needs of patients in practice. CONCLUSIONS: Simulated-patient educators can play an important role in the pharmacy curriculum, and can complement practitioner-educators in providing students with a real-world context for understanding complex patient care needs.

Peer-mentoring workshop for continuous professional development.

OBJECTIVE: To develop a Professional Skills Enhancement Workshop (PSEW) to assist practitioners who require skills training to maintain competency and meet new standards of practice. Participants for this workshop were identified as those pharmacists who completed the peer review assessment process and who did not meet standards of practice expectations. DESIGN: The full-day workshop consists of a half-day introduction to use of clinical drug information resources and approaches to addressing practice-based questions. The second part of the workshop introduces participants to the use of structured patient-interviewing techniques to elicit information using standardized patients. Participants in the workshop completed self-assessments as well as course evaluations. Subsequent to completion of the course, participants rechallenged the peer review assessment process, a test of their clinical skills consisting of a written test of clinical knowledge and an objective structured clinical examination (OSCE), to provide objective evidence of skills acquisition. ASSESSMENT: Over 90% of participants "agreed" or "strongly agreed" that the PSEW was helpful in reacquainting them with current standards of professional practice. Sixty-nine percent of participants who completed the peer review assessment rechallenge process following completion of the course were able to meet standards of practice expectations. CONCLUSIONS: In developing continuous professional development programs, first identifying the needs of all practitioners is essential. The PSEW provides one model for skills training for practitioners who, for a variety of reasons, may not have maintained the expected level of competency.

The effectiveness of unannounced standardised patients in the clinical setting as a teaching intervention.

PURPOSE: Teaching medical students to spontaneously identify biopsychosocial issues (e.g. family violence) remains a challenge. We examined the extent to which using unannounced standardised patients (SPs) presenting in a clerk's clinical setting could assist with this teaching challenge. METHODS: All clerks attended a family violence seminar in their family medicine rotation. Intervention students additionally saw an unannounced SP portraying 1 of 2 scenarios in their preceptor's office during the rotation, and received immediate feedback about their performance. An end of rotation objective structured clinical examination (OSCE) included an SP presentation similar to that seen by the intervention students. RESULTS: Clerks who received the intervention demonstrated increased questioning about family violence, from 0% (0 of 29 students) to 19% (5 of 26 students) in 1 OSCE scenario (P = 0.019), and from 40% (12 of 30 students) to 76% (19 of 25 students) in the other (P = 0.007). CONCLUSIONS: Seeing unannounced SPs had a dramatic effect on later student performance. This potentially powerful intervention could be applied to a range of clinical issues.