Resident education in orthopaedic trauma: the future role of competency-based medical education.

As residency training programmes around the globe move towards competency-based medical education (CBME), there is a need to review current teaching and assessment practices as they relate to education in orthopaedic trauma. Assessment is the cornerstone of CBME, as it not only helps to determine when a trainee is fit to practice independently, but it also provides feedback on performance and guides the development of competence. Although a standardised core knowledge base for trauma care has been developed by the leading national accreditation bodies and international agencies that teach and perform research in orthopaedic trauma, educators have not yet established optimal methods for assessing trainees' performance in managing orthopaedic trauma patients. This review describes the existing knowledge from the literature on assessment in orthopaedic trauma and highlights initiatives that have recently been undertaken towards CBME in the United Kingdom, Canada and the United States. In order to support a CBME approach, programmes need to improve the frequency and quality of assessments and improve on current formative and summative feedback techniques in order to enhance resident education in orthopaedic trauma. Cite this article: Bone Joint J 2016;98-B:1320-5.

Reflections on current methods for evaluating skills during joint replacement surgery: a scoping review.

Valid and reliable techniques for assessing performance are essential to surgical education, especially with the emergence of competency-based frameworks. Despite this, there is a paucity of adequate tools for the evaluation of skills required during joint replacement surgery. In this scoping review, we examine current methods for assessing surgeons' competency in joint replacement procedures in both simulated and clinical environments. The ability of many of the tools currently in use to make valid, reliable and comprehensive assessments of performance is unclear. Furthermore, many simulation-based assessments have been criticised for a lack of transferability to the clinical setting. It is imperative that more effective methods of assessment are developed and implemented in order to improve our ability to evaluate the performance of skills relating to total joint replacement. This will enable educators to provide formative feedback to learners throughout the training process to ensure that they have attained core competencies upon completion of their training. This should help ensure positive patient outcomes as the surgical trainees enter independent practice.

Transatlantic comparison of the competence of surgeons at the start of their professional career.

BACKGROUND: Although the objective in European Union and North American surgical residency programmes is similar-to train competent surgeons-residents' working hours are different. It was hypothesized that practice-ready surgeons with more working hours would perform significantly better than those being educated within shorter working week curricula. METHODS: At each test site, 21 practice-ready candidate surgeons were recruited. Twenty qualified Canadian and 19 qualified Dutch surgeons served as examiners. At both sites, three validated outcome instruments assessing multiple aspects of surgical competency were used. RESULTS: No significant differences were found in performance on the integrative and cognitive examination (Comprehensive Integrative Puzzle) or the technical skills test (Objective Structured Assessment of Technical Skill; OSATS). A significant difference in outcome was observed only on the Patient Assessment and Management Examination, which focuses on skills needed to manage patients with complex problems (P < 0.001). A significant interaction was observed between examiner and candidate origins for both task-specific OSATS checklist (P = 0.001) and OSATS global rating scale (P < 0.001) scores. CONCLUSION: Canadian residents, serving many more working hours, perform equivalently to Dutch residents when assessed on technical skills and cognitive knowledge, but outperformed Dutch residents in skills for patient management. Secondary analyses suggested that cultural differences influence the assessment process significantly.

The use of virtual reality for training in carotid artery stenting: a construct validation study.

BACKGROUND: Given that carotid artery stenosis (CAS) intervention is procedurally difficult, possesses an extensive learning curve, and involves a grave list of potential complications, construct validation of new non-clinical training devices is of increasing importance. PURPOSE: To evaluate the construct validity of the Procedicus-Virtual Interventional Simulator Trainer (Procedicus-VIST) and its use as a training tool. MATERIAL AND METHODS: Sixteen interventionalists (15 males, one female; mean interventional radiology [IR] experience >11 years) and 16 medical students (15 males, one female; no IR experience) received 1 hour of didactic instruction followed by an hour of familiarization training. Subjects then attempted to complete a carotid artery stenting procedure within 1 hour while their performance metrics were recorded. All participants completed a qualitative exit survey of subjective parameters using a visual analog scale. RESULTS: Procedure and fluoroscopic time was 8.7 and 8.7 min greater in the novice group (P=0.0066 and P=0.0031), respectively. There were no significant differences in performances between the two groups in the remaining metrics of cine loops (number recorded), tool/vessel ratio, coverage percentage, and placement accuracy or residual stenosis. Contrast measurement metrics were found to be too imprecise for statistical analysis. Experienced and novice opinions differed significantly for six of 10 subjective parameters. No statistically significant difference in video-gaming habits was demonstrated. CONCLUSION: With the exception of the metrics of performance time and fluoroscopic use, construct validity of the Procedicus-VIST carotid metrics were not confirmed. Virtual reality simulation as a training method was valued more by novices than by experienced interventionalists.

Getting teams to talk: development and pilot implementation of a checklist to promote interprofessional communication in the OR.

BACKGROUND: Pilot studies of complex interventions such as a team checklist are an essential precursor to evaluating how these interventions affect quality and safety of care. We conducted a pilot implementation of a preoperative team communication checklist. The objectives of the study were to assess the feasibility of the checklist (that is, team members' willingness and ability to incorporate it into their work processes); to describe how the checklist tool was used by operating room (OR) teams; and to describe perceived functions of the checklist discussions. METHODS: A checklist prototype was developed and OR team members were asked to implement it before 18 surgical procedures. A research assistant was present to prompt the participants, if necessary, to initiate each checklist discussion. Trained observers recorded ethnographic field notes and 11 brief feedback interviews were conducted. Observation and interview data were analyzed for trends. RESULTS: The checklist was implemented by the OR team in all 18 study cases. The rate of team participation was 100% (33 vascular surgery team members). The checklist discussions lasted 1-6 minutes (mean 3.5) and most commonly took place in the OR before the patient's arrival. Perceived functions of the checklist discussions included provision of detailed case related information, confirmation of details, articulation of concerns or ambiguities, team building, education, and decision making. Participants consistently valued the checklist discussions. The most significant barrier to undertaking the team checklist was variability in team members' preoperative workflow patterns, which sometimes presented a challenge to bringing the entire team together. CONCLUSIONS: The preoperative team checklist shows promise as a feasible and efficient tool that promotes information exchange and team cohesion. Further research is needed to determine the sustainability and generalizability of the checklist intervention, to fully integrate the checklist routine into workflow patterns, and to measure its impact on patient safety.

Communication failures in the operating room: an observational classification of recurrent types and effects.

BACKGROUND: Ineffective team communication is frequently at the root of medical error. The objective of this study was to describe the characteristics of communication failures in the operating room (OR) and to classify their effects. This study was part of a larger project to develop a team checklist to improve communication in the OR. METHODS: Trained observers recorded 90 hours of observation during 48 surgical procedures. Ninety four team members participated from anesthesia (16 staff, 6 fellows, 3 residents), surgery (14 staff, 8 fellows, 13 residents, 3 clerks), and nursing (31 staff). Field notes recording procedurally relevant communication events were analysed using a framework which considered the content, audience, purpose, and occasion of a communication exchange. A communication failure was defined as an event that was flawed in one or more of these dimensions. RESULTS: 421 communication events were noted, of which 129 were categorized as communication failures. Failure types included "occasion" (45.7% of instances) where timing was poor; "content" (35.7%) where information was missing or inaccurate, "purpose" (24.0%) where issues were not resolved, and "audience" (20.9%) where key individuals were excluded. 36.4% of failures resulted in visible effects on system processes including inefficiency, team tension, resource waste, workaround, delay, patient inconvenience and procedural error. CONCLUSION: Communication failures in the OR exhibited a common set of problems. They occurred in approximately 30% of team exchanges and a third of these resulted in effects which jeopardized patient safety by increasing cognitive load, interrupting routine, and increasing tension in the OR.

Interpretation of three-dimensional structure from two-dimensional endovascular images: implications for educators in vascular surgery.

PURPOSE: Endovascular therapy has had a major effect on vascular surgery; surgeons perform tasks in three dimensions (3D) while viewing two-dimensional (2D) displays. This fundamental change in how surgeons perform operations has educational implications related to learning curves and patient safety. We studied the effects of experience, training, and visual-spatial ability on 3D perception of 2D angiographic images of abdominal aortic aneurysms (AAA). METHODS: A novel computer-based method was developed to produce 3D depth maps based on subjects' interpretations of 2D images. Seven experts (certified vascular surgeons) and 20 novices (medical or surgical trainees) were presented with a 2D AAA angiographic image. With software specifically designed for this study, a depth map representing each subject's 3D interpretation of the 2D angiogram was produced. The novices were then randomized into a control group and a treatment group, who received a 5-minute AAA anatomy educational session. All subjects repeated the exercise on a second AAA image. Finally, all novices were given tests of visual-spatial ability, including the Surface Development Test and the Mental Rotations Test. Comparisons between experts and novices were made with depth map comparison, a subject's perception of overall object contour. RESULTS: The depth maps were significantly different (depth map comparison, P <.001) between the expert and both novice groups for the first image. After the educational intervention, the control group and the treatment group exhibited significantly different depth maps (depth map comparison, P <.001), with treatment group depth maps more similar to those of the expert group. There were no significant correlations between the visual-spatial tests and the novice depth map comparison with the expert group. CONCLUSIONS: This is the first study to examine perception of endovascular images in an educational context. Perception of overall surface contour of 3D structures from 2D angiographic images is affected by experience and training. With application of methods of vision science to an important problem in surgery, this research represents a first step in understanding the nature of visual perceptual processes involved in execution of an increasingly common clinical task. These results have implications for understanding and studying the endovascular learning curve. CLINICAL RELEVANCE: This research represents a unique collaboration in an effort to understand and solve one of the greatest problems facing surgical educators and surgeons. This research uses applied tools in vision science to understand the perceptual constraints involved in minimally invasive surgery. Specifically, we examined the mental three-dimensional maps experts use when viewing two-dimensional displays. Furthermore, we compared experts with novices in an effort to assist surgical trainees.

Toward reliable operative assessment: the reliability and feasibility of videotaped assessment of laparoscopic technical skills.

BACKGROUND: Decision making on the competency of surgical trainees to perform laparoscopic procedures has been hampered by the lack of reliable methods to evaluate operative performance. The goal of this study was to develop a feasible and reliable method of evaluation. METHODS: Twenty-nine senior surgical residents were videotaped performing a low anterior resection and a Nissen fundoplication in a pig. Ten blinded laparoscopists rated the videos independently on two scales. Rating time was minimized by allowing raters to fast-forward through the tapes at their discretion. Interrater reliability and the time required to rate a procedure were assessed. RESULTS: Rating time per procedure was a median of 15 min (range, 6-40). The mean interrater reliability for the two scales was 0.74. CONCLUSIONS: The use of videotapes of operations enabled multiple raters to assess a performance reliably and shortened assessment times by 80%. This assessment technique shows potential as a means of evaluating the performance of advanced laparoscopic procedures by surgical trainees.

Forming professional identities on the health care team: discursive constructions of the 'other' in the operating room.

BACKGROUND: Inter-professional health care teams represent the nucleus of both patient care and the clinical education of novices. Both activities depend upon the'talk' that team members use to interact with one another. This study explored team members' interpretations of tense team communications in the operating room (OR). METHODS: The study was conducted using 52 team members divided into 14 focus groups. Team members comprised 13 surgeons, 19 nurses, nine anaesthetists and 11 trainees. Both uni-disciplinary (n = 11) and multi-disciplinary (n = 3) formats were employed. All groups discussed three communication scenarios, derived from prior ethnographic research. Discussions were audio-recorded and transcribed. Using a grounded theory approach, three researchers individually analysed sample transcripts, after which group discussions were held to resolve discrepancies and confirm a coding structure. Using the confirmed code, the complete data set was coded using the 'NVivo' qualitative data analysis software program. RESULTS: There were substantial differences in surgeons', nurses', anaesthetists', and trainees' interpretations of the communication scenarios. Interpretations were accompanied by subjects' depictions of disciplinary roles on the team. Subjects' constructions of other professions' roles, values and motivations were often dissonant with those professions' constructions of themselves. CONCLUSIONS: Team members, particularly novices, tend to simplify and distort others' roles and motivations as they interpret tense communication. We suggest that such simplifications may be rhetorical, reflecting professional rivalries on the OR team. In addition, we theorise that novices' echoing of role simplification has implications for their professional identity formation.

Teaching invasive perinatal procedures: assessment of a high fidelity simulator-based curriculum.

OBJECTIVE: Learning curves pose a difficult problem in the teaching of technical skills: how do you teach procedural skills without compromising patients' health? A simulator-based curriculum has been designed to minimize the risks to patients undergoing amniocentesis by shifting the learning curve away from patients and into the laboratory. This study evaluated the effectiveness of a high-fidelity simulator-based curriculum in improving the performance of amniocentesis by obstetric trainees. DESIGN: Thirty trainees received a course on the practice of amniocentesis. The curriculum consisted of a lecture, a syllabus, and a hands-on training session with the simulator. Pre- and post-training performance were evaluated with two rating scales. Training and performance evaluation were completed using the same simulator. The effectiveness of the simulator-based workshop and the effect of year of training were assessed using a two-way analysis of variance. RESULTS: Performance scores improved from a mean score of 55% to 94% using checklist scoring and from 57% to 88% using global ratings. The two-way analysis of variance revealed a significant effect of training (F1,60 = 43.57; P < 0.001) accounting for 45% of the variance in scores, and a significant effect of experience level (F2,60 = 9.16; P < 0.001) accounting for 25% of the variance in scores. CONCLUSIONS: A comprehensive curriculum based on a high-fidelity simulator was effective at improving skills demonstrated on the simulator. The challenge remains to establish that skills acquired on a simulator are transferable to the clinical setting.

Structured assessment of minor surgical skills (SAMSS) for family medicine residents.

PURPOSE: To develop a valid and reliable examination to assess the technical proficiency of family medicine residents' performance of minor surgical office procedures. METHOD: A multi-station OSCE-style examination using bench-model simulations of minor surgical procedures was developed. Participants were a randomly selected group of 33 family medicine residents (PGY-1 = 16, PGY-2 = 17) and 14 senior surgical residents who functioned as a validation group. Examiners were qualified surgeons and family physicians who used both checklists and global rating scales to score the participants' performances. RESULTS: When family medicine residents were evaluated by family physicians, interstation reliabilities were .29 for checklists and .42 for global ratings. When family medicine residents were evaluated by surgeons, the reliabilities were .53 for checklists and .75 for global ratings. Interrater reliability, measured as a correlation for total examination scores, was .97. Mean scores on the examination were 60%, 64%, and 87% for PGY-1 family medicine, PGY-2 family medicine, and surgery residents, respectively. The difference in scores between family medicine and surgery residents was significant (p < .001), providing evidence of construct validity. CONCLUSION: A new examination developed for assessing family medicine residents' skills with minor surgical office procedures is reliable and has evidence for construct validity. The examination has low reliability when family physicians serve as examiners, but moderate reliability when surgeons are the evaluators.

Exporting a technical skills evaluation technology to other sites.

BACKGROUND: The Objective Structured Assessment of Technical Skill (OSATS) is a multistation performance-based examination that assesses the technical skills of surgery residents. This study explores the implementation issues involved in remote administration of the OSATS focusing on feasibility and the psychometric properties of the examination. METHODS: An eight-station OSATS was administered to surgical residents in Los Angeles and Chicago. The University of Toronto and the local institutions shared responsibility for organization and administration of the examination. RESULTS: There was good reliability for both the checklist (alpha = 0.68 for LA, 0.73 for Chicago) and global rating forms (alpha = 0.82 for both sites). Both iterations also showed evidence of construct validity, with a significant effect of training year for the checklist and global rating forms at both sites (analysis of variance: F = 8.66 to 19.93, P <0.01). Despite some challenges, the model of central organization and peripheral delivery was effective for the administration of the examinations. CONCLUSIONS: Two iterations of the OSATS at remote sites demonstrated psychometric properties that are highly consistent with previously reported data suggesting that the examination is portable. Both faculty and residents indicated satisfaction with the examination experience. A model of central administration with peripheral delivery was feasible and effective.

Using operative outcome to assess technical skill.

BACKGROUND: This study examined whether an operative product and time to completion could serve as measures of technical skill. METHODS: Nine final-year (PGY5) and 11 penultimate-year (PGY4) general surgery residents participated in a 6-station bench model examination. Time to completion was recorded. Twelve faculty surgeons (2 per station) evaluated the quality of the final product using a 5-point scale. RESULTS: The mean interrater reliability was 0. 59 for product quality. Interstation reliability was 0.59 for analysis of the final product and 0.72 for time to completion. There was 63% and 78% agreement between attendings' ratings and product quality and time scores respectively. PGY5s' mean product quality score was 4.14 +/- 0.26, compared with 3.82 +/- 0.33 for PGY4s (P < 0.05). PGY5s' mean time was 110 +/- 19 minutes compared with PGY4s' 132 +/- 15 (P < 0.05). CONCLUSIONS: Analysis of the operative end product and time to completion offer efficient alternatives to on-line examiner scoring for bench model examinations of technical competence.

Internal sphincterotomy is superior to topical nitroglycerin in the treatment of chronic anal fissure: results of a randomized, controlled trial by the Canadian Colorectal Surgical Trials Group.

PURPOSE: This was a multicenter, randomized, controlled trial to compare the effectiveness of topical nitroglycerin with internal sphincterotomy in the treatment of chronic anal fissure. METHODS: Patients with symptomatic chronic anal fissures were randomly assigned to 0.25 percent nitroglycerin tid or internal sphincterotomy. Both groups received stool softeners and fiber supplements and were assessed at six weeks and six months. RESULTS: Ninety patients were accrued, but 8 were excluded from the analysis because they refused internal sphincterotomy after randomization (6), the fissure healed before surgery (1), or a fissure was not observed at surgery (1). There were 38 patients in the internal sphincterotomy group (22 males; mean age, 40.3 years) and 44 patients in the nitroglycerin group (15 males; mean age, 38.7 years). At six weeks 34 patients (89.5 percent) in the internal sphincterotomy group compared with 13 patients (29.5 percent) in the nitroglycerin group had complete healing of the fissure (P = 5x10(-8)). Five of the 13 patients in the nitroglycerin group relapsed, whereas none in the internal sphincterotomy group did. At six months fissures in 35 (92.1 percent) patients in the internal sphincterotomy group compared with 12 (27.2 percent) patients in the nitroglycerin group had healed (P = 3x10(-9)). One (2.6 percent) patient in the internal sphincterotomy group required further surgery for a superficial fistula compared with 20 (45.4 percent) patients in the nitroglycerin group who required an internal sphincterotomy (P = 9x10(-6)). Eleven (28.9 percent) patients in the internal sphincterotomy group developed side effects compared with 37 (84 percent) patients in the nitroglycerin group (P<0.0001). Nine (20.5 percent) patients discontinued the nitroglycerin because of headaches (8) or a severe syncopal attack (1). CONCLUSIONS: Internal sphincterotomy is superior to topical nitroglycerin 0.25 percent in the treatment of chronic anal fissure, with a high rate of healing, few side effects, and low risk of early incontinence. Thus, internal sphincterotomy remains the treatment of choice for chronic anal fissure.

A comprehensive examination for senior surgical residents.

BACKGROUND: Two complimentary examinations designed to comprehensively assess competence for surgical practice have been developed. The Objective Structured Assessment of Technical Skill (OSATS) evaluates a resident's operative skill, and the Patient Assessment and Management Examination (PAME) evaluates clinical management skills. METHODS: Twenty-four postgraduate year (PGY)-4 and PGY-5 general surgery residents from four training programs were examined. Each examination had eight stations, with a total of 6 hours of testing time. RESULTS: Interstation reliability for the OSATS was 0.64, for the PAME was 0.71, and for the total test was 0. 74. Examination scores discriminated between PGY-4 and PGY-5 residents for the OSATS (t = 4.39, P <.01), the PAME (t = 1.86, P <. 05), and the total examination (t = 3.90, P <.01). Year of training accounted for 40%, of the variance of scores. CONCLUSIONS: This comprehensive examination is a reliable and valid method of assessing critical skills in senior surgical residents and may be useful for the formal assessment of readiness for practice.

Comparison of computer-assisted instruction and seminar instruction to acquire psychomotor and cognitive knowledge of epistaxis management.

Epistaxis is a common problem faced by primary care physicians. Typically, first-hand experience with this problem is not obtained in medical school. A computer learning module was developed to address practical management of epistaxis. It was evaluated by use of third-year clerks and compared with an interactive seminar based on the same material. Fifty-eight students doing their required otolaryngology rotations were randomly separated into 3 groups: (1) doing a preinstruction test, (2) using the computer module, and (3) participating in a small-group seminar. All participants were tested with a short written test and a practical test that involved performing anterior nasal pack placement in a model patient. Percentage scores for the 2 groups were compared by use of t tests, and there was no significant difference between the written, practical, or combined scores at a level where P = 0.05. This study shows that basic patient management and a simple procedure can be taught as effectively with a computer module as with a small-group interactive seminar.