Validation of a structured training and assessment curriculum for technical skill acquisition in minimally invasive surgery: a randomized controlled trial.

OBJECTIVE: : To develop and validate an ex vivo comprehensive curriculum for a basic laparoscopic procedure. BACKGROUND: : Although simulators have been well validated as tools to teach technical skills, their integration into comprehensive curricula is lacking. Moreover, neither the effect of ex vivo training on learning curves in the operating room (OR), nor the effect on nontechnical proficiency has been investigated. METHODS: : This randomized single-blinded prospective trial allocated 20 surgical trainees to a structured training and assessment curriculum (STAC) group or conventional residency training. The STAC consisted of case-based learning, proficiency-based virtual reality training, laparoscopic box training, and OR participation. After completion of the intervention, all participants performed 5 sequential laparoscopic cholecystectomies in the OR. The primary outcome measure was the difference in technical performance between the 2 groups during the first laparoscopic cholecystectomy. Secondary outcome measures included differences with respect to learning curves in the OR, technical proficiency of each sequential laparoscopic cholecystectomy, and nontechnical skills. RESULTS: : Residents in the STAC group outperformed residents in the conventional group in the first (P = 0.004), second (P = 0.036), third (P = 0.021), and fourth (P = 0.023) laparoscopic cholecystectomies. The conventional group demonstrated a significant learning curve in the OR (P = 0.015) in contrast to the STAC group (P = 0.032). Residents in the STAC group also had significantly higher nontechnical skills (P = 0.027). CONCLUSIONS: : Participating in the STAC shifted the learning curve for a basic laparoscopic procedure from the operating room into the simulation laboratory. STAC-trained residents had superior technical proficiency in the OR and nontechnical skills compared with conventionally trained residents. (The study registration ID is NCT01560494.).

A novel approach to assessing technical competence of colorectal surgery residents: the development and evaluation of the Colorectal Objective Structured Assessment of Technical Skill (COSATS).

OBJECTIVE: To develop and evaluate an objective method of technical skills assessment for graduating subspecialists in colorectal (CR) surgery-the Colorectal Objective Structured Assessment of Technical Skill (COSATS). BACKGROUND: It may be reasonable for the public to assume that surgeons certified as competent have had their technical skills assessed. However, technical skill, despite being the hallmark of a surgeon, is not directly assessed at the time of certification by surgical boards. METHODS: A procedure-based, multistation technical skills examination was developed to reflect a sample of the range of skills necessary for CR surgical practice. These consisted of bench, virtual reality, and cadaveric models. Reliability and construct validity were evaluated by comparing 10 graduating CR residents with 10 graduating general surgery (GS) residents from across North America. Expert CR surgeons, blinded to level of training, evaluated performance using a task-specific checklist and a global rating scale. The mean global rating score was used as the overall examination score and a passing score was set at "borderline competent for CR practice." RESULTS: The global rating scale demonstrated acceptable interstation reliability (0.69) for a homogeneous group of examinees. Both the overall checklist and global rating scores effectively discriminated between CR and GS residents (P < 0.01), with 27% of the variance attributed to level of training. Nine CR residents but only 3 GS residents were deemed competent. CONCLUSIONS: The Colorectal Objective Structured Assessment of Technical Skill effectively discriminated between CR and GS residents. With further validation, the Colorectal Objective Structured Assessment of Technical Skill could be incorporated into the colorectal board examination where it would be the first attempt of a surgical specialty to formally assess technical skill at the time of certification.

Competency-based education: a new model for teaching orthopaedics.

The current methods used to train residents to become orthopaedic surgeons are based on tradition, not evidence-based models. Educators have only a limited ability to assess trainees for competency using validated tests in various domains. The reduction in resident work hours limits the time available for clinical training, which has resulted in some calls for lengthening the training process. Another approach to address limited training hours is to focus training in a program that allows residents to graduate from a rotation based on demonstrated competency rather than on time on a service. A pilot orthopaedic residency curriculum, which uses a competency-based framework of resident training and maximizes the use of available training hours, has been designed and is being implemented.

A comparison of 2 ex vivo training curricula for advanced laparoscopic skills: a randomized controlled trial.

OBJECTIVE: To compare the effectiveness and cost of 2 ex vivo training curricula for laparoscopic suturing. BACKGROUND: Although simulators have been developed to teach laparoscopic suturing, a barrier to their wide implementation in training programs is a lack of knowledge regarding their relative training benefit and their associated cost. METHOD: This prospective single-blinded randomized trial allocated 24 surgical residents to train to proficiency using either a virtual reality (VR) simulator or box trainer. All residents then placed intracorporeal laparoscopic stitches during a Nissen fundoplication on a patient. The operating room (OR) cases were video-recorded and technical proficiency was assessed using 2 validated tools. OR performance of both groups was compared to that of conventionally trained residents and to fellowship-trained surgeons. A cost analysis of box training, VR training, and conventional residency training across Canadian surgical programs was performed. RESULTS: After ex vivo training, no significant differences in laparoscopic suturing in the OR were found between the 2 groups with respect to time (P = 0.74)-global rating score (P = 0.65) or checklist score (P = 0.97). It took conventionally trained residents 6 practice attempts in the OR to achieve the technical proficiency of the ex vivo trained groups (P = 0.83). VR training was more efficient than box training (transfer effectiveness ratio of 2.31 vs 1.13). The annual cost of training 5 residents on the FLS trainer box was $11,975.00, on the VR simulator was $77,500.00, and conventional residency training was $17,380.00. Over 5 years, box training was the most cost-effective option for all programs, and VR training was more cost-effective for programs with more 10 residents. CONCLUSIONS: Training on either a VR simulator or on a box trainer significantly decreased the learning curve necessary to learn laparoscopic suturing. VR training, however, is the more efficient training modality, whereas box training the more cost-effective option.

A surgical safety checklist to reduce morbidity and mortality in a global population.

BACKGROUND: Surgery has become an integral part of global health care, with an estimated 234 million operations performed yearly. Surgical complications are common and often preventable. We hypothesized that a program to implement a 19-item surgical safety checklist designed to improve team communication and consistency of care would reduce complications and deaths associated with surgery. METHODS: Between October 2007 and September 2008, eight hospitals in eight cities (Toronto, Canada; New Delhi, India; Amman, Jordan; Auckland, New Zealand; Manila, Philippines; Ifakara, Tanzania; London, England; and Seattle, WA) representing a variety of economic circumstances and diverse populations of patients participated in the World Health Organization's Safe Surgery Saves Lives program. We prospectively collected data on clinical processes and outcomes from 3733 consecutively enrolled patients 16 years of age or older who were undergoing noncardiac surgery. We subsequently collected data on 3955 consecutively enrolled patients after the introduction of the Surgical Safety Checklist. The primary end point was the rate of complications, including death, during hospitalization within the first 30 days after the operation. RESULTS: The rate of death was 1.5% before the checklist was introduced and declined to 0.8% afterward (P=0.003). Inpatient complications occurred in 11.0% of patients at baseline and in 7.0% after introduction of the checklist (P<0.001). CONCLUSIONS: Implementation of the checklist was associated with concomitant reductions in the rates of death and complications among patients at least 16 years of age who were undergoing noncardiac surgery in a diverse group of hospitals.

Differences in the perceived impact of sleep deprivation among surgical and non-surgical residents.

OBJECTIVE: Resident work hour restrictions have been mandated in the USA largely out of concern that sleep deprivation compromises doctor performance and patient care. However, individuals' ability to recognise the effects of sleep deprivation has not been studied in medical education. We examined the perceived impact of sleep deprivation among different groups of postgraduate medical trainees. METHODS: A survey addressing work hours, sleepiness and daily functioning was mailed to all residents in the internal medicine, surgery and psychiatry programmes at the University of Toronto who were working at 6 different teaching hospitals. The mailing included the Epworth Sleepiness Scale (ESS), measuring acute sleepiness, and a new Sleep Deprivation Impact (SDI) scale, consisting of 12 items designed to measure the perceived impact of sleep deprivation on an individual's own performance. RESULTS: Overall, 62.5% of surgery (95/152) and 59.5% of non-surgery residents (194/326) completed the survey. Surgery residents reported working longer hours per week (83.0 versus 62.5 hours; P < 0.01), and scored higher on the ESS (12.8 versus 9.2; P < 0.01) compared with other residents. Surgery residents scored significantly lower than others on the SDI scale (45.2 versus 51.5, P < 0.01), indicating less perceived impact of sleep deprivation on performance. CONCLUSIONS: These results are consistent with the presence of an underlying culture within surgery in which individuals may be less willing to accept a natural limitation of individual performance. Whether these findings represent an actual resilience to sleep deprivation among surgery residents or a misperception within this group remains to be determined.

The Surgical Skills and Technology Elective Program (SSTEP): A comprehensive simulation-based surgical skills initiative for preclerkship medical students.

BACKGROUND: The Surgical Skills and Technology Elective Program (SSTEP) is a voluntary preclerkship surgical bootcamp that uses simulation learning to build procedural knowledge and technical skills before clerkship. METHODS: Eighteen second year students (n = 18) participated in simulation workshops over the course of 7 days to learn clerkship-level procedural skills. A manual was supplied with the program outline. Assessment of the participants involved: 1) a written exam 2) a single videotaped Objective Structured Assessment of Technical Skill (OSATS) station 3) an exit survey to document changes in career choices. RESULTS: Compared to the mean written pre-test score students scored significantly higher on the written post-test (35.83 ± 6.56 vs. 52.11 ± 5.95 out of 73) (p = 0.01). Technical skill on the OSATS station demonstrated improved performance and confidence following the program (10.10 vs. 17.94 out of 25) (p = 0.05). Most participants (72%) re-considered their choices of surgical electives. CONCLUSIONS: A preclerkship surgical skills program not only stimulates interest in surgery but can also improve surgical knowledge and technical skills prior to clerkship.

Patient care is a collective responsibility: perceptions of professional responsibility in surgery.

BACKGROUND: Changes in training are likely to affect the professionalization process, but such complex social phenomena are poorly studied by quantitative research methodologies. In contrast, qualitative research designs are more effective in exploring complex social processes. The objective of this study was to use a qualitative methodology to explore how professional responsibilities are perceived by surgical trainees and faculty in the current academic environment. METHODS: Semi-structured individual interviews of 43 surgical residents and faculty (ranging from second year residents to senior faculty) were conducted at 2 academic institutions. The interviews consisted of open-ended questions, followed by discussion of 4 written, case-based scenarios on specific issues related to professional responsibilities. All interviews were audio-recorded and transcribed, and then analyzed for emergent themes by 3 researchers using a grounded theory approach. RESULTS: In discussing professional responsibilities, the motivations that shaped participants' responses reflected a balance between 4 major factors: (1) patient care, (2) education, (3) self, and (4) collegial relationships. Patient care was described as being at the center of professional responsibility, but it did not necessarily supersede other factors. Rather, patient care was described as a collective responsibility, operationalized through teamwork, communication, and trust. CONCLUSIONS: Traditional medical ethics have largely focused on professional responsibility from the standpoint of individual healthcare providers. Our findings suggest it is a much more complex construct characterized by competing responsibilities and an evolving perception of patient care as a collective responsibility. Explicit acknowledgment of this framework sets the stage for educational interventions to support residents' professional development and enhance cooperative behavior among participants.

Teaching suturing and knot-tying skills to medical students: a randomized controlled study comparing computer-based video instruction and (concurrent and summary) expert feedback.

BACKGROUND: We carried out a prospective, randomized, 4-arm study including control arm, blinding of examiners to determine effectiveness of computer-based video instruction (CBVI) and different types of expert feedback (concurrent and summary) on learning of a basic technical skill. METHODS: Using bench models, participants were pre-tested on a suturing and instrument knot-tying skill after viewing an instructional video. The students were subsequently assigned randomly to 4 practice conditions: no additional intervention (control), self study with CBVI, expert feedback during practice trials (concurrent feedback), and expert feedback after practice trials (summary feedback). All participants underwent 19 trials of practice, over 1 hour, in their assigned training condition. The effectiveness of training was assessed both at an immediate post-test and 1 month later at a retention test. Performance was evaluated using both expert-based (Global Rating Scores) and computer-based assessment (Hand Motion Analysis). Data were analyzed using repeated-measures ANOVA. RESULTS: There were no differences in GRS between groups at pre-test. The CBVI, concurrent feedback and summary feedback methods were equally effective initially for the instruction of this basic technical skill to naive medical students and displayed better performance than control (control, 12.71 [10.79 to 14.62]; CBVI, 16.39 [14.38 to 18.40]; concurrent, 16.97 [15.79 to 18.15]; summary, 16.09 [13.57 to 18.62]; P < .001 each). At retention. however, only CBVI and summary feedback groups retained superior suturing and knot-tying performance versus control (control, 8.13 [6.94 to 9.85]; CBVI, 11.92 [10.19 to 14.99] P = .037; concurrent, 9.80 [8.55 to 13.45] P = .635; summary, 111.19 [10.27 to 14.29] P = .037). Hand motion data displayed a similar pattern of results. There were no group differences in the rate of learning (P > .05). CONCLUSION: Our study showed that CBVI can be as effective as summary expert feedback in the instruction of basic technical skills to medical students. Thoughtfully incorporated into technical curricula, CBVI can make efficient use of faculty time and serve as a useful pedagogic adjunct for basic skills training. Additionally, our study provides evidence supporting an increased role of summary feedback to effectively train novices in technical skills.

Needs assessment of neurosurgery trainees: a survey study of two large training programs in the developing and developed worlds.

BACKGROUND: There are challenges facing surgical education in both the developing and the developed worlds. Few studies have examined trainee perceptions of their educational needs in a systematic way. We undertook a study to examine this issue, focusing on two large training programs, one in the developed world and one in the developing world. METHODS: Neurosurgical trainees at the University of Toronto, Toronto, Canada, and at Hasan Sadikin Hospital in Bandung, Indonesia, were surveyed with a comprehensive questionnaire assessing both the content and the methods of their training. The questionnaire had 37 quantitative questions requesting responses on a 7-point Likert scale and three open-ended questions to give more qualitative data. RESULTS: Sixty-four percent of all trainees responded. A number of interesting findings about the strengths and weaknesses of training emerged. For example, Bandung trainees felt they had excellent training in trauma but not in specialty areas, especially spine and vascular, with ample opportunity to operate as the primary surgeon. Toronto trainees felt that the volume and the variety of cases were excellent but they did not have enough ambulatory experience, and that they had suboptimal experience as the primary surgeon. Trainees in both centers agree that they will feel competent to practice neurosurgery upon completion of their training. CONCLUSION: This study defined different educational needs for neurosurgical trainees in two centers that reflect both their individual training environments and the local culture of medicine. As such, trainees' perceptions of these needs represent an important adjunct to program evaluation.

Student-led learning: a new teaching paradigm for surgical skills.

BACKGROUND: Competency-based education and simulation are being used more frequently in surgical skills curricula. We explored a novel student-led learning paradigm, which allows trainees to become more active participants in the learning process while maintaining expert guidance and supervision. METHODS: Twelve first-year orthopedic residents were randomized to either a student-led (SL) or a traditional instructor-led group during an intensive, month-long, laboratory-based technical skills training course. A rigorous qualitative-description approach was used for analysis. RESULTS: Four prominent themes emerged: instructional style, feedback, peer and instructor collaboration, and self-efficacy. Compared with the instructor-led group, there was more peer assistance, feedback, collaboration, and hands-on and active learning observed in the SL group. CONCLUSIONS: The flexible and socially rich nature of the SL learning environment may aid in development of both technical and nontechnical skills early in residency and ultimately privilege later clinical learning.

Visual-spatial ability correlates with efficiency of hand motion and successful surgical performance.

BACKGROUND: This study examines the influence of visual-spatial ability and manual dexterity on surgical performance across 3 levels of expertise. METHODS: Dental students, surgical residents, and staff surgeons completed standardized tests of manual dexterity and visual-spatial ability and were assessed objectively while performing the rigid fixation of an anterior mandible on bench model simulations. Outcome variables included expert assessment of technical performance and efficiency of hand motion during the procedure (recorded using electromagnetic sensors). RESULTS: Visual-spatial scores correlated significantly with surgical performance scores within the group of dental students (r=.40 to.73), but this was not the case for residents or staff surgeons. For all groups, manual dexterity did not correlate with hand motion parameters. There were no differences between groups in visual-spatial ability or manual dexterity, but highly significant differences were seen in surgical performance scores (P<.001), in that surgeons outperformed residents, who in turn outperformed students. CONCLUSIONS: Among novices, visual-spatial ability is associated with skilled performance on a spatially complex surgical procedure. However, advanced trainees and experts do not score any higher on carefully selected visual-spatial tests, suggesting that practice and surgical experience may supplant the influence of visual-spatial ability over time. Thus, the use of these tests for the selection of residents is not currently recommended; they may be of more use in identifying those novice trainees (ie, those with lower test scores) who might benefit most from brief supplementary instruction on specific technical tasks.

Evaluating the effectiveness of a 2-year curriculum in a surgical skills center.

BACKGROUND: This study was a formative evaluation of a 2-year Surgical Skills Center Curriculum (SSCC) using objective measures of surgical performance and self-reported process-oriented evaluations. METHODS: Fifty postgraduate third-year (PGY-3) residents participated in an Objective Structured Assessment of Technical Skills (OSATS) examination. Nineteen residents underwent the SSCC and 31 residents did not. During the SSCC, self-reported student and faculty evaluations were completed after each session. RESULTS: For the OSATS examination, scores were not significantly different between treatment and control groups, on either the checklist (66.4 +/- 6.1 versus 64.1 +/- 10.8) or global rating scale scores (66.9 +/- 6.9 versus 68.0 +/- 9.6). Further comparisons between groups on individual OSATS stations revealed no significant differences between groups. The majority of student and faculty evaluation remarks were highly positive. CONCLUSIONS: The OSATS results failed to support our hypothesis that training on a core procedure in a single session during a SSCC would have an appreciable and sustained effect after 2 years. Self-reported process-oriented evaluations support the utility of our SSCC.

Faculty development in assessment: the missing link in competency-based medical education.

As the medical education community celebrates the 100th anniversary of the seminal Flexner Report, medical education is once again experiencing significant pressure to transform. Multiple reports from many of medicine's specialties and external stakeholders highlight the inadequacies of current training models to prepare a physician workforce to meet the needs of an increasingly diverse and aging population. This transformation, driven by competency-based medical education (CBME) principles that emphasize the outcomes, will require more effective evaluation and feedback by faculty.Substantial evidence suggests, however, that current faculty are insufficiently prepared for this task across both the traditional competencies of medical knowledge, clinical skills, and professionalism and the newer competencies of evidence-based practice, quality improvement, interdisciplinary teamwork, and systems. The implication of these observations is that the medical education enterprise urgently needs an international initiative of faculty development around CBME and assessment. In this article, the authors outline the current challenges and provide suggestions on where faculty development efforts should be focused and how such an initiative might be accomplished. The public, patients, and trainees need the medical education enterprise to improve training and outcomes now.

Changes in safety attitude and relationship to decreased postoperative morbidity and mortality following implementation of a checklist-based surgical safety intervention.

OBJECTIVES: To assess the relationship between changes in clinician attitude and changes in postoperative outcomes following a checklist-based surgical safety intervention. DESIGN: Pre- and post intervention survey. SETTING: Eight hospitals participating in a trial of a WHO surgical safety checklist. PARTICIPANTS: Clinicians actively working in the designated study operating rooms at the eight hospitals. SURVEY INSTRUMENT: Modified operating-room version Safety Attitudes Questionnaire (SAQ). MAIN OUTCOME MEASURES: Change in mean safety attitude score and correlation between change in safety attitude score and change in postoperative outcomes, plus clinician opinion of checklist efficacy and usability. RESULTS: Clinicians in the preintervention phase (n=281) had a mean SAQ score of 3.91 (on a scale of 1 to 5, with 5 representing better safety attitude), while the postintervention group (n=257) had a mean of 4.01 (p=0.0127). The degree of improvement in mean SAQ score at each site correlated with a reduction in postoperative complication rate (r=0.7143, p=0.0381). The checklist was considered easy to use by 80.2% of respondents, while 19.8% felt that it took a long time to complete, and 78.6% felt that the programme prevented errors. Overall, 93.4% would want the checklist used if they were undergoing operation. CONCLUSIONS: Improvements in postoperative outcomes were associated with improved perception of teamwork and safety climate among respondents, suggesting that changes in these may be partially responsible for the effect of the checklist. Clinicians held the checklist in high regard and the overwhelming majority would want it used if they were undergoing surgery themselves.

Observation, reflection, and reinforcement: surgery faculty members' and residents' perceptions of how they learned professionalism.

PURPOSE: To explore perceptions of how professionalism is learned in the current academic environment. Professionalism is a core competency in surgery (as in all of medical practice), and its presence or absence affects all aspects of clinical education and practice, but the ways in which professional values and attitudes are best transmitted to developing generations of surgeons have not been well defined. METHOD: The authors conducted 34 semistructured interviews of individual surgery residents and faculty members at two academic institutions from 2004 to 2006. Interviews consisted of open-ended questions on how the participants learned professionalism and what they perceived as challenges to learning professionalism. Two researchers analyzed the interview transcripts for emergent themes by using a grounded-theory approach. RESULTS: Faculty members' and residents' perceptions of how they learned professionalism reflected four major themes: (1) personal values and upbringing, including premedical education experiences, (2) learning by example from professional role models, (3) the structure of the surgery residency, and (4) formal instruction on professionalism. Of these, role modeling was the dominant theme: Participants identified observation, reflection, and reinforcement as playing key roles in their learning from role models and in distinguishing the sometimes blurred boundary between positive and negative role models. CONCLUSIONS: The theoretical framework generated out of this study proposes a focus on specific activities to improve professional education, including an active approach to role modeling through the intentional and explicit demonstration of professional behavior during the course of everyday work; structured, reflective self-examination; and timely and meaningful evaluation and feedback for reinforcement.

Training tomorrow's surgeons: what are we looking for and how can we achieve it?

The aim of a surgical residency program is to produce competent professionals in a safe and pedagogically efficient environment. For many years, there has been an overemphasis on technical attributes as the fundamental competencies of a trained surgeon. With the advent of new frameworks for defining the outcomes of surgical training, such as CanMeds from the Royal College of Physicians and Surgeons of Canada and the six competencies outlined by the Accreditation Council for Graduate Medical Education in USA, there has been a broadening of the focus of surgical training. Although technical proficiency is definitely an important prerequisite for a successful outcome, other qualities such as intellectual abilities, personality and communication skills, and a commitment to practice are important elements in the profile of a competent surgeon. Recently, there is a growing appreciation for the heterogeneity in achievement of technical competence among our trainees, with some residents able to quickly master technical skill in contrast to others who may never achieve mastery in the technical domain. The questions of how to select, teach and grant privileges for independent practice requires an understanding of the components of surgical competence and implementation of evidence based tools for training and assessment of these competencies.