Effects of implementation of an urgent surgical care service on subspecialty general surgery training.

BACKGROUND: In July 2007, a large Canadian teaching hospital realigned its general surgery services into elective general surgery subspecialty-based services (SUBS) and a new urgent surgical care (USC) service (also know in the literature as an acute care surgery service). The residents on SUBS had their number of on-call days reduced to enable them to focus on activities related to SUBS. Our aim was to examine the effect of the creation of the USC service on the educational experiences of SUBS residents. METHODS: We enrolled residents who were on SUBS for the 6 months before and after the introduction of the USC service. We collected data by use of a survey, WEBeVAL and recorded attendance at academic half days. Our 2 primary outcomes were residents' attendance at ambulatory clinics and compliance with the reduction in the number of on-call days. Our secondary outcomes included residents' time for independent study, attendance at academic half days, operative experience, attendance at multidisciplinary rounds and overall satisfaction with SUBS. RESULTS: Residents on SUBS had a decrease in the mean number of on-call days per resident per month from 6.28 to 1.84 (p = 0.006), an increase in mean attendance at academic half days from 65% to 87% (p = 0.028), at multidisciplinary rounds (p = 0.002) and at ambulatory clinics and an increase in independent reading time (p = 0.015), and they reported an improvement in their work environment. There was no change in the amount of time residents spent in the operating room or in their overall satisfaction with SUBS. CONCLUSION: Residents' education in the SUBS structure was positively affected by the creation of a USC service. Compliance with the readjustment of on-call duties was high and was identified as the single most significant factor in enabling residents to take full advantage of the unique educational opportunities available only while on SUBS.

Reliability and acceptance of the mini-clinical evaluation exercise as a performance assessment of practicing physicians.

BACKGROUND: The purpose of this study was to evaluate the reliability and acceptance of the mini-Clinical Evaluation Exercise (mini-CEX) as an assessment of practicing primary care physicians. METHOD: Six raters were recruited to conduct the assessments. After a training session, their ability to discriminate between levels of performance was evaluated using videotaped clinical scenarios. Fifteen physicians were assessed in an office setting by the raters who scored multiple clinical encounters using a validated mini-CEX form for each encounter. Participants were given a postassessment survey regarding the process. RESULTS: Raters distinguished between performance levels on the videotaped scenarios (P < .001). A total of 188 physician-patient interactions were assessed. The generalizability coefficient for 10 encounters was 0.92. In the postassessment survey, the raters (94%) and physicians assessed (75%) both felt that the mini-CEX is an acceptable assessment. CONCLUSIONS: The mini-CEX seems to be a reliable and acceptable instrument for the assessment of practicing physicians.

Objective assessment of surgical decision making in trauma after a laboratory-based course: durability of cognitive skills.

BACKGROUND: Because surgical trainees have less exposure to surgical trauma, there is a greater potential of having gaps in decision-making skills. We previously validated a novel assessment tool for decision making in surgical trauma and have documented improvement in resident decision-making skills after a hands-on course. However, brief intensive courses have been criticized for not imparting long-term changes in practice. The purpose of this study was to assess the durability of cognitive skills learned after a 2-day course. METHODS: Twenty-two residents participated in a 2-day interactive didactic lecture series as well as an animal laboratory focused on practical strategies in dealing with surgical trauma. All participants underwent precourse and immediate postcourse assessment of surgical decision making through a validated short-answer examination. Six months after the course, 12 of these 22 residents completed a third similar examination-the retention test. RESULTS: The retention test showed good reliability (Cronbach's alpha, .81) and construct validity as evidenced by a positive correlation between test scores and postgraduate year level (r = .9, P < .001). There was no significant difference between retention test scores and posttest scores. However, both were significantly higher than pretest scores (P < .05). This did not change after adjusting for differing degrees of difficulty between the examinations. CONCLUSIONS: In the context of residency trauma education, there is a measurable positive impact of an intensive, hands-on course on surgical decision making. This impact is durable and cognitive skills persist after the immediate postcourse period. These data support the continued supplementation of traditional residency experiential learning with appropriate laboratory-based skills training.

Teaching and learning in the operating room is a two-way street: resident perceptions.

BACKGROUND: The transformation of a trainee into a surgeon is influenced strongly by communication patterns in the operating room (OR). In the current era of limited educational opportunities, elucidation of teaching and learning strategies in this environment is critical. The aim of this study was to further understand the elements of an effective communicative instructional interaction (CII) as perceived by surgical residents. METHODS: Qualitative research methodology was used to explore University of British Columbia surgery residents' perceptions of what constitutes an effective CII in the OR. Purposeful sampling was used to select participants from various years of training. Eighteen residents participated in semistructured interviews to facilitate reflection of their OR experiences. Interviews were transcribed, analyzed, and fed back to residents to confirm their accuracy. Independent coding and analysis led to the development of key emergent themes. RESULTS: Themes represented the interplay of ideals expressed by the residents. The primary emergent theme was that both teacher and learner play a major role in the creation of an effective CII. The ideal teacher had an instructional plan, facilitated surgical independence, and showed support and empathy for the surgical resident. The ideal resident was receptive, prepared, and acknowledged limitations. The contextual constraints of the OR played a central role in learning, and residents identified ways to maintain educational value despite primarily nonmodifiable contextual elements (ie, time constraints). CONCLUSIONS: In a unique environment such as the OR, both teacher and learner may benefit by an enhanced understanding of the elements of an effective CII.

A universal global rating scale for the evaluation of technical skills in the operating room.

BACKGROUND: The ideal assessment of technical skills should be defensible and practical. The purpose of this study was to evaluate the utility of a Global Rating Scale (GRS) Assessment tool of resident operating room performance. METHODS: Residents were assessed in the operating room on multiple occasions during a 6-month study period using a 9-item GRS. Data were analyzed to assess scale reliability and sensitivity to year of training. Feasibility was evaluated with a post-study questionnaire. RESULTS: Seven residents had a total of 32 procedures assessed. One-way analysis of variance (ANOVA) showed that scores increased with year of training (P = .009). Reliability was excellent. (Cronbach's alpha .91). The post-study survey identified feedback and faculty interaction as strengths of this tool, but time constraint was a barrier. CONCLUSIONS: The GRS tool is a valid and reliable method that has the potential to be a practical, useful assessment tool of resident operating room performance.

Vascular surgery training, exposure, and knowledge during general surgery residency: implications for the future.

BACKGROUND: Vascular surgery (VS) has been removed from the Canadian general surgery (GS) objectives and has become a primary certificate specialty in the United States, leaving its status in GS uncertain. The purpose of this study was to determine GS residents' perceived competence in VS and to assess their knowledge of VS. METHODS: GS residents at a university-based program answered (1) a survey of attitudes and experience in VS and (2) a short-answer examination to assess fundamental vascular knowledge. RESULTS: Twenty-nine of 35 residents participated (83%). Residents reported being inadequately trained in 10 of 13 procedures surveyed despite 6 of these being reported as essential. Although 26 of 29 residents reported an intention to perform vascular procedures, none planned on pursuing a fellowship. The mean examination score was 47%. CONCLUSIONS: Despite mandatory VS rotations, GS residents feel inadequately trained in VS and have marginal knowledge. Current trainees may lack the skills and abilities to deal with vascular emergencies.

Visual-spatial ability and fMRI cortical activation in surgery residents.

BACKGROUND: We previously reported that a particular type of visual-spatial ability, mental rotation of visual forms, correlates with surgical performance in residents. In the current study, we used functional magnetic resonance imaging (fMRI) to identify patterns of cortical activation associated with mental rotation ability in those same residents. METHODS: Seventeen surgery residents underwent fMRI scan while performing a mental rotations test (MRT) and a perceptual matching task as a control (CON) for nonimagery components, such as visual attention. A contrast analysis (MRT greater than CON) revealed cortical regions engaged during mental rotation by all participants, and parametric statistical analysis identified regions having the strongest association with MRT performance. RESULTS: Significant bilateral (left greater than right) activation was seen in all participants for rotation-versus-perceptual CON contrast. Better MRT performance was associated with greater activation in several cortical regions related to visual imagery and motion processing. COMMENTS: Surgery residents represent a unique population in which to study individual differences in visual-spatial abilities and associated neural substrates because they may relate to technical skills. These findings suggest that variation in performance on spatially complex tasks involving imagery may reflect different spatial problem-solving strategies in surgery students.

Laboratory-based vascular anastomosis training: a randomized controlled trial evaluating the effects of bench model fidelity and level of training on skill acquisition.

BACKGROUND: Although there is growing evidence that practice on bench model simulators can improve the acquisition of technical skill in surgery, the degree to which these models have to approximate real-world conditions (model fidelity) to optimize learning is unclear. Previous research suggests that low-fidelity models may be adequate for novice learners. The purpose of this study was to assess the effect of model fidelity and surgical expertise on the acquisition of vascular anastomosis skill. METHODS: Twenty-seven surgical residents participated in this institutional review board-approved study. Junior residents (postgraduate year 1 and 2) and senior residents (postgraduate year 4 or higher) were randomized into two groups: low-fidelity (n = 13) and high-fidelity (n = 14) model training. Both groups were given a 3-hour hands-on training session: the low-fidelity group used plastic models, and the high-fidelity group used human cadaver arms (brachial arteries) to practice graft-to-arterial anastomosis. One week later, all subjects participated in an animal laboratory in which they performed a single vascular anastomosis on a live, anesthetized pig (femoral artery). A blinded vascular surgeon scored candidate performance in the animal laboratory by using previously validated end points, including a checklist and final product analysis score. RESULTS: Acquisition of skill was significantly affected by model fidelity and level of training as measured by both the checklist (P = .03) and final product analysis (P = .01; Kruskal-Wallis). Specifically, junior residents practicing on high-fidelity models scored better on the checklist (P = .05) and final product analysis (P = .04). Senior residents practicing on high-fidelity models scored better on final product analysis (P < .05). CONCLUSIONS: Training in the laboratory does improve skill when assessed in a realistic setting. Both expertise groups showed better skill transfer from the bench model to live animals when practicing on high-fidelity models. For vascular anastomosis, it is important to provide appropriate model fidelity for trainees of different abilities to optimize the effectiveness of bench model training.

Resident continuity of care experience: a casualty of ambulatory surgery and current patient admission practices.

BACKGROUND: The purpose of this study was to define and assess the impact of changes in health care delivery on the current continuity of care experience of surgical residents. METHODS: This 4-week, prospective cohort study included all patients who underwent a general surgical procedure at the University of British Columbia if a resident was present at the operation. The residents' perioperative involvement in each patient's care was recorded. RESULTS: Of the 592 eligible cases, 74.8% were elective same-day admissions, 5.4% elective previously admitted patients, and 19.8% emergencies. The overall rate of assessment was 27% preoperatively, 84% postoperatively on the ward, and <1% in oupatient clinic postdischarge. Elective cases were associated with significantly lower rates of preoperative assessment compared with emergency cases (15% versus 74%, P < .001). CONCLUSIONS: Changes in health care delivery have outpaced changes in the structure of surgical education, resulting in suboptimal continuity of care experiences for trainees. Residency programs must adapt their curricula to include adequate ambulatory experience.

Self-assessment during a 2-day laparoscopic colectomy course: can surgeons judge how well they are learning new skills?

BACKGROUND: The objectives of this study were to (1) establish the utility of an assessment tool for participants in a laparoscopic colectomy course and (2) to determine the accuracy of technical skill self-assessment in this group. METHODS: Twenty-two surgeons enrolled in a 2-day course participated. During the animal laboratory, each participant's operative performance was videotaped. Participants completed a global rating scale (GRS) instrument to self-assess their performances. By using the same GRS, 2 trained raters independently assessed each performance by videotape review. RESULTS: For the trained raters, the GRS showed excellent interrater reliability (r = .76, P < .001). There was no correlation between trained rater scores and self-assessment scores. Furthermore, the trained rater scores (mean, 2.62 and 2.99) were significantly lower than the self-assessment scores (4.05, P < .001). CONCLUSIONS: Surgeons consistently overestimated their performance during a laparoscopic colectomy course as measured by reliable GRS. This finding highlights the issue of credentialing and the importance of preceptorship for surgeons completing such courses.

Continuity of care experience of residents in an academic vascular department: are trainees learning complete surgical care?

BACKGROUND: It is widely accepted that exemplary surgical care involves a surgeon's involvement in the preoperative, perioperative, and postoperative periods. In an era of ever-expanding therapeutic modalities available to the vascular surgeon, it is important that trainees gain experience in preoperative decision-making and how this affects a patient's operative and postoperative course. The purpose of this study was to define the current experience of residents on a vascular surgery service regarding the continuity of care they are able to provide for patients and the factors affecting this experience. METHODS: This prospective cohort study was approved by the Institutional Review Board and conducted at the University of British Columbia during January 2005. All patients who underwent a vascular procedure at either of the two teaching hospitals were included. In addition to type of case (emergent, outpatient, inpatient), resident demographic data and involvement in each patient's care (preoperative assessment, postoperative daily assessment, and follow-up clinic assessment) were recorded. Categoric data were analyzed with the chi2 test. RESULTS: The study included 159 cases, of which 65% were elective same-day admission patients, 20% were elective previously admitted patients; and 15% were emergent. The overall rate of preoperative assessment was 67%, involvement in the decision to operate, 17%; postoperative assessment on the ward, 79%; and patient follow-up in clinic, 3%. The rate of complete in-hospital continuity of care (assessing patient pre-op and post-op) was 57%. Emergent cases were associated with a significantly higher rate of preoperative assessment (92% vs 63%, P < .05). For elective cases admitted before the day of surgery compared with same-day admission patients, the rates of preoperative assessment (78% vs 58%, P < .05) and involvement in the decision to operate (16% vs 4%, P < .05) were significantly higher. CONCLUSIONS: The continuity-of-care experiences of vascular trainees are suboptimal. This is especially true for postoperative clinic assessment. Same-day admission surgery accounted for most of the cases and was associated with the poorest continuity of care. To provide complete surgical training in an era of changing therapeutic modalities and same-day admission surgery, vascular programs must be creative in structuring training to include adequate ambulatory experience.

Using a comprehensive examination to assess multiple competencies in surgical residents: does the oral examination still have a role?

BACKGROUND: While specialty-level evaluations evolve from traditional examinations to objective structured clinical examination-like assessments, a broader range of competencies are tested; consequently, examiners are forced to integrate results when making a determination of competency. The aim of this study was to describe how experts weigh relative performances on specific components of a comprehensive examination to make decisions of overall competency. STUDY DESIGN: The Patient Assessment and Management Examination is a standardized patient-based assessment of general surgery in which each 25-minute station encompasses four components: history and physical examination, investigation interpretation, diagnosis and treatment discussion with the patient, and a structured oral examination (SOE). A six-station Patient Assessment and Management Examination was administered to 21 senior surgery residents. Surgeons marked each station with global rating scales and, in addition, provided an end-of-station overall global assessment of performance. A "gold-standard" examination pass-or-fail decision was determined through videotape review of each candidate's performance across six stations by two blinded surgeons. Multiple linear regression analysis was used to determine which components were associated with the end-of-station overall global assessments. Multivariable logistic regression was used to determine which components were associated with the final "gold-standard" pass-or-fail assessment. RESULTS: The only component notably (p < 0.005) associated with end-of-station global assessment for all six stations was the SOE. Mean SOE score was the only notable independent variable associated with the gold-standard pass-or-fail decision (R(2) = 0.63, p < 0.001). CONCLUSIONS: Performance on the SOE section of a multicompetency examination is markedly associated with the final determination of competency. These results have implications for the design and implementation of comprehensive specialty-level assessments.