Transatlantic comparison of the competence of surgeons at the start of their professional career.

BACKGROUND: Although the objective in European Union and North American surgical residency programmes is similar-to train competent surgeons-residents' working hours are different. It was hypothesized that practice-ready surgeons with more working hours would perform significantly better than those being educated within shorter working week curricula. METHODS: At each test site, 21 practice-ready candidate surgeons were recruited. Twenty qualified Canadian and 19 qualified Dutch surgeons served as examiners. At both sites, three validated outcome instruments assessing multiple aspects of surgical competency were used. RESULTS: No significant differences were found in performance on the integrative and cognitive examination (Comprehensive Integrative Puzzle) or the technical skills test (Objective Structured Assessment of Technical Skill; OSATS). A significant difference in outcome was observed only on the Patient Assessment and Management Examination, which focuses on skills needed to manage patients with complex problems (P < 0.001). A significant interaction was observed between examiner and candidate origins for both task-specific OSATS checklist (P = 0.001) and OSATS global rating scale (P < 0.001) scores. CONCLUSION: Canadian residents, serving many more working hours, perform equivalently to Dutch residents when assessed on technical skills and cognitive knowledge, but outperformed Dutch residents in skills for patient management. Secondary analyses suggested that cultural differences influence the assessment process significantly.

Interpretation of three-dimensional structure from two-dimensional endovascular images: implications for educators in vascular surgery.

PURPOSE: Endovascular therapy has had a major effect on vascular surgery; surgeons perform tasks in three dimensions (3D) while viewing two-dimensional (2D) displays. This fundamental change in how surgeons perform operations has educational implications related to learning curves and patient safety. We studied the effects of experience, training, and visual-spatial ability on 3D perception of 2D angiographic images of abdominal aortic aneurysms (AAA). METHODS: A novel computer-based method was developed to produce 3D depth maps based on subjects' interpretations of 2D images. Seven experts (certified vascular surgeons) and 20 novices (medical or surgical trainees) were presented with a 2D AAA angiographic image. With software specifically designed for this study, a depth map representing each subject's 3D interpretation of the 2D angiogram was produced. The novices were then randomized into a control group and a treatment group, who received a 5-minute AAA anatomy educational session. All subjects repeated the exercise on a second AAA image. Finally, all novices were given tests of visual-spatial ability, including the Surface Development Test and the Mental Rotations Test. Comparisons between experts and novices were made with depth map comparison, a subject's perception of overall object contour. RESULTS: The depth maps were significantly different (depth map comparison, P <.001) between the expert and both novice groups for the first image. After the educational intervention, the control group and the treatment group exhibited significantly different depth maps (depth map comparison, P <.001), with treatment group depth maps more similar to those of the expert group. There were no significant correlations between the visual-spatial tests and the novice depth map comparison with the expert group. CONCLUSIONS: This is the first study to examine perception of endovascular images in an educational context. Perception of overall surface contour of 3D structures from 2D angiographic images is affected by experience and training. With application of methods of vision science to an important problem in surgery, this research represents a first step in understanding the nature of visual perceptual processes involved in execution of an increasingly common clinical task. These results have implications for understanding and studying the endovascular learning curve. CLINICAL RELEVANCE: This research represents a unique collaboration in an effort to understand and solve one of the greatest problems facing surgical educators and surgeons. This research uses applied tools in vision science to understand the perceptual constraints involved in minimally invasive surgery. Specifically, we examined the mental three-dimensional maps experts use when viewing two-dimensional displays. Furthermore, we compared experts with novices in an effort to assist surgical trainees.

A comprehensive examination for senior surgical residents.

BACKGROUND: Two complimentary examinations designed to comprehensively assess competence for surgical practice have been developed. The Objective Structured Assessment of Technical Skill (OSATS) evaluates a resident's operative skill, and the Patient Assessment and Management Examination (PAME) evaluates clinical management skills. METHODS: Twenty-four postgraduate year (PGY)-4 and PGY-5 general surgery residents from four training programs were examined. Each examination had eight stations, with a total of 6 hours of testing time. RESULTS: Interstation reliability for the OSATS was 0.64, for the PAME was 0.71, and for the total test was 0. 74. Examination scores discriminated between PGY-4 and PGY-5 residents for the OSATS (t = 4.39, P <.01), the PAME (t = 1.86, P <. 05), and the total examination (t = 3.90, P <.01). Year of training accounted for 40%, of the variance of scores. CONCLUSIONS: This comprehensive examination is a reliable and valid method of assessing critical skills in senior surgical residents and may be useful for the formal assessment of readiness for practice.

Comparison of computer-assisted instruction and seminar instruction to acquire psychomotor and cognitive knowledge of epistaxis management.

Epistaxis is a common problem faced by primary care physicians. Typically, first-hand experience with this problem is not obtained in medical school. A computer learning module was developed to address practical management of epistaxis. It was evaluated by use of third-year clerks and compared with an interactive seminar based on the same material. Fifty-eight students doing their required otolaryngology rotations were randomly separated into 3 groups: (1) doing a preinstruction test, (2) using the computer module, and (3) participating in a small-group seminar. All participants were tested with a short written test and a practical test that involved performing anterior nasal pack placement in a model patient. Percentage scores for the 2 groups were compared by use of t tests, and there was no significant difference between the written, practical, or combined scores at a level where P = 0.05. This study shows that basic patient management and a simple procedure can be taught as effectively with a computer module as with a small-group interactive seminar.

Needs assessment for an undergraduate otolaryngology curriculum.

A needs assessment was conducted to determine what family doctors need to know about otolaryngology. A survey was mailed to a group of community otolaryngologists and family doctors. They were asked to rate the importance of 46 otolaryngologic topics. A true needs assessment was carried out with a small group of family practice residents to examine which needs were perceived to be significantly different in importance between these 2 survey groups. A list of important topics was generated from the survey. There were many statistically significant differences between the survey groups, but only 2, vertigo and epistaxis, appeared to be potentially important. The family practice residents (n = 8) were asked to complete a short-answer test about epistaxis management and then to demonstrate placement of an anterior nasal pack. Despite having already faced this problem in 50% of cases, the residents had average scores of less than 30%, indicating a lack of knowledge in this area.

Assessment of technical skills transfer from the bench training model to the human model.

BACKGROUND: This study examines whether technical skills learned on a bench model are transferable to the human cadaver model. METHODS: Twenty-three first-year residents were randomly assigned to three groups receiving teaching on six procedures. For each procedure, one group received training on a cadaver model, one received training on a bench model, and one learned independently from a prepared text. Following training, all residents were assessed on their ability to perform the six procedures. RESULTS: Repeated measures analysis of variance revealed a significant effect of training modality for both checklist scores (F(2,44) = 3.49, P <0.05) and global scores (F(2,44) = 7.48, P <0.01). Post-hoc tests indicated that both bench and cadaver training were superior to text learning and that bench and cadaver training were equivalent. CONCLUSIONS: Training on a bench model transfers well to the human model, suggesting strong potential for transfer to the operating room.

Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination.

PURPOSE: To compare the psychometric properties of checklists, global rating scales preceded by a checklist, and global rating scales alone in assessing surgery residents' performances on an OSCE-like technical skills examination. METHOD: In 1996, 53 general surgery residents with one to six years of postgraduate training participated in a performance-based examination of technical skills consisting of eight 15-minute stations (bench-model simulations of operative procedures in general surgery). Two qualified surgeons marked at each station, one using a task-specific checklist (C) and a subsequent global rating scale (Gc), the other using a global rating scale only (G). RESULTS: Interstation reliabilities measured by Cronbach's alpha were .79 for C, .89 for Gc, and .85 for G. A series of multiple regressions predicting level of training from test scores revealed an R2 of .584 for C alone, which increased to .711 when Gc was entered after (p < .001), and increased to .704 when G was entered after C (p < .001). However, R2 for Gc alone was .711, and for G alone was .704, neither of which changed when C was entered into the prediction (p > .10). The R2 for Gc and G predicting level of training (.725) was not significantly greater than that of either Gc or G alone. A very similar pattern of results was seen when C, Gc, and G were used to predict independent evaluations of the operative outcomes. CONCLUSIONS: Global rating scales scored by experts showed higher inter-station reliability, better construct validity, and better concurrent validity than did checklists. Further, the presence of the checklists did not improve the reliability or validity of the global rating scale over that of the global rating scale alone. These results suggest that global rating scales administered by experts are a more appropriate summative measure when assessing candidates on performance-based examinations.

Description and evaluation of the Surgeons as Educators course.

BACKGROUND: Since 1993, the American College of Surgeons has sponsored an annual 6-day course entitled the Surgeons as Educators. The course was designed to provide academic surgeons with the knowledge and skills necessary to enhance the surgical education curriculum, teaching strategies, educational program administration, and performance evaluation. This article describes the development, implementation, and effect of the course on the classes graduating in 1993 and 1994. STUDY DESIGN: The effect of the course was studied by using a longitudinal survey approach. A survey was mailed to participants 3 to 6 months after they completed the course. Graduates were asked to describe any education-related actions taken attributable to attending the Surgeons as Educators course. The quality of course content and presentations were evaluated by using end-of-course evaluation forms and daily feedback forms and by an external reviewer. RESULTS: Within 6 months of returning from the course, more than one half of the graduates initiated actions related to curriculum development, teaching strategies, or educational administration. One third or more of the graduates modified their performance and program evaluation systems. Using a five-point scale, ratings of the course content ranged from 3.78 to 4.64 for "value of topic" and from 3.77 to 4.76 for "quality of presentation." Items evaluated by the graduates on the end-of-course evaluation forms ranged from 7.8 to 8.7 on a nine-point scale. CONCLUSIONS: The Surgeons as Educators course offered an opportunity for participants to interact among themselves and with course faculty about educational issues and to practice teaching skills. The course was highly rated for educational quality and value. The retreat environment and the length of the program helped attendees become immersed during this "protected time" to analyze strengths and weaknesses of their programs and devise achievable plans to improve their abilities as educators and the effectiveness of their programs.

The efficacy of a computer-assisted preoperative tutorial for clinical clerks.

OBJECTIVE: To assess the effectiveness of computer-assisted preoperative tutorials on human anatomy in improving the operating-room learning experience for clinical clerks. DESIGN: Crossover trials with immediate assessment and approximately 1-week delay between trials. SETTING: General surgery operating room of a university teaching hospital. SUBJECTS: Eight 4th-year clinical clerks on a single 8-week surgical rotation. MAIN OUTCOME MEASURES: The senior surgeon's assessment of the clerk's knowledge and understanding of the operation, according to a six-item questionnaire, and the clerk's own assessment of the experience in the operating room, according to an eight-item questionnaire. RESULTS: Surgeons rated the clerks' performances in the operating room as better when they had received the preoperative tutorial (mean [and standard deviation], 3.7 [0.4]) than when they had not received the preoperative tutorial (3.0 [0.3]). The difference (0.7 [0.6]) was statistically significant (t7 = 3.3, p < 0.01). Similarly, clerks rated their own experience more positively when they had received the tutorial (4.0 [0.2]) than when they had not (3.1 [0.3]), with the difference (0.9 [0.5]) being statistically significant (t7 = 4.9, p < 0.001). CONCLUSIONS: Short, preoperative, computer-assisted tutorials on human anatomy can have a positive impact on the clerk's level of knowledge and confidence in the operating room. Further research is warranted into the extent to which students spontaneously make use of these tutorials.

Who should rate candidates in an objective structured clinical examination?

PURPOSE: To determine who is the better rater of history taking in an objective structured clinical examination (OSCE): a physician or a standardized patient (SP). METHOD: During the 1991 pilot administration of an OSCE for the Medical Council of Canada's qualifying examination, five history-taking stations were videotaped. Candidates at these stations were scored by three raters: a physician (MD), an SP observer (SPO), and an SP rating from recall (SPR). To determine the validity of each rater's scores, these scores were compared with a "gold standard", which was the average of videotape ratings by three physicians, each scoring independently. Analysis included both correlations with the standard and a repeated-measures analysis of variance (ANOVA) comparing raters' mean scores on each station with mean scores of the gold standard. RESULTS: Ninety-one videotapes were scored by the "gold-standard" physicians. Correlations with the standard showed no clear preference for MD, SPO, or SPR raters. ANOVAs revealed significant differences from the standard on three stations for the SPR, two stations for the SPO, and one stations for the MD. CONCLUSIONS: An MD rater is less likely to differ from a standard established by a consensus of MD ratings than are SP raters rating from recall. If an MD cannot be used, an SP observer is preferable to an SP rating from recall.

Meta-analysis of highly selective vagotomy versus truncal vagotomy and pyloroplasty in the surgical treatment of uncomplicated duodenal ulcer.

OBJECTIVE: To compare critically by meta-analysis the postoperative outcomes after truncal vagotomy and pyloroplasty (TVP) and after highly selective vagotomy (HSV). DATA SOURCES: A search was conducted on MEDLINE with the keywords "peptic ulcer" and "vagotomy". Bibliographics of the articles retrieved and review articles pertaining to the subject were scanned further. STUDY SELECTION: Included were only published prospective, randomized controlled trials comparing TVP and HSV in the English-language literature up to April 1991. Twelve studies were found. DATA EXTRACTION: Two authors, working independently, abstracted raw data on mortality, recurrence, dumping, diarrhea and postoperative Visick grading. DATA SYNTHESIS: A ratio greater than 1.0 indicates a higher proportion occurring with TVP and less than 1.0 a higher proportion with HSV. [Table: See Text] CONCLUSION: The results suggest that the likelihood of adverse long-term sequelae is higher with TVP whereas the likelihood of recurrence is higher with HSV.

Health care system and medical education in Canada. 1. Review of the health care context in a time of change.

Over the past three decades Canada has developed an exemplary system of universal health care. However, current financial constraints threaten to undermine the very foundation of the system that represents Canada's respect for social justice. The first of these two articles (Part 1) discusses the unique characteristics of the system, a comparative view of universal health care systems, the reaction of the medical profession to governmental control, and finally the funding and manpower shortages that are compelling a review of the values and organization that have sustained the system thus far. The second article (Part 2) reviews the organization and funding of medical education and the impact of some critical changes in the health care system on postgraduate medical education. Issues related to new licensure and certification requirements, changing population demographics and approaches to health care delivery, manpower needs, and the attitude and expectations of the public are discussed.

Health care system and medical education in Canada. 2. Impact of changes in the health care system on medical education.

Over the past three decades Canada has developed an exemplary system of universal health care. However, current financial constraints threaten to undermine the very foundation of the system that represents Canada's respect for social justice. The first of these two articles (Part 1) discusses the unique characteristics of the system, a comparative view of universal health care systems, the reaction of the medical profession to governmental control, and finally the funding and manpower shortages that are compelling a review of the values and organization that have sustained the system thus far. The second article (Part 2) reviews the organization and funding of medical education and the impact of some critical changes in the health care system on postgraduate medical education. Issues related to new licensure and certification requirements, changing population demographics and approaches to health care delivery, manpower needs, and the attitude and expectations of the public are discussed.