Attendings' Perceptions of Authentic Evaluation Criteria for Effective Surgical Consults.

OBJECTIVE: The authors aimed to investigate faculty evaluation criteria for an effective oral surgical presentation in actual patient care contexts. DESIGN: We conducted a 2-step observation-based qualitative study. Residents audiotaped oral presentations of a surgical consult to an attending. Evaluation panels listened to the recordings and discussed to develop joint feedback for the resident. The panel discussions were recorded and served as the data source for this study. We analyzed the data following the grounded theory approach using open coding and axial coding. SETTING: The study setting was at Southern Illinois University School of Medicine, a 5-year general surgery residency program in Springfield, Illinois. PARTICIPANTS: Thirteen residents out of 19 in the program participated by virtue of having submitted recordings of a patient care consult presentation via phone. Evaluation panels consisted of general surgery academic and community faculty, as well as senior residents. RESULTS: Several criteria for effective oral presentations emerged that have rarely been discussed in prior literature. Themes included: (1) The strategic opening is critical as it "sets the stage" and frames how the attending will listen. Situational factors, such as consideration of time of the day and urgency, should be accounted for in the opening. (2) A deductive structure defines the relevance of the presented information. Clinical judgement should precede supporting evidence. Attending physicians perceive important information as unnecessary if provided outside of this framework. (3) Established trust between a resident and a surgeon determines the level of detail expected of the presenting resident. With increasing trust, surgeons expect residents to present fewer details; if too much detail is included, the presentation may be assessed as ineffective. (4) Surgical descriptions are appreciated for their value in promoting the attending's visualization or mental picture of the patient condition. (5) Oral emphasis using voice tone and pace can be helpful for capturing attending attention. CONCLUSIONS: These findings can be utilized to improve the current training program and assessment rubrics toward contextualized work-based assessment practices in surgery. Oral patient presentation skills are neither static nor universal, but fluid and reflexive, based on trust, and situational factors.

Medical students' reflections on surgical educators' professionalism: Contextual nuances in the hidden curriculum.

INTRODUCTION: Surgical educators' professional behavior constitutes a hidden curriculum and impacts trainee's professional identity formation. This study explores the nuances of professional behaviors as observed in varying surgical settings. METHODS: 411 Transcripts originated from essays written by MS3 students during their surgical clerkship from 2010 to 2016 were collated. Employing a qualitative research methodology, we conducted a thematic analysis to uncover specific meaning emerging from medical student reflections' on surgical professionalism. RESULTS: In clinics, taking time and protecting patient privacy; in the OR, control over emotion during difficult situations and attention to learners; and in the inpatient setting, showing accountability above normal expected behavior were noted as professional. Similarly, unprofessional behaviors in these contexts paralleled lack of these attributes. CONCLUSIONS: Behaviors observed and the attributes of professionalism in the surgical learning environment have contextual nuances. These variations in professionalism can be utilized in deliberate development of professionalism in surgery.

Mock Pages Are a Valid Construct for Assessment of Clinical Decision Making and Interprofessional Communication.

IMPORTANCE: Answering pages from nurses about patients in need of immediate attention is one of the most difficult challenges a resident faces during their first days as a physician. A Mock Page program has been developed and adopted into a national surgical resident preparatory curriculum to prepare senior medical students for this important skill. OBJECTIVE: The purpose of this study is to assess standardized mock page cases as a valid construct to assess clinical decision making and interprofessional communication skills. DESIGN, SETTING, PARTICIPANTS: Mock page cases (n = 16) were administered to 213 senior medical students from 12 medical schools participating in a national surgical resident preparatory curriculum in 2013 and 2014. MAIN OUTCOME MEASURES: Clinical decision making and interprofessional communication were measured by case-specific assessments evaluating these skills which have undergone rigorous standard-setting to determine pass/fail cut points. RESULTS: Students' performance improved in general for both communication and clinical decision making over the 4-week course. Cases have been identified that seem to be best suited for differentiating high- from low-performing students. Chest pain, pulmonary embolus, and mental status change cases posed the greatest difficulty for student learners. CONCLUSIONS AND RELEVANCE: Simulated mock pages demonstrate an innovative technique for training students in both effective interprofessional communication and management of common postoperative conditions they will encounter as new surgical interns.

Do Surgeons and Gastroenterologists Describe Endoscopic Retrograde Cholangiopancreatography Differently? A Qualitative Study.

INTRODUCTION: Endoscopic retrograde cholangiopancreatography (ERCP) is a technically challenging procedure performed by both surgeons and gastroenterologists. There is controversy in the field regarding the training necessary to perform ERCP. Widely disparate requisite volumes of experience to achieve proficiency have been published by representatives of each specialty. The basis for these differences has not been fully explored, with particular reference to the cognitive mindset of the different specialties. METHODS: Structured cognitive task analytic interviews were conducted with 7 expert gastroenterologists and 4 expert surgeons from 4 institutions, each of whom performs ERCP as a common procedure in their clinical practice. A qualitative analysis and grounded theory approach was used, focusing specifically on duct cannulation as a critical procedural element. Transcripts were analyzed using Atlas.ti software. RESULTS: The qualitative analysis of 11 transcripts identified 173 unique codes from a total of 653 quotes. In all, 5 themes were found to describe the codes: judgment, teaching, techniques, principles, and equipment. Significant differences were noted between gastroenterologists and surgeons across these themes. Gastroenterologists placed emphasis on issues of judgment including rationale, and emphasized explanation and clarification in teaching. Surgeons placed more emphasis on use of visual cues, and emphasized technique and equipment nuances. CONCLUSION: The data suggest that gastroenterologists deconstruct ERCP competence based on application of rules and rationale through reflection. Surgeons focus more on visual and tactile cues in task deconstruction, and may be more likely to measure proficiency based on technical response to such cues. Based on this study, it is proposed that both specialties might have complementary roles in training therapeutic endoscopists.

Interdisciplinary cognitive task analysis: a strategy to develop a comprehensive endoscopic retrograde cholangiopancreatography protocol for use in fellowship training.

BACKGROUND: Endoscopic retrograde cholangiopancreatography is a challenging procedure performed by surgeons and gastroenterologists. We employed cognitive task analysis to identify steps and decision points for this procedure. METHODS: Standardized interviews were conducted with expert gastroenterologists (7) and surgeons (4) from 4 institutions. A procedural step and cognitive decision point protocol was created from audio-taped transcriptions and was refined by 5 additional surgeons. RESULTS: Conceptual elements, sequential actions, and decision points were iterated for 5 tasks: patient preparation, duodenal intubation, selective cannulation, imaging interpretation with related therapeutic intervention, and complication management. A total of 180 steps were identified. Gastroenterologists identified 34 steps not identified by surgeons, and surgeons identified 20 steps not identified by gastroenterologists. CONCLUSION: The findings suggest that for complex procedures performed by diverse practitioners, more experts may help delineate distinctive emphases differentiated by training background and type of practice.

How prepared are your interns to take calls? Results of a multi-institutional study of simulated pages to prepare medical students for surgery internship.

BACKGROUND: This study evaluated a simulated pages curriculum that was developed to assess communication and clinical decision making in medical students and interns. METHODS: A curriculum consisting of 14 simulated pages was administered across 5 institutions to 150 senior medical students. A 3-case subset was administered to interns who did not participate in the curriculum. Six expert surgeons identified critical fails and set passing scores for case-specific assessments using the Graphical Hofstee Method. RESULTS: Participants in the curriculum demonstrated superior clinical decision making compared with non-participants across all cases scenarios (P < .01). Average medical student scores for clinical decision making were 46.9%. Global ratings averaged 6.0 for communication and 5.2 for patient care. Passing rates averaged 46%. CONCLUSIONS: Participation in a mock page curriculum improved performance. The performance of participants based on expert standards set for simulated page performance highlight the need for innovative approaches to improve interns' preparedness to take calls.

Foundations for teaching surgeons to address the contributions of systems to operating room team conflict.

BACKGROUND: Prior research has shown that surgeons who effectively manage operating room conflict engage in a problem-solving stage devoted to modifying systems that contribute to team conflict. The purpose of this study was to clarify how systems contributed to operating room team conflict and clarify what surgeons do to modify them. METHODS: Focus groups of circulating nurses and surgeons were conducted at 5 academic medical centers. Narratives describing the contributions of systems to operating room conflict and behaviors used by surgeons to address those systems were analyzed using the constant comparative approach associated with a constructivist grounded theory approach. RESULTS: Operating room team conflict was affected by 4 systems-related factors: team features, procedural-specific staff training, equipment management systems, and the administrative leadership itself. Effective systems problem solving included advocating for change based on patient safety concerns. CONCLUSIONS: The results of this study provide clarity about how systems contribute to operating room conflict and what surgeons can do to effectively modify these systems. This information is foundational material for a conflict management educational program for surgeons.

The effect of model fidelity on colonoscopic skills acquisition. A randomized controlled study.

INTRODUCTION: Colonoscopic simulators offer the opportunity for skill acquisition in the preclinical setting. Currently available simulators vary widely with respect to level of fidelity and technological sophistication. Despite the belief that more realistic is better, there is a paucity of evidence regarding the relative effectiveness of simulator fidelity (high vs low) on the acquisition of basic colonoscopic skills. We hypothesized that novice learners can acquire basic colonoscopic skills using simulators, however fidelity of the simulator does not make a difference. METHODS: We randomly assigned novice third-year and fourth-year medical students to practice on either a low-fidelity or high-fidelity colonoscopy model. The low-fidelity model used is described in the module 16 of the American College of Surgeons/Association of Program Directors in Surgery surgical skills curriculum for residents, Phase 1: basic or core skills and tasks < http://elearning.facs.org/mod/resource/view.php?1d=450 >. The high-fidelity model was the AccuTouch colonoscopy simulator, Immersion Medical (AccuTouch CS) that has 6 different simulated scenarios for diagnostic colonoscopy (level 1-6). Both groups had 16 students and were given standard instruction by an expert with respect to the procedure and instrument handling on both models. Both groups were pretested and posttested on level 1 of the AccuTouch CS. The high-fidelity group practiced on level 2 and 4 of the AccuTouch CS, whereas the low-fidelity group practiced on the low-fidelity model for 2 sessions of 1 hour each. The computer-based evaluation parameters available on the AccuTouch CS were used to compare performances. RESULTS: Both groups had similar demographics. There were no significant differences in the baseline performances of either group. Each group demonstrated significant improvement for insertion time and percentage of mucosa visualized. However, there were no significant differences between the groups on posttesting on any of the measured parameters. CONCLUSIONS: Colonoscopic skill training on a low-fidelity model appears to be as effective as high-fidelity model training for basic endoscopic skill acquisition for novice learners.

Surgeons managing conflict in the operating room: defining the educational need and identifying effective behaviors.

BACKGROUND: Developing an operating room conflict management educational program for surgeons requires a formal needs assessment and information about behaviors that represent effective conflict management. METHODS: Focus groups of circulating room nurses and surgeons were conducted at 5 participating centers. Participants responded to queries about conflict management training, conflict consequences, and effective conflict management behaviors. Transcripts of these sessions served as the data for this study. RESULTS: Educational preparation for conflict management was inadequate consisting of trial and error with observed behaviors. Conflict and conflict mismanagement had negative consequences for team members and team performance. Four behaviors emerge as representing effective ways for surgeons to manage conflict. CONCLUSIONS: There is a clear educational need for conflict management education. Target behaviors have now been identified that can provide the basis for a theoretically grounded and contextually adapted instruction and assessment of surgeon conflict management.

Pursuing professional accountability: an evidence-based approach to addressing residents with behavioral problems.

OBJECTIVE: To develop an evidence-based approach to the identification, prevention, and management of surgical residents with behavioral problems. DESIGN: The American College of Surgeons and Southern Illinois University Department of Surgery hosted a 1-day think tank to develop strategies for early identification of problem residents and appropriate interventions. Participants read a selection of relevant literature before the meeting and reviewed case reports. SETTING: American College of Surgeons headquarters, Chicago, Illinois. PARTICIPANTS: Medical and nursing leaders in the field of resident education; individuals with expertise in dealing with academic law, mental health issues, learning deficiencies, and disruptive physicians; and surgical residents. MAIN OUTCOME MEASURES: Evidence-based strategies for the identification, prevention, and management of problem residents. RESULTS: Recommendations based on the literature and expert opinions have been made for the identification, remediation, and reassessment of problem residents. CONCLUSIONS: It is essential to set clear expectations for professional behavior with faculty and residents. A notice of deficiency should define the expected acceptable behavior, timeline for improvement, and consequences for noncompliance. Faculty should note and address systems problems that unintentionally reinforce and thus enable unprofessional behavior. Complaints, particularly by new residents, should be investigated and addressed promptly through a process that is transparent, fair, and reasonable. The importance of early intervention is emphasized.

Career development resource: educational leadership in a department of surgery: vice chairs for education.

The growing appreciation of the need to adopt an evidence-based approach to teaching and assessment has led to a demand for faculty who are well versed in best practices in education. Surgeons with interest and expertise in instruction, curriculum development, educational research, and evaluation can have an important impact on the educational mission of a department of surgery. The increased fervor for accountability in education together with the challenges imposed by accreditation agencies and hospitals has made educational leadership responsibilities more time consuming and complex. In response to this, an increasing number of department chairs created Vice Chair for Education positions to support clerkship and program directors and ensure the department's education mission statement is fulfilled.

Using the hidden curriculum to teach professionalism during the surgery clerkship.

BACKGROUND: It has been shown that medical student professionalism is influenced by the hidden curriculum, although the extent to which this occurs during the surgery clerkship is unknown. Furthermore, the processes within the hidden curriculum have been used to teach professionalism to medical students, but this strategy has not been used during the surgery clerkship. The purpose of this study was to review a 2-year experience with a surgery clerkship instructional session where the hidden curriculum was used to teach professionalism to medical students. STUDY DESIGN: Medical student essays were analyzed to evaluate the influence of the hidden curriculum on their ideas about professionalism and to identify specific behaviors that they regarded as professional and unprofessional. The instructional session was evaluated using the average satisfaction session ratings and through an analysis of medical student session evaluation comments. RESULTS: Seventy-five percent of medical students reported that their ideas about professionalism changed. This change involved their general concepts about professionalism, identifying specific behaviors that they planned to adopt or avoid, or developing opinions about the professionalism of surgeons. The average satisfaction rating was consistently high throughout the study period, and the most helpful session feature was reported as the opportunity to share and discuss their observations. CONCLUSIONS: The hidden curriculum has a substantial influence on the development of professionalism of medical students during the surgery clerkship. It was possible to illuminate and use the hidden curriculum to create an instructional session devoted to professionalism for medical students on the surgery clerkship.

Career development needs of vice chairs for education in departments of surgery.

AIM: To identify the career development needs Vice Chair for Education in Surgery Departments (VCESDs). METHODS: In all, 33 VCESDs were invited to complete an online survey to identify the scope of duties, scholarly activity, job satisfaction, and career development needs. RESULTS: A total of 29/33 (88%) VCESDs responded. Time constraints were the most frequent impediment for MDs vs. PhDs (p < 0.05). Dominant faculty development needs were conducting educational research (2.0 ± 0.78 for MDs, 1.33 ± 0.76 for PhDs), developing resident selection systems (1.68 ± 0.73), and mentorship programs (1.95 ± 0.77) for MDs, and developing teach the teacher programs (1 ± 0), and program performance evaluation systems (1.33 ± 0.76) for PhDs. The skills deemed to be of greatest importance were ability to communicate effectively (1.27 ± 0.55), resolve personnel conflicts (1.32 ± 0.57), and introduce change (1.41 ± 0.59). PhDs revealed a greater need to learn strategies for dealing with disruptive faculty (1.0 ± 0 vs 2.15 ± 0.87). CONCLUSIONS: This information will inform the future career development of VCESDs and will assist Department Chairs who wish to recruit and retain VCESDs.

Engaging medical students in the feedback process.

BACKGROUND: There are potential advantages to engaging medical students in the feedback process, but efforts to do so have yielded mixed results. The purpose of this study was to evaluate a student-focused feedback instructional session in an experimental setting. METHODS: Medical students were assigned randomly to either the intervention or control groups and then assigned randomly to receive either feedback or compliments. Tests of knowledge, skills, and attitudes were given before and after the intervention. RESULTS: There was a significant gain of knowledge and skill in the group that received instruction. Satisfaction was higher after compliments in the control group but higher after feedback in the instructional group. There was no change in the subject's willingness to seek feedback. CONCLUSIONS: A student-focused component should be carefully included as part of an overall effort to improve feedback in surgical education. The role of medical student attitudes about feedback requires further investigation.

Knowledge and attitudes regarding colorectal cancer screening among medical students: a tale of two schools.

Colorectal cancer (CRC) screening has been shown to decrease the incidence of CRC cancers and decrease mortality. Studies show that the most important predictor of patient compliance with CRC screening is physician recommendation. We assessed the knowledge and attitudes of medical students regarding cancer screening. A study-specific questionnaire was distributed to medical students (MS) at two medical schools. There was a significant difference in the percentage of correctly answered questions regarding screening recommendations between first year MS and all other years for both schools. However, MS attitudes towards CRC screening were consistent between classes and schools. Although most MS had positive attitudes regarding cancer screening our survey identified several important deficits in knowledge.

Skills coaches as part of the educational team: a randomized controlled trial of teaching of a basic surgical skill in the laboratory setting.

BACKGROUND: The aim of this study was to compare the laboratory teaching of a basic technical skill by a nonphysician skills coach and a faculty surgeon. METHODS: Medical students were randomized to instruction of skin suturing in the skills laboratory by a faculty surgeon or by a nonphysician skills coach. Testing of performance occurred at 3 time points. Other faculty surgeons, blinded to identities and training groups, rated performance. RESULTS: Forty-nine students participated. Baseline fourth-year student mean scores showed no significant difference between training groups. Third-year and fourth-year student performance showed no difference between training groups on postintervention testing. Delayed testing also showed no difference in third-year student scores. CONCLUSIONS: Training by either a nonsurgeon skills coach or a faculty surgeon resulted in no difference in performance on a basic surgical skill. This was true for students with and without prior experience and was also true after subsequent clinical experiences. Nonphysician coaches may ease the teaching burden of surgical faculty members while providing similar quality of instruction for trainees.

Robot-assisted laparoscopic skills development: formal versus informal training.

INTRODUCTION: The learning curve for robotic surgery is not completely defined, and ideal training components have not yet been identified. We attempted to determine whether skill development would be accelerated with formal, organized instruction in robotic surgical techniques versus informal practice alone. MATERIALS AND METHODS: Forty-three medical students naive to robotic surgery were randomized into two groups and tested on three tasks using the robotic platform. Between the testing sessions, the students were given equally timed practice sessions. The formal training group participated in an organized, formal training session with instruction from an attending robotic surgeon, whereas the informal training group participated in an equally timed unstructured practice session with the robot. The results were compared based on technical score and time to completion of each task. RESULTS: There was no difference between groups in prepractice testing for any task. In postpractice testing, there was no difference between groups for the ring transfer tasks. However, for the suture placement and knot-tying task, the technical score of the formal training group was significantly better than that of the informal training group (p < 0.001), yet time to completion was not different. CONCLUSION: Although formal training may not be necessary for basic skills, formal instruction for more advanced skills, such as suture placement and knot tying, is important in developing skills needed for effective robotic surgery. These findings may be important in formulating potential skills labs or training courses for robotic surgery.

Refining the evaluation of operating room performance.

PURPOSE: An accurate and consistent evaluation of resident operative performance is necessary but difficult to achieve. This study continues the examination of the Southern Illinois University (SIU) operative performance rating system (OPRS) by studying additional factors that may influence reliability, accuracy, and interpretability of results. METHODS: OPRS evaluations of surgical residents by faculty at SIU, from 2001 to 2008, were analyzed for the most frequently rated procedures to determine (1) the elapsed time from the procedure until completion of rating, (2) the patterns in responses of procedure-specific and global surgical skills items, and (3) whether particular evaluating surgeons differed in their stringency of ratings of resident operative performance. RESULTS: In all, 566 evaluations were analyzed, which consisted of open colectomy (n = 125), open inguinal hernia (n = 103), laparoscopic cholecystectomy (n = 199), and excisional biopsy (n = 139). The number of residents evaluated per training level (PGY) ranged from 88 to 161. The median time to completion of evaluations was 11 days, 9 hours. The quickest evaluation was 18 hours after assignment. Most were completed within 4.5 to 22 days. Procedure-specific and global scale scores resulted in similar rank-ordering of performances (single-measure intraclass correlation using the consistency model = 0.88; 95% confidence interval [CI] = 0.87-0.90) and similar absolute OPRS scores (single-measure intraclass correlation using the consistency model = 0.89; 95% CI, 0.87-0.90). Evaluating surgeons differed in stringency of ratings across procedures (average difference = 1.4 points of 5 possible points). Resident performance improved with increasing PGY level for all 4 procedures. CONCLUSIONS: Substantial time elapses between performance in the operating room and the completion of the evaluation. This raises the question of whether surgeons remember the nuances of the procedure well enough to rate performance accurately. The item type used for rating does not affect the absolute rating assigned or the rank ordering of the performance. Differences in stringency of evaluators indicate the need for multiple resident performance observations by multiple surgeons. These findings are the foundation for an upcoming multi-institutional trial.

A theory-based curriculum for enhancing surgical skillfulness.

BACKGROUND: Curricula for surgical technical skills laboratories have traditionally been designed to accommodate the clinical activities of residents, so they typically consist of individual, episodic training sessions. We believe that the skills laboratory offers an opportunity to design a surgical skills curriculum based on the fundamental elements known to be important for motor skill instruction. We hypothesized that training novices with such a curriculum for a 1-month period would yield skills performance levels equivalent to those of second year surgery residents who had trained in a traditional program. STUDY DESIGN: Fourth-year medical students served as study subjects (novice group) during a 4-week senior elective. They were taught each skill during a 1-week period. Subjects received instruction by a content expert followed by a 1-week period of deliberate practice with feedback. The novice performances were videotaped both before and after the intervention, and each videotape was evaluated in a blinded fashion by experts using a validated evaluation instrument. These results were compared with skill performance ratings of first- and second-year surgery residents that had been accumulated over the previous 3 years. RESULTS: Average performance ratings for the novices substantially improved for all four skills after training. There was no marked difference between average performance ratings of postintervention novice scores when compared with the average scores in the resident group. Inter-rater agreement in scoring for the videotaped novice performances exceeded 0.87 (intraclass correlation) for all ratings of pre- and posttraining. CONCLUSIONS: These results demonstrate the effectiveness of a laboratory-based training program that includes fundamentals of motor skills acquisition.

Surgeon information transfer and communication: factors affecting quality and efficiency of inpatient care.

OBJECTIVE: To determine the nature of surgeon information transfer and communication (ITC) errors that lead to adverse events and near misses. To recommend strategies for minimizing or preventing these errors. SUMMARY BACKGROUND DATA: Surgical hospital practice is changing from a single provider to a team-based approach. This has put a premium on effective ITC. The Information Transfer and Communication Practices (ITCP) Project is a multi-institutional effort to: 1) better understand surgeon ITCP and their patient care consequences, 2) determine what has been done to improve ITCP in other professions, and 3) recommend ways to improve these practices among surgeons. METHODS: Separate, semi-structured focus group sessions were conducted with surgical residents (n = 59), general surgery attending physicians (n = 36), and surgical nurses (n = 42) at 5 medical centers. Case descriptions and general comments were classified by the nature of ITC lapses and their effects on patients and medical care. Information learned was combined with a review of ITC strategies in other professions to develop principles and guidelines for re-engineering surgeon ITCP. RESULTS: : A total of 328 case descriptions and general comments were obtained and classified. Incidents fell into 4 areas: blurred boundaries of responsibility (87 reports), decreased surgeon familiarity with patients (123 reports), diversion of surgeon attention (31 reports), and distorted or inhibited communication (67 reports). Results were subdivided into 30 contributing factors (eg, shift change, location change, number of providers). Consequences of ITC lapses included delays in patient care (77% of cases), wasted surgeon/staff time (48%), and serious adverse patient consequences (31%). Twelve principles and 5 institutional habit changes are recommended to guide ITCP re-engineering. CONCLUSIONS: Surgeon communication lapses are significant contributors to adverse patient consequences, and provider inefficiency. Re-engineering ITCP will require significant cultural changes.