Using the Operating Room Black Box to Assess Surgical Team Member Adaptation Under Uncertainty: An Observational Study.

OBJECTIVE: Identify how surgical team members uniquely contribute to teamwork and adapt their teamwork skills during instances of uncertainty. BACKGROUND: The importance of surgical teamwork in preventing patient harm is well documented. Yet, little is known about how key roles (nurse, anesthesiologist, surgeon, and medical trainee) uniquely contribute to teamwork during instances of uncertainty, particularly when adapting to and rectifying an intraoperative adverse event (IAE). METHODS: Audiovisual data of 23 laparoscopic cases from a large community teaching hospital were prospectively captured using OR Black Box. Human factors researchers retrospectively coded videos for teamwork skills (backup behavior, coordination, psychological safety, situation assessment, team decision-making, and leadership) by team role under 2 conditions of uncertainty: associated with an IAE versus no IAE. Surgeons identified IAEs. RESULTS: In all, 1015 instances of teamwork skills were observed. Nurses adapted to IAEs by expressing more backup behavior skills (5.3× increase; 13.9 instances/hour during an IAE vs 2.2 instances/hour when no IAE) while surgeons and medical trainees expressed more psychological safety skills (surgeons: 3.6× increase; 30.0 instances/hour vs 6.6 instances/hour and trainees: 6.6× increase; 31.2 instances/hour vs 4.1 instances/hour). All roles expressed fewer situation assessment skills during an IAE versus no IAE. CONCLUSIONS: OR Black Box enabled the assessment of critically important details about how team members uniquely contribute during instances of uncertainty. Some teamwork skills were amplified, while others dampened when dealing with IAEs. The knowledge of how each role contributes to teamwork and adapts to IAEs should be used to inform the design of tailored interventions to strengthen interprofessional teamwork.

Eye Tracking Use in Surgical Research: A Systematic Review.

INTRODUCTION: Eye tracking (ET) is a popular tool to study what factors affect the visual behaviour of surgical team members. To our knowledge, there have been no reviews to date that evaluate the broad use of ET in surgical research. This review aims to identify and assess the quality of this evidence, to synthesize how ET can be used to inform surgical practice, and to provide recommendations to improve future ET surgical studies. METHODS: In line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, a systematic literature review was conducted. An electronic search was performed in MEDLINE, Cochrane Central, Embase, and Web of Science databases up to September 2020. Included studies used ET to measure the visual behaviour of members of the surgical team during surgery or surgical tasks. The included studies were assessed by two independent reviewers. RESULTS: A total of 7614 studies were identified, and 111 were included for data extraction. Eleven applications were identified; the four most common were skill assessment (41%), visual attention assessment (22%), workload measurement (17%), and skills training (10%). A summary was provided of the various ways ET could be used to inform surgical practice, and three areas were identified for the improvement of future ET studies in surgery. CONCLUSIONS: This review provided a comprehensive summary of the various applications of ET in surgery and how ET could be used to inform surgical practice, including how to use ET to improve surgical education. The information provided in this review can also aid in the design and conduct of future ET surgical studies.

System Factors Affecting Patient Safety in the OR: An Analysis of Safety Threats and Resiliency.

OBJECTIVE: The objective of this study is to determine the characteristics and frequency of intraoperative safety threats and resilience supports using a human factors measurement tool. BACKGROUND: Human factors analysis can provide insight into how system elements contribute to intraoperative adverse events. Empiric evidence on safety threats and resilience in surgical practice is lacking. METHODS: A cross-sectional study of 24 patients undergoing elective laparoscopic general surgery at a single center in the Netherlands from May to November, 2017 was conducted. Video, audio, and patient physiologic data from all included procedures were obtained through a multichannel synchronized recording device. Trained analysts reviewed the recordings and coded safety threats and resilience supports. The codes were categorized into 1 of 6 categories (person, task, tools and technology, physical environment, organization, and external environment). RESULTS: A median of 14 safety threats [interquartile range (IQR) 11-16] and 12 resilience supports (IQR 11-16) were identified per case. Most safety threat codes (median 9, IQR 7-12) and resilience support codes (median 10, IQR 7-12) were classified in the person category. The organization category contained a median of 2 (IQR 1-2) safety threat codes and 2 (IQR 2-3) resilience support codes per case. The tools and technology category contributed a small number of safety threats (median 1 per case, IQR 0-1), but rarely provided resilience support. CONCLUSIONS: Through a detailed human factors analysis of elective laparoscopic general surgery cases, this study provided a quantitative analysis of the existing safety threats and resilience supports in a modern endoscopic operating room.

Quantifying the "Assistant Effect" in Robotic-Assisted Radical Prostatectomy (RARP): Measures of Technical Performance.

PURPOSE: Surgeons are reliant on the bedside assistant during robotic surgeries. Using a modified global rating scale (GRS), we aim to assess the association between an assistant's technical skill on surgeon performance in Robotic-Assisted Radical Prostatectomy (RARP). METHODS: Prospective, intraoperative video from RARP cases at three centers were collected. Baseline demographic and RARP-experience data were collected from participating surgeons and trainees. The dissection of the prostatic pedicle and neurovascular bundle step (NVB) was analyzed. Expert analysts scored the console surgeon performance using the Global Evaluative Assessment of Robotic Skills (GEARS), and the bedside assistant performance using a modified Objective Structured Assessment of Technical Skills (aOSATS). The primary outcome is the association between console surgeon performance, as measured by GEARS, and assistant skill, as measured by aOSATS. Spearman's rho correlations were used to test the relationship between assistant and surgeon technical performance, and a multivariable linear regression model was created to test this association while controlling for patient factors. RESULTS: 92 RARP cases were available for the analysis, comprising 14 console surgeons and 22 different bedside assistants. In only 5 (5.4%) cases, the neurovascular bundle step was completed by a trainee, and in 13 (14.1%) of cases, a staff-level surgeon acted as the bedside assistant. aOSATS score was significantly associated with robotic console experience (P = 0.011), and prior laparoscopic experience (P < 0.001). Assistant aOSATS score showed a weak but significant correlation with surgeon GEARS score during the neurovascular bundle step (spearman's rho = 0.248, P = 0.028). On linear regression, aOSATS remained a significant predictor of console surgeon performance (P = 0.016), after controlling for patient age and BMI, prostate volume, tumor stage, and presence of nerve-sparing. CONCLUSIONS: This is the first study to assess the association between assistant technical skill and surgeon performance in RARP. Additionally, we have provided validity evidence for a modified OSATS global rating scale for training and assessing bedside assistant performance.

Development of a Model for Video-Assisted Postoperative Team Debriefing.

BACKGROUND: Video-assisted debriefing may be a powerful tool to improve surgical team performance. Nevertheless, a true operating team debriefing culture is lacking to date. This study aimed to find evidence on how to debrief the surgical team and develop a model suitable for debriefing using a video and medical data recorder (MDR) in the operating room (OR). METHODS: A review of the PubMed and Embase databases and Cochrane Library was performed. The identified literature was studied and combined with a conceptual framework to develop a model for postoperative video-assisted team debriefing. Thirty-five surgical cases were recorded with an MDR and debriefed with the operating team using the proposed debrief model and a standardized video-assisted performance report. A questionnaire was used to assess the participants' satisfaction with this debrief model. RESULTS: Debrief models and methods are extensively described in the current medical literature. An overview was provided. The OR team needs a structured debrief model, minimizing resource, effort, and motivational constraints. A structured six-step team debrief model suitable for video-assisted OR team debriefing was developed. The model was tested in 35 multidisciplinary MDR-assisted debriefing sessions and the debriefing sessions were overall rated with a mean of 7.8 (standard deviation 1.4, 10-point Likert scale) by participants. CONCLUSIONS: Debriefing surgical teams using a video and MDR in the OR requires a model on how to use such recordings optimally. To date, no such model existed. The proposed debrief model was tested using a multisource MDR and may be used to facilitate OR debriefing across various settings.

Six Sigma in surgery: how to create a safer culture in the operating theatre using innovative technology.

Safe delivery of patient care in the operating theatre is complex and co-dependent of many individual, organisational, and environmental factors, including patient, task and technology, individual, and human factors. The Six Sigma approach aims to implement a data-driven strategy to reduce variability and consequently improve safety. Analytical data platforms such as a Black Box ought to be embraced to support process optimisation and ultimately create a higher level of Six Sigma safety performance of the operating theatre team.

System Factors Affecting Intraoperative Risk and Resilience: Applying a Novel Integrated Approach to Study Surgical Performance and Patient Safety.

OBJECTIVE: To identify and categorize system factors in complex laparoscopic surgery that have the potential to either threaten patient safety or support system resilience. BACKGROUND: The operating room is a uniquely complex sociotechnical work system wherein surgical successes prevail despite pervasive safety threats. Holistically characterizing intraoperative factors that thus support system resilience in addition to those that threaten patient safety using contextual methodologies is critical for optimizing surgical safety overall. METHOD: In this prospective descriptive interdisciplinary study, 19 audio/video recordings of complex laparoscopic general surgical procedures were directly observed and transcribed. Using a qualitative systems-based approach, intraoperative human factors with the potential to impact patient safety, either as a safety threat or as a support for resilience, were identified. Adverse events were further assessed for shared threats and supports. Data collection was guided by the Systems Engineering Initiative for Patient Safety 2.0 work system model. RESULTS: A total of 1083 relevant observations were made over 39.8 hours of operative time, enabling the identification of 79 distinct safety threats and 67 resilience supports within the surgical system. Safety threats associated with the physical environment, tasks, organization, and equipment were prevalent and observed in equal measure, whereas supports for resilience were predominantly attributed to clinician behaviors, including proactive team management and skills coaching. Two subclinical adverse events were identified; shared safety threats included suboptimal technology design, whereas shared resilience supports included calm clinician behavior and redundant intraoperative resourcing. CONCLUSIONS: Safety threats and resilience supports were found to be systematic in the surgical setting. Identified safety threats should be prioritized for remediation, and clinician behaviors that contribute to fostering resilience should be valued and protected.

A Novel Method of Setting Performance Standards in Surgery Using Patient Outcomes.

OBJECTIVE: To describe a novel, outcome-based method of standard setting that differentiates between clinical outcomes rather than arbitrary educational goals. BACKGROUND: Standard setting methods used in assessments of procedural skill are currently not evidence-driven or outcome-based. This represents a potential obstacle for the broad implementation of these evaluations in summative assessments such as certification and credentialing. METHODS: The concept is based on deriving a receiver operating characteristic curve from a regression model that incorporates measures of intraoperative surgeon performance and confounding patient characteristics. This allows the creation of a performance standard that best predicts a clinically significant outcome of interest. The discovery cohort used to create the predictive model was derived from pilot data that used the Global Evaluative Assessment of Robotic Skill assessment tool to predict patient urinary continence 3 months following robotic-assisted radical prostatectomy. RESULTS: A receiver operating characteristic curve with an area under the curve of 0.75 was created from predicted probability statistic generated by the predictive model. We chose a predicted probability of 0.35, based on an optimal tradeoff in sensitivity and specificity (Youden Index). Rearranging the regression equation, we determined the performance score required to predict a 35%, patient-adjusted probability of postoperative urinary incontinence. CONCLUSIONS: This novel methodology is context, patient, and assessment-specific. Current standard setting methods do not account for the heterogeneity of the clinical environment. Workplace-based assessments in competency-based medical education require standards that are credible to the educator and the trainee. High-stakes assessments must ensure that surgeons have been evaluated to a standard that prioritizes satisfactory patient outcomes and safety.

Technical Performance as a Predictor of Clinical Outcomes in Laparoscopic Gastric Cancer Surgery.

OBJECTIVE: The purpose of this study was to evaluate the relationship between technical performance and patient outcomes in laparoscopic gastric cancer surgery. BACKGROUND: Laparoscopic gastrectomy for cancer is an advanced procedure with high rate of postoperative morbidity and mortality. Many variables including patient, disease, and perioperative management factors have been shown to impact postoperative outcomes; however, the role of surgical performance is insufficiently investigated. METHODS: A retrospective review was performed for all patients who had undergone laparoscopic gastrectomy for cancer at 3 teaching institutions between 2009 and 2015. Patients with available, unedited video-recording of their procedure were included in the study. Video files were rated for technical performance, using Objective Structured Assessments of Technical Skills (OSATS) and Generic Error Rating Tool instruments. The main outcome variable was major short-term complications. The effect of technical performance on patient outcomes was assessed using logistic regression analysis with backward selection strategy. RESULTS: Sixty-one patients with available video recordings were included in the study. The overall complication rate was 29.5%. The mean Charlson comorbidity index, type of procedure, and the global OSATS score were included in the final predictive model. Lower performance score (OSATS ≤29) remained an independent predictor for major short-term outcomes (odds ratio 6.49), while adjusting for comorbidities and type of procedure. CONCLUSIONS: Intraoperative technical performance predicts major short-term outcomes in laparoscopic gastrectomy for cancer. Ongoing assessment and enhancement of surgical skills using modern, evidence-based strategies might improve short-term patient outcomes. Future work should focus on developing and studying the effectiveness of such interventions in laparoscopic gastric cancer surgery.

Underreporting of Veress Needle Injuries: Comparing Direct Observation and Chart Review Methods.

BACKGROUND: Adverse events in surgery occur frequently, increase likelihood of postoperative morbidity, and mostly take place in the operating rooms. Several surgeons have advocated for learning from adverse events and near misses to help improve patient safety. To do so, one must first understand how to accurately identify and report intraoperative events. MATERIALS AND METHODS: Consecutive laparoscopic cases performed in a referral center were included in the cohort. Veress needle (VN) injuries were characterized according to a priori established criteria. Two methods were used to identify VN injuries: direct observation and patient chart review. For direct observation, trained surgeon assessors identified the outcomes using a comprehensive data capture platform called the operating room black box. On the other hand, operative reports and patient charts were reviewed by trained assessors to identify reported VN injuries. RESULTS: Hundred thirty-one cases were analyzed. There were 12 (9%) VN injuries identified by direct observation compared to 3 (2%) identified in patient chart review method. Injuries to the liver and stomach were identified by both methods, whereas injuries to the omentum were not reported in patient charts even if they required rectification. There were seven VN injuries that required rectification, lasting up to 12% of the operating time. There were 47 (35%) near misses identified through direct observation, whereas none was reported in patient charts. CONCLUSIONS: Direct observation enables characterization of VN injury and near misses with far greater detail and accuracy than patient chart review.

Video Technologies for Recording Open Surgery: A Systematic Review.

Video recording of surgical procedures is an important tool for surgical education, performance enhancement, and error analysis. Technology for video recording open surgery, however, is limited. The objective of this article is to provide an overview of the available literature regarding the various technologies used for intraoperative video recording of open surgery. A systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines using the MEDLINE, Cochrane Central, and EMBASE databases. Two authors independently screened the titles and abstracts of the retrieved articles, and those that satisfied the defined inclusion criteria were selected for a full-text review. A total of 2275 publications were initially identified, and 110 were included in the final review. The included articles were categorized based on type of article, surgical subspecialty, type and positioning of camera, and limitations identified with their use. The most common article type was primary-technical (29%), and the dominant specialties were general surgery (22%) and plastic surgery (18%). The most commonly cited camera used was the GoPro (30%) positioned in a head-mount configuration (60%). Commonly cited limitations included poor video quality, inadequate battery life, light overexposure, obstruction by surgical team members, and excessive motion. Open surgery remains the mainstay of many surgical specialties today, and technological innovation is absolutely critical to fulfill the unmet need for better video capture of open surgery. The findings of this article will be valuable for guiding future development of novel technology for this purpose.

Implementing assessments of robot-assisted technical skill in urological education: a systematic review and synthesis of the validity evidence.

OBJECTIVES: To systematically review and synthesise the validity evidence supporting intraoperative and simulation-based assessments of technical skill in urological robot-assisted surgery (RAS), and make evidence-based recommendations for the implementation of these assessments in urological training. MATERIALS AND METHODS: A literature search of the Medline, PsycINFO and Embase databases was performed. Articles using technical skill and simulation-based assessments in RAS were abstracted. Only studies involving urology trainees or faculty were included in the final analysis. RESULTS: Multiple tools for the assessment of technical robotic skill have been published, with mixed sources of validity evidence to support their use. These evaluations have been used in both the ex vivo and in vivo settings. Performance evaluations range from global rating scales to psychometrics, and assessments are carried out through automation, expert analysts, and crowdsourcing. CONCLUSION: There have been rapid expansions in approaches to RAS technical skills assessment, both in simulated and clinical settings. Alternative approaches to assessment in RAS, such as crowdsourcing and psychometrics, remain under investigation. Evidence to support the use of these metrics in high-stakes decisions is likely insufficient at present.

Approach to asymptomatic paraesophageal hernia: watchful waiting or elective laparoscopic hernia repair?

BACKGROUND: There is no consensus on whether asymptomatic paraesophageal hernia (PEH) should be operated. Some argue that surgery is necessary prophylaxis against potentially catastrophic consequences of acute complications in untreated PEH. Others reason that the acute complications are rare and emergent operations have relatively low mortality. In the laparoscopic era, elective operations have become safer and less morbid. However, recent studies report high incidence of recurrent hernia, some of which affect quality of life and require further interventions. In light of these new findings, we investigated whether asymptomatic PEH should receive elective laparoscopic hernia repair (ELHR) or watchful waiting (WW). METHODS: A Markov Monte Carlo microsimulation decision analysis model followed a hypothetical cohort of asymptomatic PEH patients who have predominantly female gender and normally distributed mean age of 62.5 years for the lifetime. Accrued health benefits expressed in quality-adjusted life months (QALM) were compared between two strategies: WW and ELHR. Two-dimensional simulations were performed to account for uncertainties in the model. Deterministic sensitivity analyses were performed to test key assumptions. RESULTS: After considering both individual- and parameter-level uncertainties in the two-dimensional simulations, WW was the superior strategy in 82% of the simulations, accumulating mean 5 QALM more than ELHR (168 vs. 163). Our model was robust to deterministic sensitivity analyses and was internally validated, which supported the validity of our results. CONCLUSIONS: Patients with asymptomatic PEH are more likely to achieve greater health outcomes if they undergo WW as initial treatment than ELHR.

The Effect of Technical Performance on Patient Outcomes in Surgery: A Systematic Review.

OBJECTIVE: Systematic review of the effect of intraoperative technical performance on patient outcomes. BACKGROUND: The operating room is a high-stakes, high-risk environment. As a result, the quality of surgical interventions affecting patient outcomes has been the subject of discussion and research for years. METHODS: MEDLINE, EMBASE, PsycINFO, and Cochrane databases were searched. All surgical specialties were eligible for inclusion. Data were reviewed in regards to the methods by which technical performance was measured, what patient outcomes were assessed, and how intraoperative technical performance affected patient outcomes. Quality of evidence was assessed using the Medical Education Research Study Quality Instrument (MERSQI). RESULTS: Of the 12,758 studies initially identified, 24 articles (7775 total participants) were ultimately included in this review. Seventeen studies assessed the performance of the faculty alone, 2 assessed both the faculty and trainees, 1 assessed trainees alone, and in 4 studies, the level of the operating surgeon was not specified. In 18 studies, a performance assessment tool was used. Patient outcomes were evaluated using intraoperative complications, short-term morbidity, long-term morbidity, short-term mortality, and long-term mortality. The average MERSQI score was 11.67 (range 9.5-14.5). Twenty-one studies demonstrated that superior technical performance was related to improved patient outcomes. CONCLUSIONS: The results of this systematic review demonstrated that superior technical performance positively affects patient outcomes. Despite this initial evidence, more robust research is needed to directly assess intraoperative technical performance and its effect on postoperative patient outcomes using meaningful assessment instruments and reliable processes.

Setting Performance Standards for Technical and Nontechnical Competence in General Surgery.

OBJECTIVES: The objectives of this study were to (1) create a technical and nontechnical performance standard for the laparoscopic cholecystectomy, (2) assess the classification accuracy and (3) credibility of these standards, (4) determine a trainees' ability to meet both standards concurrently, and (5) delineate factors that predict standard acquisition. BACKGROUND: Scores on performance assessments are difficult to interpret in the absence of established standards. METHODS: Trained raters observed General Surgery residents performing laparoscopic cholecystectomies using the Objective Structured Assessment of Technical Skill (OSATS) and the Objective Structured Assessment of Non-Technical Skills (OSANTS) instruments, while as also providing a global competent/noncompetent decision for each performance. The global decision was used to divide the trainees into 2 contrasting groups and the OSATS or OSANTS scores were graphed per group to determine the performance standard. Parametric statistics were used to determine classification accuracy and concurrent standard acquisition, receiver operator characteristic (ROC) curves were used to delineate predictive factors. RESULTS: Thirty-six trainees were observed 101 times. The technical standard was an OSATS of 21.04/35.00 and the nontechnical standard an OSANTS of 22.49/35.00. Applying these standards, competent/noncompetent trainees could be discriminated in 94% of technical and 95% of nontechnical performances (P < 0.001). A 21% discordance between technically and nontechnically competent trainees was identified (P < 0.001). ROC analysis demonstrated case experience and trainee level were both able to predict achieving the standards with an area under the curve (AUC) between 0.83 and 0.96 (P < 0.001). CONCLUSIONS: The present study presents defensible standards for technical and nontechnical performance. Such standards are imperative to implementing summative assessments into surgical training.

Progressive learning in endoscopy simulation training improves clinical performance: a blinded randomized trial.

BACKGROUND AND AIMS: A structured comprehensive curriculum (SCC) that uses simulation-based training (SBT) can improve clinical colonoscopy performance. This curriculum may be enhanced through the application of progressive learning, a training strategy centered on incrementally challenging learners. We aimed to determine whether a progressive learning-based curriculum (PLC) would lead to superior clinical performance compared with an SCC. METHODS: This was a single-blinded randomized controlled trial conducted at a single academic center. Thirty-seven novice endoscopists were recruited and randomized to either a PLC (n = 18) or to an SCC (n = 19). The PLC comprised 6 hours of SBT, which progressed in complexity and difficulty. The SCC included 6 hours of SBT, with cases of random order of difficulty. Both groups received expert feedback and 4 hours of didactic teaching. Participants were assessed at baseline, immediately after training, and 4 to 6 weeks after training. The primary outcome was participants' performance during their first 2 clinical colonoscopies, as assessed by using the Joint Advisory Group Direct Observation of Procedural Skills assessment tool (JAG DOPS). Secondary outcomes were differences in endoscopic knowledge, technical and communication skills, and global performance in the simulated setting. RESULTS: The PLC group outperformed the SCC group during first and second clinical colonoscopies, measured by JAG DOPS (P < .001). Additionally, the PLC group had superior technical and communication skills and global performance in the simulated setting (P < .05). There were no differences between groups in endoscopic knowledge (P > .05). CONCLUSIONS: Our findings demonstrate the superiority of a PLC for endoscopic simulation, compared with an SCC. Challenging trainees progressively is a simple, theory-based approach to simulation whereby the performance of clinical colonoscopies can be improved. (Clinical trial registration number: NCT02000180.).

Practice does not always make perfect: need for selection curricula in modern surgical training.

BACKGROUND: It is hypothesized that not all surgical trainees are able to reach technical competence despite ongoing practice. The objectives of the study were to assess a trainees' ability to reach technical competence by assessing learning patterns of the acquisition of surgical skills. Furthermore, it aims to determine whether individuals' learning patterns were consistent across a range of open and laparoscopic tasks of variable difficulty. METHODS: Sixty-five preclinical medical students participated in a training curriculum with standardized feedback over forty repetitions of the following laparoscopic and open technical tasks: peg transfer (PT), circle cutting (CC), intracorporeal knot tie (IKT), one-handed tie, and simulated laparotomy closure. Data mining techniques were used to analyze the prospectively collected data and stratify the students into four learning clusters. Performance was compared between groups, and learning curve characteristics unique to trainees who have difficulty reaching technical competence were quantified. RESULTS: Top performers (22-35%) and high performers (32-42%) reached proficiency in all tasks. Moderate performers (25-37%) reached proficiency for all open tasks but not all laparoscopic tasks. Low performers (8-15%) failed to reach proficiency in four of five tasks including all laparoscopic tasks (PT 7.8%; CC 9.4%; IKT 15.6%). Participants in lower performance clusters demonstrated sustained performance disadvantage across tasks, with widely variable learning curves and no evidence of progression towards a plateau phase. CONCLUSIONS: Most students reached proficiency across a range of surgical tasks, but low-performing trainees failed to reach competence in laparoscopic tasks. With increasing use of laparoscopy in surgical practice, screening potential candidates to identify the lowest performers may be beneficial.

Structured Training to Improve Nontechnical Performance of Junior Surgical Residents in the Operating Room: A Randomized Controlled Trial.

OBJECTIVE: The objective of the study was to evaluate the effectiveness of structured training on junior trainees' nontechnical performance in an operating room (OR) environment. BACKGROUND: Nontechnical skills (NTS) have been identified as critical competencies of surgeons in the OR, and regulatory bodies have mandated their integration in postgraduate surgical curricula. Strong evidence supporting the effectiveness of curricular NTS training, however, is lacking. METHODS: Junior surgical residents were randomized to receive either conventional residency training or additional NTS training in a 2-month curriculum. Learning was assessed through a knowledge quiz and an attitudes survey. Nontechnical performance was evaluated by blinded assessment of standardized OR crisis simulations at baseline (BL) and posttraining (PT) using the Nontechnical Skills for Surgeons (NOTSS) and Objective Structured Assessment of Nontechnical Skills (OSANTS) rating systems. Results are reported as median (interquartile ranges). RESULTS: Of 23 participants, 22 completed BL and PT assessments. Groups were equal at BL. At PT, curriculum-trained residents (n = 11) scored higher than conventionally trained residents (n = 11) in knowledge [12 (11-13) vs 8 (6-10), P < 0.001] and attitudes [4.58 (4.37-4.73) vs 4.20 (4.00-4.50), P = 0.008] about NTS. In a simulated OR, nontechnical performance of curriculum-trained residents improved significantly from BL to PT [NOTSS: 10 (7-11) vs 13 (10-15), P = 0.012; OSANTS: 23 (17-28) vs 31 (25-33), P = 0.012] whereas conventionally trained residents did not improve [NOTSS: 10 (10-13) vs 11 (9-14), P = 1.00; OSANTS: 26 (24-32) vs 24 (23-32), P = 0.713]. CONCLUSIONS: The results demonstrate the effectiveness of structured curricular training in improving nontechnical performance in the first year of surgical residency, supporting routine implementation of nontechnical components in postgraduate surgical curricula.

Can We Predict Technical Aptitude?: A Systematic Review.

OBJECTIVE: To identify background characteristics and cognitive tests that may predict surgical trainees' future technical performance, and therefore be used to supplement existing surgical residency selection criteria. BACKGROUND: Assessment of technical skills is not commonly incorporated as part of the selection process for surgical trainees in North America. Emerging evidence, however, suggests that not all trainees are capable of reaching technical competence. Therefore, incorporating technical aptitude into selection processes may prove useful. METHODS: A systematic search was carried out of the MEDLINE, PsycINFO, and Embase online databases to identify all studies that assessed associations between surrogate markers of innate technical abilities in surgical trainees, and whether these abilities correlate with technical performance. The quality of each study was evaluated using the Grading of Recommendations, Assessment, Development, and Evaluation system. RESULTS: A total of 8035 records were identified. After screening by title, abstract, and full text, 52 studies were included. Very few surrogate markers were found to predict technical performance. Significant associations with technical performance were seen for 1 of 23 participant-reported surrogate markers, 2 of 25 visual spatial tests, and 2 of 19 dexterity tests. The assessment of trainee Basic Performance Resources predicted technical performance in 62% and 75% of participants. CONCLUSIONS: To date, no single test has been shown to reliably predict the technical performance of surgical trainees. Strategies that rely on assessing multiple innate abilities, their interaction, and their relationship with technical skill may ultimately be more likely to serve as reliable predictors of future surgical performance.