Nontechnical Skill Assessment of the Collective Surgical Team Using the Non-Technical Skills for Surgeons (NOTSS) System.

OBJECTIVE: To validate the Non-Technical Skills for Surgeons (NOTSS) system for assessment of the collective surgical teams' nontechnical skills after observing recordings of actual OR environment. BACKGROUND: The NOTSS system is a widely accepted tool to measure nontechnical skills of individual surgeons, and has mostly been used in the simulated setting. Surgical procedures are rarely performed by a single surgeon, but by a surgical team of attending surgeons, surgical assistants, and surgical trainees. Therefore, assessment of nontechnical skills may benefit from holistic assessment of the collective surgical teams. METHODS: Five trained participants assessed surgical team and attending surgeon using the NOTSS system after watching ten 20-minute long videos obtained from live OR. A set of reference ratings was provided by a multidisciplinary expert committee. We performed analyses to assess system sensitivity; examine inter-rater reliability of ratings; investigate concurrent construct validity; and assess feasibility and acceptability of using the NOTSS system to measure surgical team performance. RESULTS: There was adequate system sensitivity when comparing participants' and reference ratings. Inter-rater reliability among the participants' ratings was good except for decision-making category. The level of inter-rater reliability was similar when rating teams and attending surgeons. There was strong positive correlation between teams' and attending surgeons' NOTSS ratings at category [Pearson coefficient 0.86, 95% confidence interval (CI) 0.82-0.89] and element levels (0.83, 95% CI 0.80-0.85), demonstrating evidence of concurrent construct validity. The participants felt that the use of NOTSS system to measure teams' nontechnical skills was acceptable and feasible to a fair extent. CONCLUSION: The NOTSS system, although developed for assessment of individual surgeons, is a useful tool for observing and rating surgical teams.

Predicting venous thrombosis in patients undergoing elective splenectomy.

BACKGROUND: Venous thrombosis (VT) is an ongoing problem for patients undergoing elective splenectomy. There is limited data evaluating risk factors for VTs. An increase in platelet counts is commonly seen after splenectomy; however, there is a paucity of literature evaluating post-operative platelet counts as a risk factor for VTs in this patient cohort. The objective of this study was to determine the incidence of VT events and to use the platelet count as a predictor for VT development. METHODS: A retrospective review was undertaken at Brigham Women's Hospital, evaluating elective splenectomy patients between 1997 and 2018. Descriptive statistics were utilized to determine the incidence of VTs. Receiver operator characteristic (ROC) curves were utilized to identify platelet counts that could predict VTs. RESULTS: Five hundred and twenty splenectomies were included in the study of which 344 were completed in an open manner and 176 were done laparoscopically. The overall incidence of VT events was 6.7% (35/520), 6.1% (21/344) for open, and 8.0% (14/176) for laparoscopic approaches (p = 0.43). ROC curves demonstrated platelet counts to be a good predictor for the development of VTs with an area under the curve (AUC) of 0.77 (95% CI 0.69-0.86; p < 0.001) for all splenectomy patients, 0.70 (95% CI 0.59-0.81; p < 0.001) for those completed in an open manner, and 0.88 (95% CI 0.77-0.99; p < 0.001) for those done laparoscopically. The optimal platelet cutoff was found to be 545 for the overall splenectomy cohort, 457 for the open, and 659 for the laparoscopic cohorts. These platelet counts had a diagnostic accuracy that ranged from 61 to 86% and a negative predictive value (NPV) that ranged from 97 to 99%. CONCLUSION: These results suggest platelet cutoffs that predict VTs. This information can be used to individualize prophylactic strategies.

Feasibility of adapting the fundamentals of laparoscopic surgery trainer box to endoscopic skills training tool.

BACKGROUND: The fundamentals of laparoscopic surgery (FLS) training box is a validated tool, already accessible to surgical trainees to hone their laparoscopic skills. We aim to investigate the feasibility of adapting the FLS box for the practice and assessment of endoscopic skills. This would allow for a highly available, reusable, low-cost, mechanical trainer. METHODS: The design and development process was based on a user-centered design, which is a combination of the design thinking method and cognitive task analysis. The process comprises four phases: empathy, cognitive, prototyping/adaptation, and end user testing. The underlying idea was to utilize as many of the existing components of FLS training to maintain simplicity and cost effectiveness while allowing for the practice of clinically relevant endoscopic skills. A sample size of 18 participants was calculated to be sufficient to detect performance differences between experts and trainees using a two tailed t test with alpha set at 0.05, standard deviation of 5.5, and a power of 80%. RESULTS: Adaptation to the FLS box included two fundamental attachments: a front panel with an insertion point for an endoscope and a shaft which provides additional support and limits movement of the scope. The panel also allows for mounting of retroflexion tasks. Six endoscopic tasks inspired by FLS were designed (two of which utilize existing FLS components). Pilot testing with 38 participants showed high user's satisfaction and demonstrated that the trainer was robust and reliable. Task performance times was able to discriminate between trainees and experts for all six tasks. CONCLUSIONS: A mechanical, reusable, low-cost adaptation of the FLS training box for endoscopic skills is feasible and has high user satisfaction. Preliminary testing shows that the simulator is able to discriminate between trainees and experts. Following further validation, this adaptation may act as a supplement to the FES program.

The Effect of Technical Performance on Patient Outcomes in Surgery: A Systematic Review.

OBJECTIVE: Systematic review of the effect of intraoperative technical performance on patient outcomes. BACKGROUND: The operating room is a high-stakes, high-risk environment. As a result, the quality of surgical interventions affecting patient outcomes has been the subject of discussion and research for years. METHODS: MEDLINE, EMBASE, PsycINFO, and Cochrane databases were searched. All surgical specialties were eligible for inclusion. Data were reviewed in regards to the methods by which technical performance was measured, what patient outcomes were assessed, and how intraoperative technical performance affected patient outcomes. Quality of evidence was assessed using the Medical Education Research Study Quality Instrument (MERSQI). RESULTS: Of the 12,758 studies initially identified, 24 articles (7775 total participants) were ultimately included in this review. Seventeen studies assessed the performance of the faculty alone, 2 assessed both the faculty and trainees, 1 assessed trainees alone, and in 4 studies, the level of the operating surgeon was not specified. In 18 studies, a performance assessment tool was used. Patient outcomes were evaluated using intraoperative complications, short-term morbidity, long-term morbidity, short-term mortality, and long-term mortality. The average MERSQI score was 11.67 (range 9.5-14.5). Twenty-one studies demonstrated that superior technical performance was related to improved patient outcomes. CONCLUSIONS: The results of this systematic review demonstrated that superior technical performance positively affects patient outcomes. Despite this initial evidence, more robust research is needed to directly assess intraoperative technical performance and its effect on postoperative patient outcomes using meaningful assessment instruments and reliable processes.

Setting Performance Standards for Technical and Nontechnical Competence in General Surgery.

OBJECTIVES: The objectives of this study were to (1) create a technical and nontechnical performance standard for the laparoscopic cholecystectomy, (2) assess the classification accuracy and (3) credibility of these standards, (4) determine a trainees' ability to meet both standards concurrently, and (5) delineate factors that predict standard acquisition. BACKGROUND: Scores on performance assessments are difficult to interpret in the absence of established standards. METHODS: Trained raters observed General Surgery residents performing laparoscopic cholecystectomies using the Objective Structured Assessment of Technical Skill (OSATS) and the Objective Structured Assessment of Non-Technical Skills (OSANTS) instruments, while as also providing a global competent/noncompetent decision for each performance. The global decision was used to divide the trainees into 2 contrasting groups and the OSATS or OSANTS scores were graphed per group to determine the performance standard. Parametric statistics were used to determine classification accuracy and concurrent standard acquisition, receiver operator characteristic (ROC) curves were used to delineate predictive factors. RESULTS: Thirty-six trainees were observed 101 times. The technical standard was an OSATS of 21.04/35.00 and the nontechnical standard an OSANTS of 22.49/35.00. Applying these standards, competent/noncompetent trainees could be discriminated in 94% of technical and 95% of nontechnical performances (P < 0.001). A 21% discordance between technically and nontechnically competent trainees was identified (P < 0.001). ROC analysis demonstrated case experience and trainee level were both able to predict achieving the standards with an area under the curve (AUC) between 0.83 and 0.96 (P < 0.001). CONCLUSIONS: The present study presents defensible standards for technical and nontechnical performance. Such standards are imperative to implementing summative assessments into surgical training.

Practice does not always make perfect: need for selection curricula in modern surgical training.

BACKGROUND: It is hypothesized that not all surgical trainees are able to reach technical competence despite ongoing practice. The objectives of the study were to assess a trainees' ability to reach technical competence by assessing learning patterns of the acquisition of surgical skills. Furthermore, it aims to determine whether individuals' learning patterns were consistent across a range of open and laparoscopic tasks of variable difficulty. METHODS: Sixty-five preclinical medical students participated in a training curriculum with standardized feedback over forty repetitions of the following laparoscopic and open technical tasks: peg transfer (PT), circle cutting (CC), intracorporeal knot tie (IKT), one-handed tie, and simulated laparotomy closure. Data mining techniques were used to analyze the prospectively collected data and stratify the students into four learning clusters. Performance was compared between groups, and learning curve characteristics unique to trainees who have difficulty reaching technical competence were quantified. RESULTS: Top performers (22-35%) and high performers (32-42%) reached proficiency in all tasks. Moderate performers (25-37%) reached proficiency for all open tasks but not all laparoscopic tasks. Low performers (8-15%) failed to reach proficiency in four of five tasks including all laparoscopic tasks (PT 7.8%; CC 9.4%; IKT 15.6%). Participants in lower performance clusters demonstrated sustained performance disadvantage across tasks, with widely variable learning curves and no evidence of progression towards a plateau phase. CONCLUSIONS: Most students reached proficiency across a range of surgical tasks, but low-performing trainees failed to reach competence in laparoscopic tasks. With increasing use of laparoscopy in surgical practice, screening potential candidates to identify the lowest performers may be beneficial.

Can We Predict Technical Aptitude?: A Systematic Review.

OBJECTIVE: To identify background characteristics and cognitive tests that may predict surgical trainees' future technical performance, and therefore be used to supplement existing surgical residency selection criteria. BACKGROUND: Assessment of technical skills is not commonly incorporated as part of the selection process for surgical trainees in North America. Emerging evidence, however, suggests that not all trainees are capable of reaching technical competence. Therefore, incorporating technical aptitude into selection processes may prove useful. METHODS: A systematic search was carried out of the MEDLINE, PsycINFO, and Embase online databases to identify all studies that assessed associations between surrogate markers of innate technical abilities in surgical trainees, and whether these abilities correlate with technical performance. The quality of each study was evaluated using the Grading of Recommendations, Assessment, Development, and Evaluation system. RESULTS: A total of 8035 records were identified. After screening by title, abstract, and full text, 52 studies were included. Very few surrogate markers were found to predict technical performance. Significant associations with technical performance were seen for 1 of 23 participant-reported surrogate markers, 2 of 25 visual spatial tests, and 2 of 19 dexterity tests. The assessment of trainee Basic Performance Resources predicted technical performance in 62% and 75% of participants. CONCLUSIONS: To date, no single test has been shown to reliably predict the technical performance of surgical trainees. Strategies that rely on assessing multiple innate abilities, their interaction, and their relationship with technical skill may ultimately be more likely to serve as reliable predictors of future surgical performance.

Implementation of an Effective Strategy for Teaching Nontechnical Skills in the Operating Room: A Single-blinded Nonrandomized Trial.

OBJECTIVE: To evaluate the effectiveness of debriefing and feedback on intraoperative nontechnical performance as an instructional strategy in surgical training. BACKGROUND: Regulatory authorities for accreditation in North America have included nontechnical skills such as communication and teamwork in the competencies to be acquired by surgical residents before graduation. Concrete recommendations regarding the training and assessment of these competencies, however, are lacking. METHODS: Nonrandomized, single-blinded study using an interrupted time-series design. Eleven senior surgical residents were observed during routine cases in the operating room (OR) at baseline and post-training. The Non-Technical Skills for Surgeons (NOTSS) rating system was used. Observers were trained in NOTSS and blinded to the study purpose. Independent of the blinded observations, a surgeon educator conducted intraoperative observations, which served as the basis for the structured debriefing and feedback intervention. The intervention was administered to participants after a set of (blinded) baseline observations had been completed. Primary outcome was nontechnical performance in the OR as measured by the NOTSS system. Secondary outcome was perceived utility as assessed by a post-training questionnaire. RESULTS: Twelve senior trainees were recruited, 11 completed the study. Average NOTSS scores improved significantly from 3.2 (SD 0.37) at baseline to 3.5 (SD 0.43) post-training [t(10) = -2.55, P = 0.29]. All participants felt the intervention was useful, and the majority thought that debriefing and feedback on nontechnical skills should be integrated in surgical training. CONCLUSIONS: Debriefing and feedback in the OR may represent an effective strategy to ensure development of nontechnical skills in competency-based education.

Assessing Technical Competence in Surgical Trainees: A Systematic Review.

OBJECTIVE: To systematically examine the literature describing the methods by which technical competence is assessed in surgical trainees. BACKGROUND: The last decade has witnessed an evolution away from time-based surgical education. In response, governing bodies worldwide have implemented competency-based education paradigms. The definition of competence, however, remains elusive, and the impact of these education initiatives in terms of assessment methods remains unclear. METHODS: A systematic review examining the methods by which technical competence is assessed was conducted by searching MEDLINE, EMBASE, PsychINFO, and the Cochrane database of systematic reviews. Abstracts of retrieved studies were reviewed and those meeting inclusion criteria were selected for full review. Data were retrieved in a systematic manner, the validity and reliability of the assessment methods was evaluated, and quality was assessed using the Grading of Recommendations Assessment, Development and Evaluation classification. RESULTS: Of the 6814 studies identified, 85 studies involving 2369 surgical residents were included in this review. The methods used to assess technical competence were categorized into 5 groups; Likert scales (37), benchmarks (31), binary outcomes (11), novel tools (4), and surrogate outcomes (2). Their validity and reliability were mostly previously established. The overall Grading of Recommendations Assessment, Development and Evaluation for randomized controlled trials was high and low for the observational studies. CONCLUSIONS: The definition of technical competence continues to be debated within the medical literature. The methods used to evaluate technical competence predominantly include instruments that were originally created to assess technical skill. Very few studies identify standard setting approaches that differentiate competent versus noncompetent performers; subsequently, this has been identified as an area with great research potential.

Intrinsic directionality of migrating vascular smooth muscle cells is regulated by NAD(+) biosynthesis.

Cell migration is central to tissue repair and regeneration but must proceed with precise directionality to be productive. Directional migration requires external cues but also depends on the extent to which cells can inherently maintain their direction of crawling. We report that the NAD(+) biosynthetic enzyme, nicotinamide phosphoribosyltransferase (Nampt/PBEF/visfatin), mediates directionally persistent migration of vascular smooth muscle cells (SMCs). Time-lapse microscopy of human SMCs subjected to Nampt inhibition revealed chaotic motility whereas SMCs transduced with the Nampt gene displayed highly linear migration paths. Ordered motility conferred by Nampt was associated with downsizing of the lamellipodium, reduced lamellipodium wandering around the cell perimeter, and increased lamellipodial protrusion rates. These protrusive and polarity-stabilizing effects also enabled spreading SMCs to undergo bipolar elongation to an extent not typically observed in vitro. Nampt was found to localize to lamellipodia and fluorescence recovery of Nampt-eGFP after photobleaching revealed microtubule-dependent transport of Nampt to the leading edge. In addition, Nampt was found to associate with, and activate, Cdc42, and Nampt-driven directional persistence and lamellipodium anchoring required Cdc42. We conclude that high-fidelity SMC motility is coordinated by a Nampt-Cdc42 axis that yields protrusive but small and anchored lamellipodia. This novel, NAD(+)-synthesis-dependent control over motility may be crucial for efficient repair and regeneration of the vasculature, and possibly other tissues.

Temporal patterns for inguinal hernia recurrence operations after Shouldice Repair.

PURPOSE: Shouldice Repair for inguinal hernias results in a low recurrence rate; however, little is known about the risk factors for these relapses. In the present study, we reviewed all patient's undergoing a reoperation for recurrence after a primary Shouldice Repair. METHODS: Patients undergoing recurrent inguinal hernia repair from 2013 to 2017 were identified. The subgroup of patients with the first recurrence after a Shouldice Repair at this institution was selected and included. Data collection from the index and the reoperation surgery were performed, as well as statistical analysis. RESULTS: A total of 125 patients were included in the analysis. The mean age was 50.8 ± 13.9 (body mass index: 24.6 ± 2.6 kg/m2), 97% were male. The most common interval for a recurrence reoperation was in the first 5 years following the initial surgery (37.6% of patients; mode: 1 year; median: 7 years; mean: 13.7 ± 13.8 years). A temporal median pattern for recurrence reoperation according to age interval was noted (patients < 41 years old: 20 years; patients aged 41-60 years old: 10 years and patients > 61 years: 2 years)). CONCLUSION: Patients undergoing reoperations for recurrent inguinal hernias after a primary Shouldice Repair presented a pattern of temporal recurrence according to age. Older patients present with earlier recurrences than younger ones, an important consideration in the assessment of patients postoperatively.

Starting with 24-h levodopa carbidopa intestinal gel at initiation in a large cohort of advanced Parkinson's disease patients.

Continuous intra-jejunal infusion of levodopa-carbidopa intestinal gel (LCIG) is a long-term proven and effective treatment in advanced Parkinson's Disease (APD). Efficacy and safety of 16-h administration of LCIG has already been established. Additional benefits of 24-h LCIG administration have been reported in several case series and small clinical studies. The aim of this retrospective study was to compare the characteristics of patients who needed 24-h LCIG from the beginning of the DAT (device-aided treatment) with those who remained with the standard 16-h LCIG treatment and to identify particular motives if any. We initiated LCIG in 150 patients out of which in case of 62 patients (41,3%) due to unsatisfactory initial clinical benefits continuous 24-h LCIG was deemed necessary. Despite the subjective complaints and more severe clinical condition, at baseline evaluation we found statistically significant differences between 16-h LCIG cohort and 24-h LCIG cohort only in case of incidence of freezing (47% vs 65%, p = 0.03) and sudden off (32% vs 48%, p = 0.04). Wake hours/daytime LCIG does not always sufficiently improve the patient's quality of life in some patients due to persistent nighttime troublesome symptoms. Instead of labeling the patient as a non-responder, it is worth trying the 24-h LCIG dosage in a carefully selected group of patients, as there is currently no consensus on reliable criteria that serve the decision in these patients.

Levodopa-Entacapone-Carbidopa Intestinal Gel in the Treatment of Advanced Parkinson's Disease: A Single Center Real-World Experience.

Levodopa-entacapone-carbidopa intestinal gel infusion is a relatively new treatment option for advanced Parkinson's disease. We aimed to describe and analyze the characteristics of de novo levodopa-entacapone-carbidopa intestinal gel therapy in 20 consecutive patients with advanced Parkinson's disease. We assessed the profile of motor complications by evaluating the following: motor fluctuations, dyskinesias, and the freezing phenomenon at baseline (before the testing period) and before discharge. The treatment significantly reduced the duration of daily hours spent in off time compared with baseline pre-treatment values from a mean of 4.8 ± 0.9 h/day to a mean of 1.4 ± 0.5 h per day (p < 0.001). The duration and severity of peak-dose dyskinesia were also significantly reduced compared with baseline values. Out of the 10 patients who reported freezing, 8 did not present this complication at the pre-discharge assessment. Significant improvements were observed in Hoehn and Yahr scale scores in both the on and off states. The levodopa-entacapone-carbidopa intestinal gel therapy was well tolerated during the follow-up period immediately after initiation. Despite a relatively severe stage of the disease, all patients experienced a significant improvement in motor fluctuations, dyskinesias, and the freezing phenomenon.

Muscular groin hernias: an anatomical variation as a cause of recurrence.

PURPOSE: The objective of this study was to understand our center's experience with the rare entity of muscular groin hernias. Specifically, to determine this hernia's incidence and characteristics and provide valuable information to surgeons that can be utilized at the time of a hernia repair regardless of modality used. METHODS: This was a retrospective chart review between 2005 and 2019. Patients who had a primary or recurrent groin hernia operation for an atypically located hernia (other than direct, indirect, or femoral) were included. Descriptive statistics were utilized to present the hernia and patient data as median (interquartile range, range). RESULTS: There were 152 patients with 155 muscular hernias identified in primary operations and 41 patients with 41 muscular hernias identified in recurrent operations. In both primary and recurrent groups there were more males, and the males were on average younger with a higher body mass index (BMI) than females. Most muscular hernias were located lateral (76) or lateral/superior (33) to the internal ring with a median distance between 2 and 3 cm (1, 0.3-5; 2.8,2-5) from the internal ring. A concurrent ipsilateral inguinal hernia was found and repaired in 42.3% of left and 46.8% of right sided primary muscular hernia operations. A concurrent ipsilateral inguinal hernia was noted in 5% of left and 14.3% of right sided recurrent muscular operations. In left sided recurrent operations for a muscular hernia, 68.9% had a previous ipsilateral inguinal hernia repair and in right sided recurrent operations, 67.7% had a previous ipsilateral inguinal hernia repair. CONCLUSION: This study describes a previously unidentified groin muscular hernia in both primary and recurrent hernia operations as a potential cause of hernia recurrence and identifies muscular hernia locations. This information can be utilized at the time of surgery to identify and subsequently repair these defects, leading to improved patient and hernia outcomes.

Spigelian hernias: A high volume institutional review.

Purpose: The objective of this study was to evaluate a high-volume hernia center's experience with primary Spigelian hernias. Introduction: Spigelian hernias are rare entities presenting a diagnostic dilemma for clinicians that often lead to a delay in treatment or need for emergency surgery. Given this low incidence true patient and hernia characteristics are unknown. Methods: This was a retrospective chart review evaluating patients who underwent a Spigelian hernia repair between 2005 and 2019. Descriptive statistics were utilized. Results: The incidence of Spigelian hernias at our institution was 0.14 %. There were 141 patients with 143 spigelian hernias (two patients had bilateral hernias) repaired at our institution during the study period, the majority of these were in male patients, left sided and repaired under conscious sedation with or without the use of mesh. Conclusion: This study adds a large dataset to the paucity of literature that exists in the context of Spigelian hernias and alerts the clinician that these hernias are not exclusive to the older female population. Furthermore, this study provides the need for a high index of suspicion in patients presenting with vague abdominal pain who may not fit the typical Spigelian hernia mold.

The Shouldice Method: an expert's consensus.

BACKGROUND: Hernia repairs are a commonly performed surgical procedure and the Shouldice Repair of inguinal hernias has been well described before in terms of its outcomes. However, the purpose of this paper was to understand what experts from Shouldice Hospital consider to be the essential steps to the Shouldice Method. METHODS: Utilizing a Delphi Methodology, surgeons at Shouldice Hospital who are considered content experts, provided their answers on what is essential to the Shouldice Method. The median, interquartile range, and percent agreement from the responses as well as the Delphi's overall Cronbach's Alpha were determined. All Delphi items were ranked on a five-point Likert scale and consensus was reached when Cronbach's Alpha was ≥ 0.8. The items from the survey that ranked as 5-completely agree or 4-partially agree by ≥ 80% of participants on the five-point Likert scale were included in the final framework. RESULTS: The final consensus for the Shouldice Method included 39 items with 7 overarching steps: preoperative preparation of the patient, anesthetic component, incision and nerve identification, treatment of the cremasteric muscles, hernia identification and treatment, reconstruction of the posterior wall, and post-operative management of patients. CONCLUSION: The results of this consensus provide a step-by-step approach to the Shouldice Method, as well as information that is timely and can be utilized by surgeons incorporating non-mesh hernia repairs into their practice.

Educational Effectiveness of Telementoring as a Continuing Professional Development Intervention for Surgeons in Practice: A Systematic Review.

Objective: We performed a systematic review to determine the educational effectiveness of telementoring as a continuing professional development (CPD) intervention for surgeons in practice. Background: Surgeons can mentor their peers in remote locations using videoconferencing communication, referred to as telementoring. Methods: We searched MEDLINE and EMBASE and included studies assessing the educational effectiveness of telementoring interventions used by surgeons in practice. We excluded studies involving only trainees and those not evaluating educational effectiveness. Two reviewers independently screened, extracted data, and assessed study quality using the Medical Education Research Study Quality Instrument (MERSQI; maximum score 18). Educational outcomes were categorized using Moore's Outcomes Framework. Results: We retrieved a total of 1351 records, and 252 studies were selected for full-text review. Twenty-eight studies were included with 1 randomized controlled trial, 19 cohort studies, 5 qualitative studies, and 3 case studies, totaling 178 surgeons and 499 cases. The average MERSQI score was 10.21 ± 2.2 out of 18. Educational outcomes included surgeons' satisfaction with telementoring interventions (Moore's Level 2) in 12 studies, improvement in surgeons' procedural knowledge (Level 3b) in 3 studies, improvements in surgeons' procedural competence in an educational setting (Level 4) in 4 studies, performance in a workplace-based setting (Level 5) in 23 studies, and patient outcomes (Level 6) in 3 studies. No studies reported community health outcomes (Level 7). Conclusions: Moderate-level evidence demonstrates the use of telementoring as effective in changing surgeons' knowledge and competence in both educational and workplace-based settings. Its use is also associated with changes in patient outcomes.

Developing the Blueprint for a General Surgery Technical Skills Certification Examination: A Validation Study.

INTRODUCTION: There is a recognized need to develop high-stakes technical skills assessments for decisions of certification and resident promotion. High-stakes examinations requires a rigorous approach in accruing validity evidence throughout the developmental process. One of the first steps in development is the creation of a blueprint which outlines the potential content of examination. The purpose of this validation study was to develop an examination blueprint for a Canadian General Surgery assessment of technical skill certifying examination. METHODS: A Delphi methodology was used to gain consensus amongst Canadian General Surgery program directors as to the content (tasks or procedures) that could be included in a certifying Canadian General Surgery examination. Consensus was defined a priori as a Cronbach's α ≥ 0.70. All procedures or tasks reaching a positive consensus (defined as ≥80% of program directors rated items as ≥4 on the 5-point Likert scale) were then included in the final examination blueprint. RESULTS: Two Delphi rounds were needed to reach consensus. Of the 17 General Surgery Program directors across the country, 14 (82.4%) and 10 (58.8%) program directors responded to the first and second round, respectively. A total of 59 items and procedures reached positive consensus and were included in the final examination blueprint. CONCLUSIONS: The present study has outlined the development of an examination blueprint for a General Surgery certifying examination using a consensus-based methodology. This validation study will serve as the foundational work from which simulated model will be developed, pilot tested and evaluated.

Assessing Technical Performance and Determining the Learning Curve in Cleft Palate Surgery Using a High-Fidelity Cleft Palate Simulator.

BACKGROUND: This study assessed technical performance in cleft palate repair using a newly developed assessment tool and high-fidelity cleft palate simulator through a longitudinal simulation training exercise. METHODS: Three residents performed five and one resident performed nine consecutive endoscopically recorded cleft palate repairs using a cleft palate simulator. Two fellows in pediatric plastic surgery and two expert cleft surgeons also performed recorded simulated repairs. The Cleft Palate Objective Structured Assessment of Technical Skill (CLOSATS) and end-product scales were developed to assess performance. Two blinded cleft surgeons assessed the recordings and the final repairs using the CLOSATS, end-product scale, and a previously developed global rating scale. RESULTS: The average procedure-specific (CLOSATS), global rating, and end-product scores increased logarithmically after each successive simulation session for the residents. Reliability of the CLOSATS (average item intraclass correlation coefficient (ICC), 0.85 ± 0.093) and global ratings (average item ICC, 0.91 ± 0.02) among the raters was high. Reliability of the end-product assessments was lower (average item ICC, 0.66 ± 0.15). Standard setting linear regression using an overall cutoff score of 7 of 10 corresponded to a pass score for the CLOSATS and the global score of 44 (maximum, 60) and 23 (maximum, 30), respectively. Using logarithmic best-fit curves, 6.3 simulation sessions are required to reach the minimum standard. CONCLUSIONS: A high-fidelity cleft palate simulator has been developed that improves technical performance in cleft palate repair. The simulator and technical assessment scores can be used to determine performance before operating on patients.