Key Issues in Surgical Residency Education: Recommendations of the Blue Ribbon II Committee Residency Education Subcommittee.

In September 2022, a summit was convened by the American Board of Surgery (ABS) to discuss competency-based reform in surgical education. A key output of that summit was the recommendation that the prior work of the Blue Ribbon I Committee convened 20 years earlier be revived. With leadership from the American College of Surgeons (ACS) and the American Surgical Association (ASA) , the Blue Ribbon Committee (BRC) II was subsequently convened. This paper describes the output of the Residency Education Subcommittee of the BRC II Committee. The Subcommittee organized its work around prioritized themes including curriculum, assessment, and transition to practice. Top recommendations, time-based action steps, potential barriers, and required resources were detailed and vetted through group discussion, broader Committee review and critique, and subsequent refinement. Primary concluding emphases included transitioning to a competency-based training model, facilitating dynamically capable curricular reform emphasizing the digital transformation of surgical care, using predictive analytic assessment strategies to optimize training effectiveness and efficiency, and creating mentorship strategies to govern the transition from training to independent practice in an outcomes-accountable fashion. It was recognized that coordinated efforts across existing organizational structures will be required, informed by dataset integration strategies that meaningfully measure educational and related patient outcomes.

A cross sectional survey on burnout and career satisfaction in surgeons: a joint work from the SAGES quality, outcomes, and safety and reimaging the practice of surgery burnout task forces.

INTRODUCTION: The growth of surgeon burnout is of significant concern. As we work to reimagine the practice of surgery, an accurate understanding of the extent of surgeon burnout is essential. Our goal was to define the current prevalence of burnout and quality of life (QOL) among SAGES surgeons. METHODOLOGY: An electronic survey was administered to SAGES members to establish a current baseline for QOL, burnout, depression, and career satisfaction. To assess outcomes, we utilized the validated Maslach Burnout Inventory for Medical Personnel, the Medical Outcomes Study Short Form, and the Primary Care Evaluation of Mental Disorders. All scoring followed validated norm-based methods. RESULTS: Of 4194 active members, 604 responded (14.40%). 69% met burnout threshold, with high levels of emotional exhaustion and depersonalization, and low personal accomplishment. 81% reported "being at the end of their rope", 74% felt emotionally drained, and 65% felt used up daily. Nearly all maintained caring about what happened to their patients (96%), easily understanding how their patients feel (84.3%) and being capable of dealing effectively with their patient's problems (87.6%). However, respondents never, rarely, or occasionally felt energetic (77.5%) or experienced a sense of professional accomplishment (57.8%). The overall QOL score was 69/100, with lower Mental than Physical scores (62.69 (SD 10.20) vs.77.27 (SD 22.24)). More than half of respondents met depression criteria. While 77% supported they would become a physician again, less than half would choose surgery again or recommend surgery to their children. Furthermore, less than a third felt work allowed sufficient time for their personal lives. CONCLUSIONS: Participating SAGES surgeons reported alarmingly high rates of burnout and depression. Despite experiencing emotional exhaustion and depersonalization, they maintained a strong commitment to patient care. These findings likely reflect the broader state of surgeons, underscoring the urgent need for action to address this critical issue.

Implementation of Entrustable Professional Activities in General Surgery: Results of a National Pilot Study.

OBJECTIVE: The ongoing complexity of general surgery training has led to an increased focus on ensuring the competence of graduating residents. Entrustable professional activities (EPAs) are units of professional practice that provide an assessment framework to drive competency-based education. The American Board of Surgery convened a group from the American College of Surgeons, Accreditation Council for Graduate Medical Education (ACGME) Surgery Review Committee, and Association of Program Directors in Surgery to develop and implement EPAs in a pilot group of residency programs across the country. The objective of this pilot study was to determine the feasibility and utility of EPAs in general surgery resident training. METHODS: 5 EPAs were chosen based on the most common procedures reported in ACGME case logs and by practicing general surgeons (right lower quadrant pain, biliary disease, inguinal hernia), along with common activities covering additional ACGME milestones (performing a consult, care of a trauma patient). Levels of entrustment assigned (1 to 5) were observation only, direct supervision, indirect supervision, unsupervised, and teaching others. Participating in site recruitment and faculty development occurred from 2017 to 2018. EPA implementation at individual residency programs began on July 1, 2018, and was completed on June 30, 2020. Each site was assigned 2 EPAs to implement and collected EPA microassessments on residents for those EPAs. The site clinical competency committees (CCC) used these microassessments to make summative entrustment decisions. Data submitted to the independent deidentified data repository every 6 months included the number of microassessments collected per resident per EPA and CCC summative entrustment decisions. RESULTS: Twenty-eight sites were selected to participate in the program and represented geographic and size variability, community, and university-based programs. Over the course of the 2-year pilot programs reported on 14 to 180 residents. Overall, 6,272 formative microassessments were collected (range, 0 to 1144 per site). Each resident had between 0 and 184 microassessments. The mean number of microassessments per resident was 5.6 (SD = 13.4) with a median of 1 [interquartile range (IQR) = 6]. There were 1,763 summative entrustment ratings assigned to 497 unique residents. The average number of observations for entrustment was 3.24 (SD 3.61) with a median of 2 (IQR 3). In general, PGY1 residents were entrusted at the level of direct supervision and PGY5 residents were entrusted at unsupervised practice or teaching others. For each EPA other than the consult EPA, the degree of entrustment reported by the CCC increased by resident level. CONCLUSIONS: These data provide evidence that widespread implementation of EPAs across general surgery programs is possible, but variable. They provide meaningful data that graduating chief residents are entrusted by their faculty to perform without supervision for several common general surgical procedures and highlight areas to target for the successful widespread implementation of EPAs.

Surgical Education.

This paper summarizes the proceedings of the joint European Surgical Association ESA/American Surgical Association symposium on Surgical Education that took place in Bordeaux, France, as part of the celebrations for 30 years of ESA scientific meetings. Three presentations on the use of quantitative metrics to understand technical decisions, coaching during training and beyond, and entrustable professional activities were presented by American Surgical Association members and discussed by ESA members in a symposium attended by members of both associations.

The Quality of Operative Performance Narrative Feedback: A Retrospective Data Comparison Between End of Rotation Evaluations and Workplace-based Assessments.

OBJECTIVE: To describe the quality of operative performance feedback using evaluation tools commonly used by general surgery residency training programs. SUMMARY OF BACKGROUND DATA: The majority of surgical training programs administer an evaluation through which faculty members may rate and comment on trainee operative performance at the end of the rotation (EOR). Many programs have also implemented the system for improving and measuring procedural learning (SIMPL), a workplace-based assessment tool with which faculty can rate and comment on a trainee's operative performance immediately after a case. It is unknown how the quality of narrative operative performance feedback delivered with these tools compares. METHODS: The authors collected EOR evaluations and SIMPL narrative comments on trainees' operative performance from 3 university-based surgery training programs during the 2016-2017 academic year. Two surgeon raters categorized comments relating to operative skills as being specific or general and as encouraging and/or corrective. Comments were then classified as effective, mediocre, ineffective, or irrelevant. The frequencies with which comments were rated as effective were compared using Chi-square analysis. RESULTS: The authors analyzed a total of 600 comments. 10.7% of EOR and 58.3% of SIMPL operative performance evaluation comments were deemed effective (P < 0.0001). CONCLUSIONS: Evaluators give significantly higher quality operative performance feedback when using workplace-based assessment tools rather than EOR evaluations.

Association of socioeconomic status with 30- and 90-day readmission following open and laparoscopic hernia repair: a nationwide readmissions database analysis.

BACKGROUND: Socioeconomic disparities have been associated with outcomes in many medical conditions. The association of socioeconomic status (SES) with readmissions after ventral and inguinal hernia repair has not been well studied on a national level. This study aims to evaluate the association of SES with readmission as a significant outcome in patients undergoing ventral and inguinal hernia repair. METHODS: A retrospective cohort study was performed evaluating patients undergoing ventral hernia and inguinal hernia repair with 1:1 propensity score matching using the Nationwide Readmissions Database (2016-2017). Both 30- and 90-day readmissions were examined. After matching, a multivariate logistic regression analysis was performed using confounding variables including hospital setting, comorbidities, urgency of repair, sociodemographic status, and payer. Likelihood of readmission was reported in odds ratio form. RESULTS: Readmission rates were 11.56% (24,323 out of 210,381) and 17.94% (30,893 out of 172,210) for 30- and 90-day readmissions, respectively. Patients with Medicaid and in the lower income quartile were more likely to present in an emergent fashion for hernia repair. After matching, a multivariate logistic regression analysis showed socioeconomic status (OR 1.250 and 1.229) was a statistically significant independent predictor of readmission at 30 and 90 days, respectively. Inversely, factors associated with the least likely chance of readmission were a laparoscopic approach (OR 0.646 and 0.641), elective admission (OR 0.824 and 0.779), and care in a teaching hospital (OR 0.784 and 0.798). CONCLUSION: SES is an independent predictor of readmission at 30 and 90 days following open and laparoscopic ventral and inguinal hernia repair. Patients with a lower socioeconomic status were more likely to undergo hernia repair in the emergent setting. Efforts toward mitigating SES disparities by potentially promoting MIS techniques, enhancing access to elective cases, and systematic approaches to perioperative care for this disadvantaged population can potentially enhance overall hernia outcomes.

A Proposed Blueprint for Operative Performance Training, Assessment, and Certification.

OBJECTIVE: The aim of this study was to propose an evidence-based blueprint for training, assessment, and certification of operative performance for surgical trainees. SUMMARY BACKGROUND DATA: Operative skill is a critical aspect of surgical performance. High-quality assessment of operative skill therefore has profound implications for training, accreditation, certification, and the public trust of the profession. Current methods of operative skill assessment for surgeons rely heavily on global assessment strategies across a very broad domain of procedures. There is no mechanism to assure technical competence for individual procedures. The science and scalability of operative skill assessment has progressed significantly in recent decades, and can inform a much more meaningful strategy for competency-based assessment of operative skill than has been previously achieved. METHODS: The present article reviews the current status and science of operative skill assessment and proposes a template for competency-based assessment which could be used to update training, accreditation, and certification processes. The proposal is made in reference to general surgery but is more generally applicable to other procedural specialties. RESULTS: Streamlined, routine assessment of every procedure performed by surgical trainees is feasible and would enable a more competency-based educational paradigm. In light of the constraints imposed by both clinical volume and assessment bias, trainees should be expected to become proficient and be measured against a mastery learning standard only for the most important and highest-frequency procedures. For less frequently observed procedures, performance can be compared to a norm-referenced standard and, to provide an overall trajectory of performance, analyzed in aggregate. Key factors in implementing this approach are the number of evaluations, the number of raters, the timeliness of evaluation, and evaluation items. CONCLUSIONS: A competency-based operative skill assessment can be incorporated into surgical training, assessment, and certification. The time has come to develop a systematic approach to this issue as a means of demonstrating professional standards worthy of the public trust.

Determinants of 90-day readmission following ventral hernia repair with and without myocutaneous flap reconstruction: a National Readmissions Database analysis.

BACKGROUND: Readmission status is an important clinical component of healthcare outcomes. 90-day readmission following complex open ventral hernia repair has not been well studied with national level data. This study aims to compare readmission rates for patients undergoing standard vs. complex (myocutaneous flap-based) ventral hernia repair. We hypothesize that complexity of reconstruction will be an independent predictor of readmission after ventral hernia repair. METHODS: A retrospective cohort study was performed with 1:1 matching of hernia repair type using the National Readmissions Database. Patients were selected using ICD-9 codes corresponding to ventral hernia repair with or without myocutaneous flap. 90-day readmissions were determined on patients within the first through third quarters of each year. After matching, a multivariable logistic regression analysis was performed using confounding variables including hospital setting, comorbidities, type of repair, urgency of repair, sociodemographic status, and payer. Likelihood of 90-day readmission was calculated from odds ratios. RESULTS: Readmission rates were 19.1% (38,313 out of 200,266) and 22.5% (692 out of 3075) at 90-day for standard ventral hernia repair and complex ventral hernia repair, respectively. 3116 standard ventral hernia repair patients were matched with 3074 complex ventral hernia repair patients. After matching there was a significantly increased readmission rate for repairs involving myocutaneous flaps, with odds ratio (OR) 1.30 (95% CI 1.22-1.60). Payer status (OR 1.82; 95% CI 1.21-2.74), teaching hospital status (OR 1.42; 95% CI 1.23-1.64) and income quartile (OR 1.35; CI 1.10-1.65) were independent predictors of readmission. CONCLUSIONS: Patients undergoing myocutaneous flap-based reconstruction have higher readmission rates than those undergoing less complex ventral hernia repair. Socioeconomic disparity as reflected in payer status is a particularly strong predictor of readmission. The data support the concept that focused efforts are needed to optimize patient outcomes for patients requiring more complex repair, including socioeconomically disadvantaged patient populations.

The COVID-19 reset: lessons from the pandemic on Burnout and the Practice of Surgery.

BACKGROUND: Burnout among physicians is an increasing concern, and surgeons are not immune to this threat. The ongoing COVID-19 pandemic has caused dramatic changes to surgeon workflow, often leading to redeployment to other clinical areas, slowdown and shutdown of elective surgery practices, and an uncertain future of surgical practice in the post-pandemic setting. Paradoxically, for many surgeons who had to prepare for but not immediately care for a major surge, the crisis did allow for reflective opportunities and a resetting of priorities that could serve to mitigate chronic patterns contributory to Burnout. METHODS: SAGES Reimagining the Practice of Surgery task force convened a webinar to discuss lessons learned from the COVID pandemic that may address burnout. RESULTS: Burnout is multifactorial and may vary in cause among different generation/experience groups. Those that report burnout symptoms often complain of lacking purpose or meaning in their work. Although many mechanisms to address Burnout are from a defensive standpoint-including coping mechanisms, problem solving, and identification of a physician having wellness difficulties-offensive mechanisms such as pursuing purpose and meaning and finding joy in one's work can serve as reset points that promote thriving and fulfillment. Understanding what motivates physicians will help physician leaders to develop and sustain effective teams. Reinvigorating the surgical workforce around themes of meaning and joy in the service rendered via our surgical skills may diminish Burnout through generative and aspirational strategies, as opposed to merely reactive ones. Fostering an educational environment free of discriminatory or demeaning behavior may produce a new workforce conducive to enhanced and resilient wellbeing at the start of careers. CONCLUSION: Surgeon wellness and self-care must be considered an important factor in the future of all healthcare delivery systems, a need reaffirmed by the COVID-19 pandemic.

SAGES primer for taking care of yourself during and after the COVID-19 crisis.

COVID-19 is a pandemic which has affected almost every aspect of our life since starting globally in November 2019. Given the rapidity of spread and inadequate time to prepare for record numbers of sick patients, our surgical community faces an unforeseen challenge. SAGES is committed to the protection and care of patients, their surgeons and staff, and all who are served by the medical community at large. This includes physical health, mental health, and well-being of all involved. The fear of the unknown ahead can be paralyzing. International news media have chronicled the unthinkable situations that physicians and other health care providers have been thrust into as a result of the COVID-19 pandemic. These situations include making life or death decisions for patients and their families regarding use of limited health care resources. It includes caring for patients with quickly deteriorating conditions and limited treatments available. Until recently, these situations seemed far from home, and now they are in our own hospitals. As the pandemic broadened its reach, the reality that we as surgeons may be joining the front line is real. It may be happening to you now; it may be on the horizon in the coming weeks. In this context, SAGES put together this document addressing concerns on clinician stressors in these times of uncertainty. We chose to focus on the emotional toll of the situation on the clinician, protecting vulnerable persons, reckoning with social isolation, and promoting wellness during this crisis. At the same time, the last part of this document deals with the "light at the end of the tunnel," discussing potential opportunities, lessons learned, and the positives that can come out of this crisis.

Survival Implications of Increased Utilization of Local Excision for cT1N0 Esophageal Cancer.

OBJECTIVE: We hypothesized that patients with cT1N0 esophageal cancer undergoing local excision would have lower survival compared with esophagectomy due to potential discordant staging. BACKGROUND: Local excision has become an attractive alternative for management of early esophageal cancer, avoiding the morbidity of esophagectomy. It is uncertain if occult nodal metastasis impacts survival. METHODS: An observational study was conducted using the National Cancer Database (1998-2012) for patients with clinical T1N0 esophageal cancer who underwent local excision (n = 1625) or esophagectomy (n = 3255). RESULTS: The proportion of patients undergoing local excision increased from 12% in 1998 to 50% in 2012 (P < 0.001). After esophagectomy, 61% of cT1N0 cancers had concordant clinical and pathological staging, with 5.2% having positive nodal disease; 37% were staged concordant after local excision, with excess missing data (60%). Ninety-day mortality was 7.4% after esophagectomy compared with 2.8% after local excision (P < 0.001). While no significant difference was seen in unadjusted survival, adjusted Cox regression analysis indicated worse survival after esophagectomy compared with local excision for all cases [hazard ratio (HR) 1.57, 95% confidence interval (CI) 1.27-1.95] and for patients with concordant staging (HR 1.68, 95% CI 1.23-2.28). CONCLUSIONS: Local excision for cT1N0 esophageal cancer has increased over time. Contrary to our hypothesis, despite incomplete nodal staging, patients undergoing local excision have favorable survival, particularly in the adenocarcinoma subgroup. This may reflect early differences in mortality due to differences in procedure-related complications and/or selection bias. As this study has limited power to compare outcomes between T1a and T1b cancers, further analysis is warranted.

National Cluster-Randomized Trial of Duty-Hour Flexibility in Surgical Training.

BACKGROUND: Concerns persist regarding the effect of current surgical resident duty-hour policies on patient outcomes, resident education, and resident well-being. METHODS: We conducted a national, cluster-randomized, pragmatic, noninferiority trial involving 117 general surgery residency programs in the United States (2014-2015 academic year). Programs were randomly assigned to current Accreditation Council for Graduate Medical Education (ACGME) duty-hour policies (standard-policy group) or more flexible policies that waived rules on maximum shift lengths and time off between shifts (flexible-policy group). Outcomes included the 30-day rate of postoperative death or serious complications (primary outcome), other postoperative complications, and resident perceptions and satisfaction regarding their well-being, education, and patient care. RESULTS: In an analysis of data from 138,691 patients, flexible, less-restrictive duty-hour policies were not associated with an increased rate of death or serious complications (9.1% in the flexible-policy group and 9.0% in the standard-policy group, P=0.92; unadjusted odds ratio for the flexible-policy group, 0.96; 92% confidence interval, 0.87 to 1.06; P=0.44; noninferiority criteria satisfied) or of any secondary postoperative outcomes studied. Among 4330 residents, those in programs assigned to flexible policies did not report significantly greater dissatisfaction with overall education quality (11.0% in the flexible-policy group and 10.7% in the standard-policy group, P=0.86) or well-being (14.9% and 12.0%, respectively; P=0.10). Residents under flexible policies were less likely than those under standard policies to perceive negative effects of duty-hour policies on multiple aspects of patient safety, continuity of care, professionalism, and resident education but were more likely to perceive negative effects on personal activities. There were no significant differences between study groups in resident-reported perception of the effect of fatigue on personal or patient safety. Residents in the flexible-policy group were less likely than those in the standard-policy group to report leaving during an operation (7.0% vs. 13.2%, P<0.001) or handing off active patient issues (32.0% vs. 46.3%, P<0.001). CONCLUSIONS: As compared with standard duty-hour policies, flexible, less-restrictive duty-hour policies for surgical residents were associated with noninferior patient outcomes and no significant difference in residents' satisfaction with overall well-being and education quality. (FIRST ClinicalTrials.gov number, NCT02050789.).

Surgeon-performed endoscopic retrograde cholangiopancreatography. Outcomes of 2392 procedures at two tertiary care centers.

BACKGROUND: Endoscopic retrograde cholangiopancreatography (ERCP) is a common procedure that, in the United States, is traditionally performed by gastroenterologists. We hypothesized that when performed by well-trained surgeons, ERCP can be performed safely and effectively. The objectives of the study were to assess the rate of successful cannulation of the duct of interest and to assess the 30-day complication and mortality rates. METHODS: We retrospectively reviewed the charts of 1858 patients who underwent 2392 ERCP procedures performed by five surgeons between August 2003 and June 2016 in two centers. Demographic and historical data, indications, procedure-related data and 30-day complication and mortality data were collected and analyzed. RESULTS: The mean age was 53.4 (range 7-102) years and 1046 (56.3%) were female. 1430 (59.8%) of ERCP procedures involved a surgical endoscopy fellow. The most common indication was suspected or established uncomplicated common bile duct stones (n = 1470, 61.5%), followed by management of an existing biliary or pancreatic stent (n = 370, 15.5%) and acute biliary pancreatitis (n = 173, 7.2%). A therapeutic intervention was performed in 1564 (65.4%), a standard sphincterotomy in 1244 (52.0%), stent placement in 705 (29.5%) and stone removal in 638 (26.7%). When cannulation was attempted, the rate of successful cannulation was 94.1%. When cannulation was attempted during the patient's first ERCP the cannulation rate was 92.4%. 94 complications occurred (5.4%); the most common complication was post-ERCP pancreatitis in 75 (4.2%), significant gastrointestinal bleeding in 7 (0.4%), ascending cholangitis in 11 (0.6%) and perforation in 1 (0.05%). 11 mortalities occurred (0.5%) but none of which were ERCP-related. CONCLUSION: When performed by well-trained surgical endoscopists, ERCP is associated with high success rate and acceptable complication rates consistent with previously published reports and in line with societal guidelines.

Validity evidence for a new portable, lower-cost platform for the fundamentals of endoscopic surgery skills test.

BACKGROUND: The fundamentals of endoscopic surgery (FES) examination measures the knowledge and skills required to perform safe flexible endoscopy. A potential limitation of the FES skills test is the size and cost of the simulator on which it was developed (GI Mentor II virtual reality endoscopy simulator; Simbionix LTD, Israel). A more compact and lower-cost alternative (GI Mentor Express) was developed to address this issue. The purpose of this study was to obtain evidence for the validity of scores obtained on the Express platform, so that it can be used for testing. STUDY DESIGN: General surgery residents at various levels of training and practicing endoscopists at five institutions participated. Each completed the five FES tasks on both simulator platforms in random order, with 3-14 days between tests. Scores were calculated using the same standardized computer-generated algorithm and compared using Pearson's correlation coefficient. RESULTS: There were 58 participants (mean age 32; 76% male) with a broad range of endoscopic experience. The mean (95% confidence interval) FES scores were 72 (67:77) on the GI Mentor II and 66 (60:71) on the Express. The correlation between scores on the two platforms was 0.86 (0.77:0.91; p < 0.0001). CONCLUSION: There is a high correlation between FES manual skills scores measured on the original platform and the new Express, providing evidence to support the use of the GI Mentor Express for FES testing.

Ethical considerations regarding the implementation of new technologies and techniques in surgery.

Ethical considerations relevant to the implementation of new surgical technologies and techniques are explored and discussed in practical terms in this statement, including (1) How is the safety of a new technology or technique ensured?; (2) What are the timing and process by which a new technology or technique is implemented at a hospital?; (3) How are patients informed before undergoing a new technology or technique?; (4) How are surgeons trained and credentialed in a new technology or technique?; (5) How are the outcomes of a new technology or technique tracked and evaluated?; and (6) How are the responsibilities to individual patients and society at large balanced? The following discussion is presented with the intent to encourage thought and dialogue about ethical considerations relevant to the implementation of new technologies and new techniques in surgery.

Fundamentals of Endoscopic Surgery cognitive examination: development and validity evidence.

BACKGROUND: Flexible endoscopy is an integral part of surgical care. Exposure to endoscopic procedures varies greatly in surgical training. The Society of American Gastrointestinal and Endoscopic Surgeons has developed the Fundamentals of Endoscopic Surgery (FES), which serves to teach and assess the fundamental knowledge and skills required to practice flexible endoscopy of the gastrointestinal tract. This report describes the validity evidence in the development of the FES cognitive examination. METHODS: Core areas in the practice of gastrointestinal endoscopy were identified through facilitated expert focus groups to establish validity evidence for the test content. Test items then were developed based on the content areas. Prospective enrollment of participants at various levels of training and experience was used for beta testing. Two FES cognitive test versions then were developed based on beta testing data. The Angoff and contrasting group methods were used to determine the passing score. Validity evidence was established through correlation of experience level with examination score. RESULTS: A total of 220 test items were developed in accordance with the defined test blueprint and formulated into two versions of 120 questions each. The versions were administered randomly to 363 participants. The correlation between test scores and training level was high (r = 0.69), with similar results noted for contrasting groups based on endoscopic rotation and endoscopic procedural experience. Items then were selected for two test forms of 75 items each, and a passing score was established. CONCLUSIONS: The FES cognitive examination is the first test with validity evidence to assess the basic knowledge needed to perform flexible endoscopy. Combined with the hands-on skills examination, this assessment tool is a key component for FES certification.

Fundamentals of endoscopic surgery: creation and validation of the hands-on test.

BACKGROUND: The Fundamentals of Endoscopic Surgery™ (FES) program consists of online materials and didactic and skills-based tests. All components were designed to measure the skills and knowledge required to perform safe flexible endoscopy. The purpose of this multicenter study was to evaluate the reliability and validity of the hands-on component of the FES examination, and to establish the pass score. METHODS: Expert endoscopists identified the critical skill set required for flexible endoscopy. They were then modeled in a virtual reality simulator (GI Mentor™ II, Simbionix™ Ltd., Airport City, Israel) to create five tasks and metrics. Scores were designed to measure both speed and precision. Validity evidence was assessed by correlating performance with self-reported endoscopic experience (surgeons and gastroenterologists [GIs]). Internal consistency of each test task was assessed using Cronbach's alpha. Test-retest reliability was determined by having the same participant perform the test a second time and comparing their scores. Passing scores were determined by a contrasting groups methodology and use of receiver operating characteristic curves. RESULTS: A total of 160 participants (17 % GIs) performed the simulator test. Scores on the five tasks showed good internal consistency reliability and all had significant correlations with endoscopic experience. Total FES scores correlated 0.73, with participants' level of endoscopic experience providing evidence of their validity, and their internal consistency reliability (Cronbach's alpha) was 0.82. Test-retest reliability was assessed in 11 participants, and the intraclass correlation was 0.85. The passing score was determined and is estimated to have a sensitivity (true positive rate) of 0.81 and a 1-specificity (false positive rate) of 0.21. CONCLUSIONS: The FES hands-on skills test examines the basic procedural components required to perform safe flexible endoscopy. It meets rigorous standards of reliability and validity required for high-stakes examinations, and, together with the knowledge component, may help contribute to the definition and determination of competence in endoscopy.