A Competency-based Laparoscopic Cholecystectomy Curriculum Significantly Improves General Surgery Residents' Operative Performance and Decreases Skill Variability: Cohort Study.

OBJECTIVE: To demonstrate the feasibility of implementing a CBE curriculum within a general surgery residency program and to evaluate its effectiveness in improving resident skill. SUMMARY OF BACKGROUND DATA: Operative skill variability affects residents and practicing surgeons and directly impacts patient outcomes. CBE can decrease this variability by ensuring uniform skill acquisition. We implemented a CBE LC curriculum to improve resident performance and decrease skill variability. METHODS: PGY-2 residents completed the curriculum during monthly rotations starting in July 2017. Once simulator proficiency was reached, residents performed elective LCs with a select group of faculty at 3 hospitals. Performance at curriculum completion was assessed using LC simulation metrics and intraoperative operative performance rating system scores and compared to both baseline and historical controls, comprised of rising PGY-3s, using a 2-sample Wilcoxon rank-sum test. PGY-2 group's performance variability was compared with PGY-3s using Levene robust test of equality of variances; P < 0.05 was considered significant. RESULTS: Twenty-one residents each performed 17.52 ± 4.15 consecutive LCs during the monthly rotation. Resident simulated and operative performance increased significantly with dedicated training and reached that of more experienced rising PGY-3s (n = 7) but with significantly decreased variability in performance ( P = 0.04). CONCLUSIONS: Completion of a CBE rotation led to significant improvements in PGY-2 residents' LC performance that reached that of PGY-3s and decreased performance variability. These results support wider implementation of CBE in resident training.

A Proposed Blueprint for Operative Performance Training, Assessment, and Certification.

OBJECTIVE: The aim of this study was to propose an evidence-based blueprint for training, assessment, and certification of operative performance for surgical trainees. SUMMARY BACKGROUND DATA: Operative skill is a critical aspect of surgical performance. High-quality assessment of operative skill therefore has profound implications for training, accreditation, certification, and the public trust of the profession. Current methods of operative skill assessment for surgeons rely heavily on global assessment strategies across a very broad domain of procedures. There is no mechanism to assure technical competence for individual procedures. The science and scalability of operative skill assessment has progressed significantly in recent decades, and can inform a much more meaningful strategy for competency-based assessment of operative skill than has been previously achieved. METHODS: The present article reviews the current status and science of operative skill assessment and proposes a template for competency-based assessment which could be used to update training, accreditation, and certification processes. The proposal is made in reference to general surgery but is more generally applicable to other procedural specialties. RESULTS: Streamlined, routine assessment of every procedure performed by surgical trainees is feasible and would enable a more competency-based educational paradigm. In light of the constraints imposed by both clinical volume and assessment bias, trainees should be expected to become proficient and be measured against a mastery learning standard only for the most important and highest-frequency procedures. For less frequently observed procedures, performance can be compared to a norm-referenced standard and, to provide an overall trajectory of performance, analyzed in aggregate. Key factors in implementing this approach are the number of evaluations, the number of raters, the timeliness of evaluation, and evaluation items. CONCLUSIONS: A competency-based operative skill assessment can be incorporated into surgical training, assessment, and certification. The time has come to develop a systematic approach to this issue as a means of demonstrating professional standards worthy of the public trust.

How Many Observations are Needed to Assess a Surgical Trainee's State of Operative Competency?

OBJECTIVE: To establish the number of operative performance observations needed for reproducible assessments of operative competency. BACKGROUND: Surgical training is transitioning from a time-based to a competency-based approach, but the number of assessments needed to reliably establish operative competency remains unknown. METHODS: Using a smart phone based operative evaluation application (SIMPL), residents from 13 general surgery training programs were evaluated performing common surgical procedures. Two competency metrics were investigated separately: autonomy and overall performance. Analyses were performed for laparoscopic cholecystectomy performances alone and for all operative procedures combined. Variance component analyses determined operative performance score variance attributable to resident operative competency and measurement error. Generalizability and decision studies determined number of assessments needed to achieve desired reliability (0.80 or greater) and determine standard errors of measurement. RESULTS: For laparoscopic cholecystectomy, 23 ratings are needed to achieve reproducible autonomy ratings and 17 ratings are needed to achieve reproducible overall operative performance ratings. For the undifferentiated mix of procedures, 60 ratings are needed to achieve reproducible autonomy ratings and 40 are needed for reproducible overall operative performance ratings. CONCLUSION: The number of observations needed to achieve reproducible assessments of operative competency far exceeds current certification requirements, yet remains an important and achievable goal. Attention should also be paid to the mix of cases and raters in order to assure fair judgments about operative competency and fair comparisons of trainees.

Readiness of US General Surgery Residents for Independent Practice.

OBJECTIVE: This study evaluates the current state of the General Surgery (GS) residency training model by investigating resident operative performance and autonomy. BACKGROUND: The American Board of Surgery has designated 132 procedures as being "Core" to the practice of GS. GS residents are expected to be able to safely and independently perform those procedures by the time they graduate. There is growing concern that not all residents achieve that standard. Lack of operative autonomy may play a role. METHODS: Attendings in 14 General Surgery programs were trained to use a) the 5-level System for Improving and Measuring Procedural Learning (SIMPL) Performance scale to assess resident readiness for independent practice and b) the 4-level Zwisch scale to assess the level of guidance (ie, autonomy) they provided to residents during specific procedures. Ratings were collected immediately after cases that involved a categorical GS resident. Data were analyzed using descriptive statistics and supplemented with Bayesian ordinal model-based estimation. RESULTS: A total of 444 attending surgeons rated 536 categorical residents after 10,130 procedures. Performance: from the first to the last year of training, the proportion of Performance ratings for Core procedures (n = 6931) at "Practice Ready" or above increased from 12.3% to 77.1%. The predicted probability that a typical trainee would be rated as Competent after performing an average Core procedure on an average complexity patient during the last week of residency training is 90.5% (95% CI: 85.7%-94%). This falls to 84.6% for more complex patients and to less than 80% for more difficult Core procedures. Autonomy: for all procedures, the proportion of Zwisch ratings indicating meaningful autonomy ("Passive Help" or "Supervision Only") increased from 15.1% to 65.7% from the first to the last year of training. For the Core procedures performed by residents in their final 6 months of training (cholecystectomy, inguinal/femoral hernia repair, appendectomy, ventral hernia repair, and partial colectomy), the proportion of Zwisch ratings (n = 357) indicating near-independence ("Supervision Only") was 33.3%. CONCLUSIONS: US General Surgery residents are not universally ready to independently perform Core procedures by the time they complete residency training. Progressive resident autonomy is also limited. It is unknown if the amount of autonomy residents do achieve is sufficient to ensure readiness for the entire spectrum of independent practice.

Practice Guidelines for Operative Performance Assessments.

OBJECTIVE: To provide recommended practice guidelines for assessing single operative performances and for combining results of operative performance assessments into estimates of overall operative performance ability. SUMMARY BACKGROUND DATA: Operative performance is one defining characteristic of surgeons. Assessment of operative performance is needed to provide feedback with learning benefits to surgical residents in training and to assist in making progress decisions for residents. Operative performance assessment has been a focus of investigation over the past 20 years. This review is designed to integrate findings of this research into a set of recommended operative performance practices. METHODS: Literature from surgery and from other pertinent research areas (psychology, education, business) was reviewed looking for evidence to inform practice guideline development. Guidelines were created along with a conceptual and scientific foundation for each guideline. RESULTS: Ten guidelines are provided for assessing individual operative performances and 10 are provided for combing data from individual operative performances into overall judgments of operative performance ability. CONCLUSIONS: The practice guidelines organize available information to be immediately useful to program directors, to support surgical training, and to provide a conceptual framework upon which to build as the base of pertinent knowledge expands through future research and development efforts.

A controlled study to determine measurement conditions necessary for a reliable and valid operative performance assessment: a controlled prospective observational study.

OBJECTIVE: This study evaluated operative performance rating (OPR) characteristics and measurement conditions necessary for reliable and valid operative performance (OP) assessment. BACKGROUND: Operative performance is a signature surgical-practice characteristic that is not measured systematically and specifically during residency training. METHODS: Expert surgeon raters from multiple institutions, blinded to resident characteristics, independently evaluated 8 open and laparoscopic OP recordings immediately after observation. RESULTS: A plurality of raters agreed on operative performance ratings (OPRs) for all performances. Using 10 judges adjusted for rater idiosyncrasies. Interrater agreement was similar for procedure-specific and general items. Higher post graduate year (PGY) residents received higher OPRs. Supervising-surgeon ratings averaged 0.51 points (1.2 standard deviations) above expert ratings for the same performances. CONCLUSIONS: OPRs have measurement properties (reliability, validity) similar to those of other well-developed performance assessments (Mini-CEX [clinical evaluation exercise], standardized patient examinations) when ratings occur immediately after observation. OPRs by blinded expert judges reflect the level of resident training and are practically significant differences as the average rating for PGY 4 residents corresponded to a "Good" performance whereas those for PGY 5 residents corresponded to a "Very Good" performance. Supervising surgeon ratings are higher than expert judge ratings reflecting the effect of interpersonal factors on supervising surgeon ratings. Use of local and national norms for interpretation of OPRs would adjust for these interpersonal factors. The OPR system provides a practical means for measuring operative performance, which is a signature characteristic of surgical practice.

The case of the entitled resident: a composite case study of a resident performance problem syndrome with interdisciplinary commentary.

BACKGROUND: Residents with performance problems create substantial burden on programs and institutions. Understanding the nature and quality of performance problems can help in learning to address performance problems. AIM: We sought to illuminate the effects of resident performance problems and the potential solutions for those problems from the perspectives of people with various roles in health care. METHODS: We created a composite portrait from several residents who demonstrated a cluster of common performance characteristics and whose chronic or serious maladaptive behavior and response to situations created problems for themselves, for their clinical colleagues, and for faculty of their residency program. The composite was derived from in-depth interviews of program directors and review of resident records. We solicited practitioners from multiple fields to respond to the portrait by answering a series of questions about severity, prognosis, and how and whether one could reliably remediate a person with these performance characteristics. We present their perspectives in a manner borrowed from the New England Journal of Medicine's "Case Records of the Massachusetts General Hospital." RESULTS: We created a composite portrait of a resident whose behavior suggested he felt entitled to benefits his peers were not entitled to. Experts reflecting on his behavior varied in their opinion about the effect the resident would have on the health care system. They suggested approaches to remediation that required substantial time and effort from the faculty. CONCLUSION: Programs must balance the needs of individual residents to adjust their behaviors with the needs of the health care system and other people within it.

Career Advancement for Surgeon-Educators:Findings from a Modified Delphi Process.

OBJECTIVES: Reward and recognition of surgical education as an academic activity remains a highly variable process between institutions. The goal of this study is to provide expert consensus definition of an academic surgical educator, with focus on criteria for academic promotion. STUDY DESIGN AND SETTING: Following IRB approval, a Web-based modified Delphi process was used to generate prioritized academic promotion criteria for surgical educators. PARTICIPANTS AND SETTING: Participants were recruited nationally from a pool of senior academic surgeons who are members of the Society of University Surgeons and the Society of Surgical Chairs. RESULTS: Following a three-round modified Delphi process, the top domains of educational activity for promotion to associate professor and professor were scholarship, teaching, and administration; mentorship was also a priority category for promotion to professor. The top three activities described for promotion to Associate Professor were active participation in conferences/ departmental educational activities for medical students and residents; educational portfolio demonstrating commitment to activities as an educator; and clinical teaching excellence at their home institution. The three activities most highly scored items for promotion to Professor were mentorship of junior surgical educators; active participation in conferences/ departmental educational activities for medical students and residents; and a record of teaching excellence at the medical student and resident levels. CONCLUSIONS: These findings demonstrate a progression from teacher to scholar to leader across a surgical educator's career, with each level incorporating and building upon the prior activities. Identification of categories and criteria may meaningfully inform best practices to be incorporated into the career development and promotion processes for surgeons on an educator academic pathway.

Team training for surgical trainees.

Healthcare professionals work in teams but are rarely trained together. Realizing the adverse impact of poor teamwork on patient care, the Accreditation Council for Graduate Medical Education requires surgical trainees to demonstrate a mastery of teamwork-related competencies. A number of team training curricula are available in the USA, the best known of which is TeamSTEPPS - developed by the U.S. Department of Defense Patient Safety Program in collaboration with the Agency for Healthcare Research and Quality.

Are surgeons behind the scientific eight ball: Delayed acquisition of the NIH K08 mentored career development award.

BACKGROUND: Surgery residents complete their research training early in residency. Non-surgical trainees typically have research incorporated toward the last two years of their fellowship, conferring an advantage to apply for grants with recent research experience and preliminary data. METHODS: The NIH RePORTER database was queried for K08 awardees trained in medicine, pediatrics, and surgery from 2013 to 2017. 406 K08 recipients were identified and time from completion of clinical training to achieving a K08 award was measured. Data were compared using ANOVA and expressed as mean. P < 0.05 was considered significant. RESULTS: Surgeons took longer to obtain a K08 than those trained in internal medicine (surgery = 3.7 years, internal medicine = 2.58 years p < 0.0001)). All K08 recipients without a PhD took longer to obtain a K08 than recipients with a PhD (MD = 3.50 years and MD/PhD = 2.42 years (p=<0.0001). CONCLUSIONS: Surgeons take longer to achieve a K08 award than clinicians trained in internal medicine, possibly due to an inherent disadvantage in training structure.

Trainee Autonomy in Minimally Invasive General Surgery in the United States: Establishing a National Benchmark.

OBJECTIVE: Minimally invasive surgery (MIS) is an integral component of General Surgery training and practice. Yet, little is known about how much autonomy General Surgery residents achieve in MIS procedures, and whether that amount is sufficient. This study aims to establish a contemporary benchmark for trainee autonomy in MIS procedures. We hypothesize that trainees achieve progressive autonomy, but fail to achieve meaningful autonomy in a substantial percentage of MIS procedures prior to graduation. SETTING/PARTICIPANTS: Fifty General Surgery residency programs in the United States, from September 1, 2015 to March 19, 2020. All Categorical General Surgery Residents and Attending Surgeons within these programs were eligible. DESIGN: Data were collected prospectively from attending surgeons and categorical General Surgery residents. Trainee autonomy was assessed using the 4-level Zwisch scale (Show and Tell, Active Help, Passive Help, and Supervision Only) on a smartphone application (SIMPL). MIS procedures included all laparoscopic, thoracoscopic, endoscopic, and endovascular/percutaneous procedures performed by residents during the study. Primary outcomes of interest were "meaningful autonomy" rates (i.e., scores in the top 2 categories of the Zwisch scale) by postgraduate year (PGY), and "progressive autonomy" (i.e., differences in autonomy between PGYs) in MIS procedures, as rated by attending surgeons. Primary outcomes were determined with descriptive statistics, one-way analysis of variance (ANOVA) and Z-tests. Secondary analyses compared (i) progressive autonomy between common MIS procedures, and (ii) progressive autonomy in MIS vs. non-MIS procedures. RESULTS: A total of 106,054 evaluations were performed across 50 General Surgery residency programs, of which 38,985 (37%) were for MIS procedures. Attendings performed 44,842 (42%) of all evaluations, including 16,840 (43%) of MIS evaluations, while residents performed the rest. Overall, meaningful autonomy in MIS procedures increased from 14.1% (PGY1s) to 75.9% (PGY5s), with significant (p < 0.001) increases between each PGY level. Meaningful autonomy rates were higher in the MIS vs. non-MIS group [57.2% vs. 48.0%, p < 0.001], and progressed more rapidly in MIS vs. non-MIS, (p < 0.05). The 7 most common MIS procedures accounted for 83.5% (n = 14,058) of all MIS evaluations. Among PGY5s performing these procedures, meaningful autonomy rates (%) were: laparoscopic appendectomy (95%); laparoscopic cholecystectomy (93%); diagnostic laparoscopy (87%); upper/lower endoscopy (85%); laparoscopic hernia repair (72%); laparoscopic partial colectomy (58%); and laparoscopic sleeve gastrectomy (45%). CONCLUSIONS: US General Surgery residents receive progressive autonomy in MIS procedures, and appear to progress more rapidly in MIS versus non-MIS procedures. However, residents fail to achieve meaningful autonomy in nearly 25% of MIS cases in their final year of residency, with higher rates of meaningful autonomy only achieved in a small subset of basic MIS procedures.

Vitamin E increases biomarkers of estrogen stimulation when taken with tamoxifen.

BACKGROUND: Vitamin E (alpha-tocopherol acetate, AT) diminishes the antiproliferative effect of tamoxifen on breast cancer cells in vitro. METHODS: A prospective study of seven women taking tamoxifen for adjuvant therapy of breast cancer. Four who were already taking AT supplements had random core biopsies of the normal breast and again 30 days after discontinuing AT. Three who were not on AT had biopsies before and after adding AT 400 mg for 30 days. Biopsies were stained for estrogen receptor (ER) and the mitogen-activated protein kinase p-ERK. Tissue extracts were assayed for p-ERK by enzyme-linked immunosorbent assay. Serum levels of alpha-tocopherol and tamoxifen were measured. RESULTS: Five out of seven patients had lower tamoxifen levels while taking AT, four of these went to subtherapeutic levels. Biopsies showed 23% of ductal cells were ER positive when patients were off AT and 70% on AT (P = 0.02). P-ERK staining was 21% off AT and 82% on AT. Five of seven patients had significantly higher tissue p-ERK when on AT. CONCLUSIONS: Biomarkers of estrogen-stimulation (ER, progesterone receptor, and p-ERK) were higher in breast biopsies of women taking vitamin E supplements while taking tamoxifen. Findings suggest that vitamin E supplements may interfere with the therapeutic effects of tamoxifen.

American Ginseng inhibits induced COX-2 and NFKB activation in breast cancer cells.

BACKGROUND: Epidemiologic evidence suggests reduced breast cancer mortality in users of American Ginseng (AG) (Panax quinquefolium). We hypothesized that AG extract decreases proliferation of human breast cancer cells via an anti-inflammatory effect applicable to the prevention of breast and other cancers. MATERIAL AND METHODS: A defined lyophilized aqueous extract of AG (LEAG) was dissolved in DMSO 1mg/mL, and serially diluted in saline. The cell lines MDA MB 231 and MCF7 were stimulated with the phorbol ester PDBu and treated with 100-500 mcg/mL LEAG. Proliferation was measured by MDA assay. Induced COX-2 expression was assayed by ELISA. Activation of NFkappaB by phosphorylation of the p65 subunit was quantified by CASE (cellular activation of signaling ELISA). RESULTS: Both cell lines had reduced proliferation when treated with LEAG. PDBu stimulation of MDA MB 231 increased expression of the COX-2 protein 20-fold at 48 hours (P<0.005). COX-2 protein expression remained at baseline concentrations in PDBu- treated MDA MB 231 cells exposed to 100 mcg/mL LEAG. The CASE assay showed a 4-fold increase in p65 activation 24 hours after PDBu treatment in normal medium, while phosphorylated p65 dropped below baseline in the cells treated with PDBu plus LEAG. CONCLUSION: In MDA MB 231, COX-2 was inducible with PDBu. This induced COX-2 expression was blocked by 100 microgram/mL LEAG in a time course consistent with the decline in the activated p65 subunit of NFkappaB. These results provide an anti-inflammatory mechanism for a possible anti-cancer effect of American Ginseng.

The business of educating the next generation of surgeons.

Surgical education community needs to be informed about how education is funded and how it is threatened. In order to explore these issues the Association of Surgical Education convened a panel with significant experience in managing surgery departments to discuss the business of surgical education. They specifically addressed methods to recognize and reward faculty, educate residents on safety, quality and cost, and increase departmental revenue. This information is important in the current educational environment where there is an increased need for institutions to find alternate revenue streams to sustain graduate medical education. It is also important to find additional revenue streams to fund new residency slots to accommodate the greater number medical students who have been admitted to medical schools in response to meet the projected shortage of physicians.

"Show Me the Data": A Recipe for Quality Improvement Success in an Academic Surgical Department.

BACKGROUND: Surgeons in academic medical centers have traditionally taken a siloed approach to reducing postoperative complications. We initiated a project focusing on transparency and sharing of data to engage surgeons in collaborative quality improvement. Its key features were the development of a comprehensive department quality dashboard and the creation of the Clinical Operations Council that oversaw quality. The purpose of this study was to assess the impact of those efforts. STUDY DESIGN: We compared inpatient outcomes before and after our intervention, allowing one quarter as the diffusion period. The outcomes analyzed were: risk-adjusted length of stay, mortality, direct cost and unadjusted incidence of complications, and 30-day all-cause readmissions, as determined by the Vizient Clinical Database. We examined the outcomes of three groups: group 1 (surgery); group 2, all other surgical departments (other surgery); and group 3, all other patients (non-surgery). Two-tailed Student's t-test was used for analysis and p < 0.05 was considered statistically significant. RESULTS: Group 1 demonstrated statistically significant improvements in mortality (p = 0.01), length of stay (p = 0.002), cost (p = 0.0001), and complications (p = 0.02), and the all-cause readmission rate was unchanged, resulting in mean decrease of 0.55 length of stay days and direct cost savings of $2,300 per surgical admission. The comparison groups had only modest decreases in some of the analyzed outcomes and an increase in complication rates. CONCLUSIONS: These data suggest that a collaborative, data-driven, and transparent approach to assessing the quality of surgical care can yield significant improvements in patient outcomes.

The effect of gender on operative autonomy in general surgery residents.

BACKGROUND: Despite an increasing number of women in the field of surgery, bias regarding cognitive or technical ability may continue to affect the experience of female trainees differently than their male counterparts. This study examines the differences in the degree of operative autonomy given to female compared with male general surgery trainees. METHODS: A smartphone app was used to collect evaluations of operative autonomy measured using the 4-point Zwisch scale, which describes defined steps in the progression from novice ("show and tell") to autonomous surgeon ("supervision only"). Differences in autonomy between male and female residents were compared using hierarchical logistic regression analysis. RESULTS: A total of 412 residents and 524 faculty from 14 general surgery training programs evaluated 8,900 cases over a 9-month period. Female residents received less autonomy from faculty than did male residents overall (P < .001). Resident level of training and case complexity were the strongest predictors of autonomy. Even after controlling for potential confounding factors, including level of training, intrinsic procedural difficulty, patient-related case complexity, faculty sex, and training program environment, female residents still received less operative autonomy than their male counterparts. The greatest discrepancy was in the fourth year of training. CONCLUSION: There is a sex-based difference in the autonomy granted to general surgery trainees. This gender gap may affect female residents' experience in training and possibly their preparation for practice. Strategies need to be developed to help faculty and residents work together to overcome this gender gap.

The new ACS/APDS Skills Curriculum: moving the learning curve out of the operating room.

Surgical education has dramatically changed in response to numerous constraints placed on residency programs, but a substantial gap in uniform practices exist, especially in the area of skills laboratory availability and usage. Simulation-based training has gained significant momentum and will be a requirement for residencies in the near future. In response, the American College of Surgeons and the Association of Program Directors in Surgery have formed a Surgical Skills Curriculum Task Force with the aim of establishing a National Skills Curriculum. The first of three phases will undergo implementation in 2007, with subsequent phases scheduled for launch in 2008. The curriculum has been carefully structured and designed by content experts to enhance resident training through reproducible simulations, with verification of proficiency before operative experience. Free-of-charge distribution is planned through a web-based platform, and widespread adoption is encouraged. In the future, these simulation-based strategies may be useful in assuring the competency of practicing surgeons and for credentialing purposes.