Subjective vs. objective assessment of simulation performance on laparoscopic cholecystectomy: are we evaluating the right things?

BACKGROUND: Simulation using virtual reality (VR) simulators is an important tool in surgical training. VR laparoscopic simulators can provide immediate objective performance assessment without observer evaluation. This study aims to explore the correlation between subjective observer evaluation and VR laparoscopic simulator performance metrics in a laparoscopic cholecystectomy (LC) simulation module. METHODS: A LC simulation module using a VR laparoscopic simulator was completed by PGY2-3 general surgery residents at a single institution. Simulation performance was recorded and evaluated by a trained evaluator using the validated Global Operative Assessment of Laparoscopic Skills (GOALS) form, the Objective Structured Assessment of Technical Skills (OSATS) form, and a LC-specific simulation assessment form (LC-SIM). Objective performance metrics were also obtained from the simulator system. Performance before the curriculum (pre-test) and after the curriculum (post-test) were compared. RESULTS: Fourteen residents were included in the study. There were significant improvements from pre-test to post-test on each component of GOALS, OSATS, and LC-SIM scores (all p values < 0.05). In terms of objective simulator metrics, significant improvements were noted in time to extract gallbladder (481 ± 221 vs 909 ± 366 min, p = 0.019), total number of movements (475 ± 264 vs 839 ± 324 min, p = 0.012), and total path length (955 ± 475 vs 1775 ± 632 cm, p = 0.012) from pre-test to post-test. While number of movements and total path lengths of both hands decreased, speed of right instrument also decreased from 4.1 + 2.7 to 3.0 ± 0.7 cm/sec (p = 0.007). Average speed of left instrument was associated with respect for tissue (r = 0.60, p < 0.05) and depth perception (r = 0.68, p < 0.05) on post-test evaluations. CONCLUSION: Our study demonstrated significant improvement in technical skills based on subjective evaluator assessment as well as objective simulator metrics after simulation. The few correlations identified between the subjective evaluator and the objective simulator assessments suggest the two evaluation modalities were measuring different aspects of the technical skills and should both be considered in the evaluation process.

Comparing cost and outcomes between peroral endoscopic myotomy and laparoscopic heller myotomy.

BACKGROUND: Peroral endoscopic myotomy (POEM) has previously been shown to be equally if not more expensive than laparoscopic Heller myotomy (LHM). We compare perioperative outcomes and charges between POEM and LHM at a single institution. METHODS: Outcomes and charge data of 33 patients who underwent LHM and 126 patients who underwent POEM were analyzed. Patients who did not present electively were excluded. RESULTS: There were no demographic differences between groups. Patients who underwent POEM had a significantly shorter mean operative time and median length of stay (both p < 0.001). Patients who underwent POEM stopped narcotics earlier and had faster return to activities of daily living (both p < 0.05). When adjusted for inflation, POEM incurred less in hospital charges than LHM (35.5 ± 12.8 vs 30.7 ± 10.3 in thousands of US dollars, p = 0.006). CONCLUSIONS: Patients who underwent POEM compared to LHM had significantly better perioperative outcomes. Our results suggest POEM may be the more cost-effective option.

Predicting Progression in Barrett's Esophagus: Development and Validation of the Barrett's Esophagus Assessment of Risk Score (BEAR Score).

OBJECTIVE: To develop and validate a scoring tool capable of accurately predicting which patients with Barrett's esophagus (BE) will progress to dysplasia and/or esophageal adenocarcinoma. BACKGROUND: Endoscopic therapies have emerged capable of eradicating BE with high efficacy and low complication rates, but which patients should receive treatment is still debated. Current knowledge of risk factors is insufficient to allow for the accurate prediction of which patients will progress to dysplasia or adenocarcinoma. METHODS: We retrospectively collected data from a cohort of BE patients over a 13-year period. A multivariable logistic regression model was constructed to predict progression. A simplified risk of progression (ROP) score was developed from weighted beta coefficients. Internal validation was performed using bootstrap analysis, and model discrimination was assessed using k-fold cross-validation. RESULTS: The cohort included 2591 BE patients of which 133 progressed to dysplasia/adenocarcinoma. Multivariable analysis with bootstrap internal validation resulted in 5 variables associated with an increased ROP (age ≥70 years, male sex, lack of proton-pump inhibitor use, segment greater than 3 cm, and history of esophageal candidiasis). Using this model, we developed a simple ROP score between 0 and 8. Receiver operating characteristic analysis showed a cutoff of 3 or higher to have a sensitivity and specificity of 70% and 79%, respectively. Patients with a score of 3 or higher had an odds ratio of 9.04 (95% confidence interval 6.06-13.46). The c-statistic obtained from 10-fold cross-validation was 0.76 (95% confidence interval 0.72-0.79), indicating good overall discrimination. CONCLUSIONS: Our data show the development and internal validation of the Barrett's Esophagus Assessment of Risk Score as capable of quantifying the likelihood of progression to dysplasia/adenocarcinoma. The Barrett's Esophagus Assessment of Risk Score can be used clinically to guide treatment decisions in nondysplastic BE patients.

A Model of Cost Reduction and Standardization: Improved Cost Savings While Maintaining the Quality of Care.

BACKGROUND: Surgeon instrument choices are influenced by training, previous experience, and established preferences. This causes variability in the cost of common operations, such as laparoscopic appendectomy. Many surgeons are unaware of the impact that this has on healthcare spending. OBJECTIVE: We sought to educate surgeons on their instrument use and develop standardized strategies for operating room cost reduction. DESIGN: We collected the individual surgeon instrument cost for performing a laparoscopic appendectomy. Sixteen surgeons were educated about these costs and provided with cost-effective instruments and techniques. SETTINGS: This study was conducted in a university-affiliated hospital system. PATIENTS: Patients included those undergoing a laparoscopic appendectomy within the hospital system. MAIN OUTCOME MEASURES: Patient demographics, operating room costs, and short-term outcomes for the fiscal year before and after the education program were then compared. RESULTS: During fiscal year 2013, a total of 336 laparoscopic appendectomies were performed compared with 357 in 2014. Twelve surgeons had a ≥5% reduction in average cost per case. Overall, the average cost per case was reduced by 17% (p < 0.001). Switching from an energy device to a stapler load or reusable plastic clip applier resulted in the largest savings per case at $321 or $442 per case. There were no differences in length of stay, 30-day readmissions, postoperative infections, operating time, or reoperations. LIMITATIONS: This retrospective study is subject to the accuracy of the medical chart system. In addition, specific instrument costs are based on our institution contracts and vary compared with other institutions. CONCLUSIONS: In this study we demonstrate that operative instrument costs for laparoscopic appendectomy can be significantly reduced by informing the surgeons of their operating room costs compared with their peers and providing a low-cost standardized instrument tray. Importantly, this can be realized without any incentive or punitive measures and does not negatively impact outcomes. Additional work is needed to expand these results to more operations, hospital systems, and training programs.

Educating surgeons may allow for reduced intraoperative costs for inguinal herniorrhaphy.

BACKGROUND: Our aim was to determine the impact of surgeon education regarding disposable supply costs to reduce intraoperative costs for a common procedure such as inguinal hernia repair. STUDY DESIGN: At the end of the 2013 fiscal year (FY 13), surgeons in our department were provided with information about the cost of disposable equipment and implants used in common general surgery operations. Surgeons who historically had lower supply costs demonstrated individual techniques to their colleagues. No financial incentive or punitive measures were used to encourage behavior change. Surgical supply costs for laparoscopic and open inguinal hernia repair in FY13 were then compared with costs during fiscal year 2014 (FY14) using Mann-Whitney U tests. RESULTS: The average cost of laparoscopic inguinal hernia repairs decreased from an average $1,088±473 (±SD) in FY13 (n=258) to $860±441 (n=274) in FY14 after surgeon education, representing a 21.0% reduction in intraoperative costs (p<0.001). The most impactful adjustments to reduce costs included selective use of mesh fixation devices (22.9%) and balloon dissecting trocars (27.6%), reduction in use of disposable scissors (13.8%), and reduction in use of disposable clip appliers (3.7%). Open inguinal hernia costs were reduced from an average (±SD) of $315±$253 in FY13 (n=366) to $288±$130 in FY14 (n=286), an 8.6% reduction in cost (p<0.01). In these cases, both avoiding the use of fixation devices and using less expensive mesh implants were identified as significant factors. CONCLUSIONS: Surgeon education and empowerment may significantly reduce the cost of disposable equipment in laparoscopic and open inguinal hernia repair. This simple educational technique could prove financially beneficial throughout various procedures and disciplines.

Educating surgeons on intraoperative disposable supply costs during laparoscopic cholecystectomy: a regional health system's experience.

BACKGROUND: Surgeons play a crucial role in the cost efficiency of the operating room through total operative time, use of supplies, and patient outcomes. This study aimed to examine the effect of surgeon education on disposable supply usage during laparoscopic cholecystectomy. METHODS: Surgeons were educated about the cost of disposable equipments without incentives for achieved cost reductions. Surgical supply costs for laparoscopic cholecystectomy in fiscal year (FY) 2013 were compared with FY 2014. RESULTS: The average disposable supply cost per laparoscopic cholecystectomy was reduced from $589 (n = 586) in FY 2013 to $531 (n = 428) in FY 2014, representing a 10% reduction in supply costs (P < .001). Adjustments included reduction in the use of expensive fascial closure devices, clip appliers, suction irrigators, and specimen retrieval bags. CONCLUSIONS: Disposable equipment cost for laparoscopic cholecystectomy can be reduced by surgeon education. These techniques can likely be used to reduce costs in an array of specialties and procedures.

General surgery training without laparoscopic surgery fellows: the impact on residents and patients.

BACKGROUND: To evaluate resident case volume after discontinuation of a laparoscopic surgery fellowship, and to examine disparities in patient care over the same time period. METHODS: Resident case logs were compared for a 2-year period before and 1 year after discontinuing the fellowship, using a 2-sample t test. Databases for bariatric and esophageal surgery were reviewed to compare operative time, length of stay (LOS), and complication rate by resident or fellow over the same time period using a 2-sample t test. RESULTS: Increases were seen in senior resident advanced laparoscopic (Mean Fellow Year = 21 operations vs Non Fellow Year = 61, P < 0.01), esophageal (1 vs 11, P < .01) and bariatric volume (9 vs 36, P < .01). Junior resident laparoscopic volume increased (P < 0.05). No difference in LOS or complication rate was seen with resident vs fellow assistant. Operative time was greater for gastric bypass with resident assistant (152 ± 51 minutes vs 138 ± 53, P < .05). CONCLUSION: Discontinuing a laparoscopic fellowship significantly increases resident case volume in laparoscopic surgery. Operative time for complex operations may increase in the absence of a fellow. Other patient outcomes are not affected by this change.