Laparoscopic cholecystectomy critical view of safety (LC-CVS): a multi-national validation study of an objective, procedure-specific assessment using video-based assessment (VBA).

BACKGROUND: A novel 6-item objective, procedure-specific assessment for laparoscopic cholecystectomy incorporating the critical view of safety (LC-CVS OPSA) was developed to support trainee formative and summative assessments. The LC-CVS OPSA included two retraction items (fundus and infundibulum retraction) and four CVS items (hepatocystic triangle visualization, gallbladder-liver separation, cystic artery identification, and cystic duct identification). The scoring rubric for retraction consisted of poor (frequently outside of defined range), adequate (minimally outside of defined range) and excellent (consistently inside defined range) and for CVS items were "poor-unsafe", "adequate-safe", or "excellent-safe". METHODS: A multi-national consortium of 12 expert LC surgeons applied the OPSA-LC CVS to 35 unique LC videos and one duplicate video. Primary outcome measure was inter-rater reliability as measured by Gwet's AC2, a weighted measure that adjusts for scales with high probability of random agreement. Analysis of the inter-rater reliability was conducted on a collapsed dichotomous scoring rubric of "poor-unsafe" vs. "adequate/excellent-safe". RESULTS: Inter-rater reliability was high for all six items ranging from 0.76 (hepatocystic triangle visualization) to 0.86 (cystic duct identification). Intra-rater reliability for the single duplicate video was substantially higher across the six items ranging from 0.91 to 1.00. CONCLUSIONS: The novel 6-item OPSA LC CVS demonstrated high inter-rater reliability when tested with a multi-national consortium of LC expert surgeons. This brief instrument focused on safe surgical practice was designed to support the implementation of entrustable professional activities into busy surgical training programs. Instrument use coupled with video-based assessments creates novel datasets with the potential for artificial intelligence development including computer vision to drive assessment automation.

Subjective vs. objective assessment of simulation performance on laparoscopic cholecystectomy: are we evaluating the right things?

BACKGROUND: Simulation using virtual reality (VR) simulators is an important tool in surgical training. VR laparoscopic simulators can provide immediate objective performance assessment without observer evaluation. This study aims to explore the correlation between subjective observer evaluation and VR laparoscopic simulator performance metrics in a laparoscopic cholecystectomy (LC) simulation module. METHODS: A LC simulation module using a VR laparoscopic simulator was completed by PGY2-3 general surgery residents at a single institution. Simulation performance was recorded and evaluated by a trained evaluator using the validated Global Operative Assessment of Laparoscopic Skills (GOALS) form, the Objective Structured Assessment of Technical Skills (OSATS) form, and a LC-specific simulation assessment form (LC-SIM). Objective performance metrics were also obtained from the simulator system. Performance before the curriculum (pre-test) and after the curriculum (post-test) were compared. RESULTS: Fourteen residents were included in the study. There were significant improvements from pre-test to post-test on each component of GOALS, OSATS, and LC-SIM scores (all p values < 0.05). In terms of objective simulator metrics, significant improvements were noted in time to extract gallbladder (481 ± 221 vs 909 ± 366 min, p = 0.019), total number of movements (475 ± 264 vs 839 ± 324 min, p = 0.012), and total path length (955 ± 475 vs 1775 ± 632 cm, p = 0.012) from pre-test to post-test. While number of movements and total path lengths of both hands decreased, speed of right instrument also decreased from 4.1 + 2.7 to 3.0 ± 0.7 cm/sec (p = 0.007). Average speed of left instrument was associated with respect for tissue (r = 0.60, p < 0.05) and depth perception (r = 0.68, p < 0.05) on post-test evaluations. CONCLUSION: Our study demonstrated significant improvement in technical skills based on subjective evaluator assessment as well as objective simulator metrics after simulation. The few correlations identified between the subjective evaluator and the objective simulator assessments suggest the two evaluation modalities were measuring different aspects of the technical skills and should both be considered in the evaluation process.

Attending Surgeons Demonstrate Greater Correlations of Skill Assessment and Anticipation of Adverse Events Than Residents During Laparoscopic Cholecystectomy.

BACKGROUND: Surgical training includes the development of technical and nontechnical skills. While technical skills are more easily quantified, nontechnical skills such as situation awareness (SA) are more difficult to measure and quantify. This study investigated the relationships between different SA elements and expertise. METHODS: Twenty attending and resident surgeons rated their anticipation of an impending adverse event while watching 20 videos of laparoscopic cholecystectomies with and without adverse events. After watching each video, they assessed surgeon skills and self-assessed their anticipation ratings. All participants answered a general confidence questionnaire before and after the study. RESULTS: Videos with adverse events led to significantly higher anticipation of adverse events (P < 0.001), lower surgeon skill rating (P < 0.001), and higher self-assessment in their anticipation ratings (P < 0.001) across both participant groups. General confidence was significantly lower for residents than that for attending surgeons (P < 0.001). Compared with the residents, attendings exhibited stronger and more stable correlations between measurements of SA. When viewing videos with adverse events, attendings showed significantly higher correlation between anticipation of an impending adverse event and skill assessment of the surgeon (P = 0.005). CONCLUSIONS: This study investigated how different elements of SA and their relationships were influenced by experience. The results indicated that attendings had stronger and more stable correlations between SA elements than residents, demonstrating how measurement correlations could be meaningful and sensitive indicators of expertise and autonomy readiness.

3D-printed anatomical models of the cystic duct and its variants, a low-cost solution for an in-house built simulator for laparoscopic surgery training.

OBJECTIVES: To explore a method to create affordable anatomical models of the biliary tree that are adequate for training laparoscopic cholecystectomy with an in-house built simulator. METHODS: We used a fused deposition modeling 3D printer to create molds of Acrylonitrile Butadiene Styrene (ABS) from Digital Imaging and Communication on Medicine (DICOM) images, and the molds were filled with silicone rubber. Thirteen surgeons with 4-5-year experience in the procedure evaluated the molds using a low-cost in-house built simulator utilizing a 5-point Likert-type scale. RESULTS: Molds produced through this method had a consistent anatomical appearance and overall realism that evaluators agreed or definitely agreed (4.5/5). Evaluators agreed on recommending the mold for resident surgical training. CONCLUSIONS: 3D-printed molds created through this method can be applied to create affordable high-quality educational anatomical models of the biliary tree for training laparoscopic cholecystectomy.

The Inter-Rater Reliability of Technical Skills Assessment and Retention of Rater Training.

BACKGROUND: The inter-rater reliability (IRR) of laparoscopic skills assessment is usually determined in the context of motivated raters from a single subspecialty practice group with significant experience using similar tools. The purpose of this study was to determine the IRR among attending surgeons of different experience and practices, the extent of rater training that is necessary to achieve good IRR, and if rater training is retained over periods of nonuse. METHODS: In Part 1, 5 surgeons of different practice backgrounds assessed 3 laparoscopic cholecystectomy videos using the Global Operative Assessment of Laparoscopic Skills instrument. In Part 2, 2 of the surgeons assessed a total of 33 videos over 5 scoring sessions distributed across 6 months. They participated in 2 different training sessions, and retention was tested in the other 3 sessions. IRR was calculated for Parts 1 and 2 with an intraclass correlation (ICC) in a 2-way random-effects model. RESULTS: The ICC for Part 1 was poor (ICC = 0.26). In Part 2, the ICC was highest after each training session (scoring #1 ICC = 0.76, scoring #3 ICC = 0.74). The ICC was not retained 1.5 months after the brief video-based training session (scoring #2 ICC = -0.17). The ICC was retained 2.5 months after the in-depth discussion training session (scoring #4 ICC = 0.70), but not 4.5 months later (scoring #5 ICC = 0.04). CONCLUSIONS: Good IRR is not implicit among surgeons with varying backgrounds and experience. Good IRR can be achieved with different types of rater training, but the impact of rater training is lost in periods of nonuse. This suggests the need for further study of the IRR of technical skills assessment when performed by the wide variety of surgeon raters as is commonly encountered in the environment of postgraduate resident assessment.

Surgeons see anatomical structures faster and more accurately compared to novices: Development of a pattern recognition skill assessment platform.

BACKGROUND: This study aimed to identify differences in pattern recognition skill among individuals with varying surgical experience. METHODS: Participants reviewed laparoscopic cholecystectomy videos of various difficulty, and paused them when the cystic duct or artery was identified to outline each structure on the monitor. Time taken to identify each structure, accuracy and work load, which was assessed using the NASA-Task Load Index (TLX), were compared among the three groups. RESULTS: Ten students, ten residents and eight attendings participated in the study. Attendings identified the cystic duct and artery significantly faster and more accurately than students, and identified the cystic artery faster than residents. The NASA-TLX score of attendings was significantly lower than that of students and residents. CONCLUSIONS: Attendings identified anatomical structures faster, more accurately, and with less effort than students or residents. This platform may be valuable for the assessment and teaching of pattern recognition skill to novice surgeons. SHORT SUMMARY: Accurate anatomical recognition is paramount to proceeding safely in surgery. The assessment platform used in this study differentiated recognition skill among individuals with varing surgical experience.

Open cholecystectomy: Exposure and confidence of surgical trainees and new fellows.

BACKGROUND: The laparoscopic approach to cholecystectomy has overtaken open procedures in terms of frequency, despite open procedures playing an important role in certain clinical situations. This study explored exposure and confidence of Australasian surgical trainees and new fellows in performing an open versus laparoscopic cholecystectomy. MATERIALS AND METHODS: An online survey was disseminated via the Royal Australasian College of Surgeons to senior general surgery trainees (years 3-5 of surgical training) and new fellows (fellowship within the previous 5 years). The survey included questions regarding level of experience and confidence in performing an open cholecystectomy and converting from a laparoscopic to an open approach. RESULTS: A total of 135 participants responded; 58 (43%) were surgical trainees, 58 (43%) were fellows and 19 (14%) did not specify their level of training. Respondents who were involved in more than 20 open cholecystectomy procedures as an assistant or independent operator compared with those less exposed were more likely to feel confident to independently perform an elective open cholecystectomy (87.8% vs. 57.3%, P = 0.001), independently convert from a laparoscopic to open cholecystectomy (87.8% vs. 58.7%, P = 0.001) and independently perform an open cholecystectomy as a surgical consultant based on their level of exposure as a trainee (73.2% vs. 45.3%, P = 0.004). CONCLUSION: This study suggests the need to ensure surgical trainees are exposed to sufficient open cholecystectomies to enable confidence and skill with performing these procedures when indicated. Greater recognition of the need for exposure during training, including meaningful simulation, may assist.

A software-based tool for video motion tracking in the surgical skills assessment landscape.

BACKGROUND: The use of motion tracking has been proved to provide an objective assessment in surgical skills training. Current systems, however, require the use of additional equipment or specialised laparoscopic instruments and cameras to extract the data. The aim of this study was to determine the possibility of using a software-based solution to extract the data. METHODS: 6 expert and 23 novice participants performed a basic laparoscopic cholecystectomy procedure in the operating room. The recorded videos were analysed using Kinovea 0.8.15 and the following parameters calculated the path length, average instrument movement and number of sudden or extreme movements. RESULTS: The analysed data showed that experts had significantly shorter path length (median 127 cm vs. 187 cm, p = 0.01), smaller average movements (median 0.40 cm vs. 0.32 cm, p = 0.002) and fewer sudden movements (median 14.00 vs. 21.61, p = 0.001) than their novice counterparts. CONCLUSION: The use of software-based video motion tracking of laparoscopic cholecystectomy is a simple and viable method enabling objective assessment of surgical performance. It provides clear discrimination between expert and novice performance.

"Reflection-Before-Practice" Improves Self-Assessment and End-Performance in Laparoscopic Surgical Skills Training.

OBJECTIVE: To establish whether a systematized approach to self-assessment in a laparoscopic surgical skills course improves accordance between expert- and self-assessment. DESIGN: A systematic training course in self-assessment using Competency Assessment Tool was introduced into the normal course of evaluation within a Laparoscopic Surgical Skills training course for the test group (n = 30). Differences between these and a control group (n = 30) who did not receive the additional training were assessed. SETTING: Catharina Hospital, Eindhoven, The Netherlands (n = 27), and GSL Medical College, Rajahmundry, India (n = 33). PARTICIPANTS: Sixty postgraduate year 2 and 3 surgical residents who attended the 2-day Laparoscopic Surgical Skills grade 1 level 1 curriculum were invited to participate. RESULTS: The test group (n = 30) showed better accordance between expert- and self-assessment (difference of 1.5, standard deviation [SD] = 0.2 versus 3.83, SD = 0.6, p = 0.009) as well as half the number (7 versus 14) of cases of overreporting. Furthermore, the test group also showed higher overall mean performance (mean = 38.1, SD = 0.7 versus mean = 31.8, SD = 1.0, p < 0.001) than the control group (n = 30). The systematic approach to self-assessment can be viewed as responsible for this and can be seen as "reflection-before-practice" within the framework of reflective practice as defined by Donald Schon. CONCLUSION: Our results suggest that "reflection-before-practice" in implementing self-assessment is an important step in the development of surgical skills, yielding both better understanding of one's strengths and weaknesses and also improving overall performance.

Concerns of Quality and Safety in Public Domain Surgical Education Videos: An Assessment of the Critical View of Safety in Frequently Used Laparoscopic Cholecystectomy Videos.

BACKGROUND: Online videos are among the most common resources for case preparation. Using crowd sourcing, we evaluated the relationship between operative quality and viewing characteristics of online laparoscopic cholecystectomy videos. STUDY DESIGN: We edited 160 online videos of laparoscopic cholecystectomy to 60 seconds or less. Crowd workers (CW) rated videos using Global Objective Assessment of Laparoscopic Skills (GOALS), the critical view of safety (CVS) criteria, and assigned overall pass/fail ratings if CVS was achieved; linear mixed effects models derived average ratings. Views, likes, dislikes, subscribers, and country were recorded for subset analysis of YouTube videos. Spearman correlation coefficient (SCC) assessed correlation between performance measures. RESULTS: One video (0.06%) achieved a passing CVS score of ≥5; 23%, ≥4; 44%, ≥3; 79%, ≥2; and 100% ≥1. Pass/fail ratings correlated to CVS, SCC 0.95 (p < 0.001) and to GOALS, SCC 0.79 (p < 0.001). YouTube videos (n = 139) with higher views, likes, or subscribers did not correlate with better quality. The average CVS and GOALS scores were no different for videos with >20,000 views (22%) compared with those with <20,000 (78%). CONCLUSIONS: There is an incredibly low frequency of CVS and average GOALS technical performance in frequently used online surgical videos of LC. Favorable characteristics, such as number of views or likes, do not translate to higher quality.

Psychomotor skills assessment by motion analysis in minimally invasive surgery on an animal organ.

BACKGROUND: A high level of psychomotor skills is required to perform minimally invasive surgery (MIS) safely. To be able to measure these skills is important in the assessment of surgeons, as it enables constructive feedback during training. The aim of this study was to test the validity of an objective and automatic assessment method using motion analysis during a laparoscopic procedure on an animal organ. MATERIAL AND METHODS: Experienced surgeons in laparoscopy (experts) and medical students (novices) performed a cholecystectomy on a porcine liver box model. The motions of the surgical tools were acquired and analyzed by 11 different motion-related metrics, i.e., a total of 19 metrics as eight of them were measured separately for each hand. We identified for which of the metrics the experts outperformed the novices. RESULTS: In total, two experts and 28 novices were included. The experts achieved significantly better results for 13 of the 19 instrument motion metrics. CONCLUSIONS: Expert performance is characterized by a low time to complete the cholecystectomy, high bimanual dexterity (instrument coordination), a limited amount of movement and low measurement of motion smoothness of the dissection instrument, and relatively high usage of the grasper to optimize tissue positioning for dissection.

Assessment potential of a new suture simulator in laparoscopic surgical skills training.

BACKGROUND: The skills necessary for performing effective laparoscopic suturing are difficult to acquire; as a result, simulators for learning these skills are rapidly becoming integrated into surgical training. The aim of the study was to verify whether a new hybrid simulator has the potential to measure skill improvement in young, less experienced gastroenterological surgeons. MATERIAL AND METHODS: The study included 12 surgeons (median age, 29 (27-38)] years; 11 men (91.7%), one woman (8.3%)) who participated in a two-day laparoscopic training seminar. We used the new simulator before and after the program to evaluate individual performance. Skills were evaluated using five criteria: volume of air pressure leakage, number of full-thickness sutures, suture tension, wound area, and performance time. RESULTS: Air pressure leakage was significantly higher after than before the training (p = .027). The number of full-thickness sutures was significantly higher post-training (p < .01). Suture tension was significantly less post-training (p = .011). Wound opening areas were significantly smaller post-training (p = .018). Performance time was significantly shorter post-training (p = .032). CONCLUSIONS: Our study demonstrated the assessment quality of this new laparoscopic suture simulator.

Validation of the mobile serious game application Touch Surgery™ for cognitive training and assessment of laparoscopic cholecystectomy.

BACKGROUND: Touch Surgery™ (TS) is a serious gaming application for cognitive task simulation and rehearsal of key steps in surgical procedures. The aim was to establish face, content, and construct validity of TS for laparoscopic cholecystectomy (LC). Furthermore, learning curves with TS and a virtual reality (VR) trainer were compared in a randomized trial. METHODS: The performance of medical students and general surgeons was compared for all three modules of LC in TS to establish construct validity. Questionnaires assessed face and content validity. For analysis of learning curves, students were randomized to train on VR or TS first, and then switched to the other training modality. Performance data were recorded. RESULTS: 54 Surgeons and 51 medical students completed the validation study. Surgeons outperformed students with TS: patient preparation (students = 45.0 ± 19.1%; surgeons = 57.3 ± 15.2%; p < 0.001), access and laparoscopy (students = 70.2 ± 10.9%; surgeons = 75.9 ± 9.7%; p = 0.008) and LC (students = 69.8 ± 12.4%; surgeons = 77.7 ± 9.6%; p < 0.001). Both groups agreed that TS was a highly useful and realistic application. 46 students were randomized for learning curve analysis. It took them 2-4 attempts to reach a 100% score with TS. Training with TS first did not improve students' performance on the VR trainer; however, students who trained with VR first scored significantly higher in module 3 of TS. CONCLUSION: TS is an accepted serious gaming application for learning cognitive aspects of LC with established construct, face, and content validity. There appeared to be a synergy between TS and the VR trainer. Therefore, the two training modalities should accompany one another in a multimodal training approach to laparoscopy.

[What Surgeons Should Know about Risk Management]. / Risikomanagement in der Chirurgie ­ was muss der Chirurg wissen?

Background: The fact that medical treatment is associated with errors has long been recognized. Based on the principle of "first do no harm", numerous efforts have since been made to prevent such errors or limit their impact. However, recent statistics show that these measures do not sufficiently prevent grave mistakes with serious consequences. Preventable mistakes such as wrong patient or wrong site surgery still frequently occur in error statistics. Methods: Based on insight from research on human error, in due consideration of recent legislative regulations in Germany, the authors give an overview of the clinical risk management tools needed to identify risks in surgery, analyse their causes, and determine adequate measures to manage those risks depending on their relevance. The use and limitations of critical incident reporting systems (CIRS), safety checklists and crisis resource management (CRM) are highlighted. Also the rationale for IT systems to support the risk management process is addressed. Results/Conclusion: No single tool of risk management can be effective as a standalone instrument, but unfolds its effect only when embedded in a superordinate risk management system, which integrates tailor-made elements to increase patient safety into the workflows of each organisation. Competence in choosing adequate tools, effective IT systems to support the risk management process as well as leadership and commitment to constructive handling of human error are crucial components to establish a safety culture in surgery.

Measuring intra-operative decision-making during laparoscopic cholecystectomy: validity evidence for a novel interactive Web-based assessment tool.

BACKGROUND: Errors in judgment during laparoscopic cholecystectomy can lead to bile duct injuries and other complications. Despite correlations between outcomes, expertise and advanced cognitive skills, current methods to evaluate these skills remain subjective, rater- and situation-dependent and non-systematic. The purpose of this study was to develop objective metrics using a Web-based platform and to obtain validity evidence for their assessment of decision-making during laparoscopic cholecystectomy. METHODS: An interactive online learning platform was developed ( www.thinklikeasurgeon.com ). Trainees and surgeons from six institutions completed a 12-item assessment, developed based on a cognitive task analysis. Five items required subjects to draw their answer on the surgical field, and accuracy scores were calculated based on an algorithm derived from experts' responses ("visual concordance test", VCT). Test-retest reliability, internal consistency, and correlation with self-reported experience, Global Operative Assessment of Laparoscopic Skills (GOALS) score and Objective Performance Rating Scale (OPRS) score were calculated. Questionnaires were administered to evaluate the platform's usability, feasibility and educational value. RESULTS: Thirty-nine subjects (17 surgeons, 22 trainees) participated. There was high test-retest reliability (intraclass correlation coefficient = 0.95; n = 10) and internal consistency (Cronbach's α = 0.87). The assessment demonstrated significant differences between novices, intermediates and experts in total score (p < 0.01) and VCT score (p < 0.01). There was high correlation between total case number and total score (ρ = 0.83, p < 0.01) and between total case number and VCT (ρ = 0.82, p < 0.01), and moderate to high correlations between total score and GOALS (ρ = 0.66, p = 0.05), VCT and GOALS (ρ = 0.83, p < 0.01), total score and OPRS (ρ = 0.67, p = 0.04), and VCT and OPRS (ρ = 0.78, p = 0.01). Most subjects agreed or strongly agreed that the platform and assessment was easy to use [n = 29 (78 %)], facilitates learning intra-operative decision-making [n = 28 (81 %)], and should be integrated into surgical training [n = 28 (76 %)]. CONCLUSION: This study provides preliminary validity evidence for a novel interactive platform to objectively assess decision-making during laparoscopic cholecystectomy.

Self-assessment in laparoscopic surgical skills training: Is it reliable?

BACKGROUND: The concept of self-assessment has been widely acclaimed for its role in the professional development cycle and self-regulation. In the field of medical education, self-assessment has been most used to evaluate the cognitive knowledge of students. The complexity of training and evaluation in laparoscopic surgery has previously acted as a barrier in determining the benefits self-assessment has to offer in comparison with other fields of medical education. METHODS: Thirty-five surgical residents who attended the 2-day Laparoscopic Surgical Skills Grade 1 Level 1 curriculum were invited to participate from The Netherlands, India and Romania. The competency assessment tool (CAT) for laparoscopic cholecystectomy was used for self- and expert-assessment and the resulting distributions assessed. RESULTS: A comparison between the expert- and self-assessed aggregates of scores from the CAT agreed with previous studies. Uniquely to this study, the aggregates of individual sub-categories-'use of instruments'; 'tissue handling'; and errors 'within the component tasks' and the 'end product' from both self- and expert-assessments-were investigated. There was strong positive correlation (r s > 0.5; p < 0.001) between the expert- and self-assessment in all categories with only the 'tissue handling' having a weaker correlation (r s = 0.3; p = 0.04). The distribution of the mean of the differences between self-assessment and expert-assessment suggested no significant difference between the scores of experts and the residents in all categories except the 'end product' evaluation where the difference was significant (W = 119, p = 0.03). CONCLUSION: Self-assessment using the CAT form gives results that are consistently not different from expert-assessment when assessing one's proficiency in surgical skills. Areas where there was less agreement could be explained by variations in the level of training and understanding of the assessment criteria.

Direct Observation versus Endoscopic Video Recording-Based Rating with the Objective Structured Assessment of Technical Skills for Training of Laparoscopic Cholecystectomy.

PURPOSE: The validated Objective Structured Assessment of Technical Skills (OSATS) score is used for evaluating laparoscopic surgical performance. It consists of two subscores, a Global Rating Scale (GRS) and a Specific Technical Skills (STS) scale. The OSATS has accepted construct validity for direct observation ratings by experts to discriminate between trainees' levels of experience. Expert time is scarce. Endoscopic video recordings would facilitate assessment with the OSATS. We aimed to compare video OSATS with direct OSATS. METHODS: We included 79 participants with different levels of experience [58 medical students, 15 junior residents (novices), and 6 experts]. Performance of a cadaveric porcine laparoscopic cholecystectomy (LC) was evaluated with OSATS by blinded expert raters by direct observation and then as an endoscopic video recording. Operative time was recorded. RESULTS: Direct OSATS rating and video OSATS rating correlated significantly (x03C1; = 0.33, p = 0.005). Significant construct validity was found for direct OSATS in distinguishing between students or novices and experts. Students and novices were not different in direct OSATS or video OSATS. Mean operative times varied for students (73.4 ± 9.0 min), novices (65.2 ± 22.3 min), and experts (46.8 ± 19.9 min). Internal consistency was high between the GRS and STS subscores for both direct and video OSATS with Cronbach's α of 0.76 and 0.86, respectively. Video OSATS and operative time in combination was a better predictor of direct OSATS than each single parameter. CONCLUSION: Direct OSATS rating was better than endoscopic video rating for differentiating between students or novices and experts for LC and should remain the standard approach for the discrimination of experience levels. However, in the absence of experts for direct rating, video OSATS supplemented with operative time should be used instead of single parameters for predicting direct OSATS scores.

Validity, reliability and support for implementation of independence-scaled procedural assessment in laparoscopic surgery.

BACKGROUND: There is no widely used method to evaluate procedure-specific laparoscopic skills. The first aim of this study was to develop a procedure-based assessment method. The second aim was to compare its validity, reliability and feasibility with currently available global rating scales (GRSs). METHODS: An independence-scaled procedural assessment was created by linking the procedural key steps of the laparoscopic cholecystectomy to an independence scale. Subtitled and blinded videos of a novice, an intermediate and an almost competent trainee, were evaluated with GRSs (OSATS and GOALS) and the independence-scaled procedural assessment by seven surgeons, three senior trainees and six scrub nurses. Participants received a short introduction to the GRSs and independence-scaled procedural assessment before assessment. The validity was estimated with the Friedman and Wilcoxon test and the reliability with the intra-class correlation coefficient (ICC). A questionnaire was used to evaluate user opinion. RESULTS: Independence-scaled procedural assessment and GRS scores improved significantly with surgical experience (OSATS p = 0.001, GOALS p < 0.001, independence-scaled procedural assessment p < 0.001). The ICCs of the OSATS, GOALS and independence-scaled procedural assessment were 0.78, 0.74 and 0.84, respectively, among surgeons. The ICCs increased when the ratings of scrub nurses were added to those of the surgeons. The independence-scaled procedural assessment was not considered more of an administrative burden than the GRSs (p = 0.692). DISCUSSION/CONCLUSION: A procedural assessment created by combining procedural key steps to an independence scale is a valid, reliable and acceptable assessment instrument in surgery. In contrast to the GRSs, the reliability of the independence-scaled procedural assessment exceeded the threshold of 0.8, indicating that it can also be used for summative assessment. It furthermore seems that scrub nurses can assess the operative competence of surgical trainees.

Educating surgeons on intraoperative disposable supply costs during laparoscopic cholecystectomy: a regional health system's experience.

BACKGROUND: Surgeons play a crucial role in the cost efficiency of the operating room through total operative time, use of supplies, and patient outcomes. This study aimed to examine the effect of surgeon education on disposable supply usage during laparoscopic cholecystectomy. METHODS: Surgeons were educated about the cost of disposable equipments without incentives for achieved cost reductions. Surgical supply costs for laparoscopic cholecystectomy in fiscal year (FY) 2013 were compared with FY 2014. RESULTS: The average disposable supply cost per laparoscopic cholecystectomy was reduced from $589 (n = 586) in FY 2013 to $531 (n = 428) in FY 2014, representing a 10% reduction in supply costs (P < .001). Adjustments included reduction in the use of expensive fascial closure devices, clip appliers, suction irrigators, and specimen retrieval bags. CONCLUSIONS: Disposable equipment cost for laparoscopic cholecystectomy can be reduced by surgeon education. These techniques can likely be used to reduce costs in an array of specialties and procedures.

Validity and reliability of global operative assessment of laparoscopic skills (GOALS) in novice trainees performing a laparoscopic cholecystectomy.

PURPOSE: Global Operative Assessment of Laparoscopic Skills (GOALS) assessment has been designed to evaluate skills in laparoscopic surgery. A longitudinal blinded study of randomized video fragments was conducted to estimate the validity and reliability of GOALS in novice trainees. METHODS: In total, 10 trainees each performed 6 consecutive laparoscopic cholecystectomies. Sixty procedures were recorded on video. Video fragments of (1) opening of the peritoneum; (2) dissection of Calot's triangle and achievement of critical view of safety; and (3) dissection of the gallbladder from the liver bed were blinded, randomized, and rated by 2 consultant surgeons using GOALS. Also, a grade was given for overall competence. The correlation of GOALS with live observation Objective Structured Assessment of Technical Skills (OSATS) scores was calculated. Construct validity was estimated using the Friedman 2-way analysis of variance by ranks and the Wilcoxon signed-rank test. The interrater reliability was calculated using the absolute and consistency agreement 2-way random-effects model intraclass correlation coefficient. RESULTS: A high correlation was found between mean GOALS score (r = 0.879, p = 0.021) and mean OSATS score. The GOALS score increased significantly across the 6 procedures (p = 0.002). The trainees performed significantly better on their sixth when compared with their first cholecystectomy (p = 0.004). The consistency agreement interrater reliability was 0.37 for the mean GOALS score (p = 0.002) and 0.55 for overall competence (p < 0.001) of the 3 video fragments. CONCLUSION: The validity observed in this randomized blinded longitudinal study supports the existing evidence that GOALS is a valid tool for assessment of novice trainees. A relatively low reliability was found in this study.