Establishing validity evidence for device-assisted advanced laparoscopic suturing tasks using simulation.

BACKGROUND: Laparoscopic suturing (LS) is an essential technique required for a wide range of procedures, and it is one of the most challenging for surgical trainees to master. We designed and collected validity evidence for advanced LS tasks using an automated suturing device and evaluated the perceived educational value of the tasks. METHODS: This project was a multicentre prospective study involving McGill University, University of Toronto (UofT), and Louisiana State University (LSU) Health New Orleans. Novice (NS) and experienced (ES) surgeons performed suturing under tension (UT) and continuous suturing (CS) tasks. ES performed the tasks twice to establish proficiency benchmarks, and they were interviewed to develop formative feedback tools (FFT). Participants were assessed on completion time, error, Global Operative Assessment of Laparoscopic Skills (GOALS), and FFT. Data were analyzed using descriptive and inferential statistical methods. RESULTS: Twenty-seven participants (13 ES, 14 NS, median age 34 years; 85% male) completed the study. Eight were attending surgeons, 7 fellows, 4 PGY5, 5 PGY4, and 3 PGY3 (18 from McGill, 5 UofT, and 4 LSU). Comparing ES and NS, for UT task, ES had significantly greater task scores (570 [563-648] vs 323 [130-464], p value 0.00036) and GOALS scores (14 [13-16] vs 10 [8-12], p value 0.0038). Similarly, for CS, ES had significantly greater task scores (976 [959-1010] vs 785 [626-856], p value 0.00009) and GOALS scores (16 [12-17] vs 12.5 [8.25-15], p value 0.028). After FFTs were developed, comparing ES and NS, for both UT and CS tasks, ES had significantly greater FFT scores (UT 25 [24-26] vs 17 [14-20], p value 0.0016 and CS 30 [27-32] vs 22[17.2-25.8], p value 0.00061). CONCLUSION: In conclusion, preliminary validity evidence was provided for the tasks. Once further validity evidence is established, incorporating the tasks into the training curricula could improve trainee skills and help to meet the need for better advanced suturing models.

Development of a simulation curriculum to teach and assess advanced laparoscopic suturing skills using telesimulation: a feasibility study.

BACKGROUND: Telesimulation helps overcome limitations in time and local expertise by eliminating the need for the learner and educator to be physically co-located, especially important during COVID-19. We investigated whether teaching advanced laparoscopic suturing (ALS) through telesimulation is feasible, effective, and leads to improved suturing in the operating room (OR). METHODS: In this prospective feasibility study, three previously developed 3D-printed ALS tasks were used: needle handling (NH), suturing under tension (UT), and continuous suturing (CS). General surgery residents (PGY4-5) underwent 1-month of telesimulation training, during which an expert educator at one site remotely trained residents at the other site over 2-3 teaching sessions. Trainees were assessed in the three tasks and in the OR at three time points: baseline(A1), control period(A2), and post-intervention(A3) and completed questionnaires regarding educational value and usability of telesimulation. Paired t-test was used to compare scores between the three assessment points. RESULTS: Six residents were included. Scores for UT improved significantly post-intervention A3(568 ± 60) when compared to baseline A1(416 ± 133) (p < 0.019). Similarly, scores for CS improved significantly post-intervention A3(756 ± 113) vs. baseline A1(539 ± 211) (p < 0.02). For intraoperative assessments, scores improved significantly post-intervention A3(21 ± 3) when compared to both A1(17 ± 4) (p < 0.018) and A2(18 ± 4) (p < 0.0008). All residents agreed that tasks were relevant to practice, helped improve technical competence, and adequately measured suturing skill. All residents found telesimulation easy to use, had strong educational value, and want the system to be incorporated into their training. CONCLUSION: The use of telesimulation for remotely training residents using ALS tasks was feasible and effective. Residents found value in training using the tasks and telesimulation system, and improved ALS skills in the OR. As the pandemic has caused a major structural shift in resident education, telesimulation can be an effective alternative to on-site simulation programs. Future research should focus on how telesimulation can be effectively incorporated into training programs.

Defining the key skills required to perform advanced laparoscopic procedures: a qualitative descriptive study.

BACKGROUND: Simulation is widely used to teach and assess fundamental laparoscopic skills; however, program directors have reported that current simulation programs do not meet the needs for trainees and surgeons learning advanced laparoscopic procedures (ALP). The purpose of our study was to identify the key skills required to perform ALP, to serve as the basis to establish an advanced laparoscopic skills training program. METHODS: Semi-structured interviews were conducted with attending surgeons, fellows, and senior residents in general surgery, gynaecology, and urology. The questions were developed through an iterative process using relevant literature, expert opinions, and in consultation with a qualitative researcher. Interviews were conducted in person, over the phone, or by videoconference, and inductive thematic analysis was performed. RESULTS: 25 interviews were conducted with 16 attending surgeons and 9 fellows/residents from 9 institutions in Canada and USA. Twenty-one skills were identified to be important when performing ALP. The skills most commonly described by faculty were the following : (a) suturing, (b) dissection, (c) procedural expertise, (d) retraction and exposure, and (e) familiarity with relevant anatomy as viewed through the laparoscope. The skills most commonly described by trainees were the following: (a) suturing, (b) dissection, (c) procedural expertise, (d) trocar positioning, and (e) patient factors. There was a large difference between the importance the faculty attributed to the 'Retraction and Exposure' skill compared to the trainees. CONCLUSION: This study identified key skills that are important when performing ALP. In order to address the current needs of trainees/surgeons learning ALP, this work provides the building blocks for the development of an advanced laparoscopic surgery simulation program.

Is there a gender bias in the advancement to SAGES leadership?

BACKGROUND: The proportion of women in surgery has risen significantly yet there remains gender discrepancies in upper leadership positions in academia. Specialty societies play an important role in academic advancement but the progression of women in surgical societies has not been studied. The purpose of this study was to determine if there are gender differences in advancement within the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) leadership. METHODS: A retrospective audit of all SAGES committee members (CM) from 1992 to 2018 was performed. The overall membership gender distribution was available from 2010 to 2018. Leadership positions included Committee Chair/Co-chair, Board of Governors, and Executive Committee. Three phenomena were investigated: "pipeline," by determining the change in women CMs compared to overall membership over time; "sticky floors," by comparing advancement beyond CM by gender; "glass-ceiling," by analyzing the promotion trajectory and time to leadership positions between genders. Statistical analysis comparing trends over time was performed using Kendall-tau. RESULTS: There were 1546 surgeons who served on at least one committee during the study period. Women represented 21% of CMs, 18% of chairs/co-chairs, 16% of board members and 14% of executives, with one woman President. The proportion of women CMs has significantly increased over time from 3% in 1992 to 27% in 2018 (p-trend < 0.001). A similar proportion of women and men advanced beyond CM (17% vs. 14%, p = 0.194), with no difference in time to advancement. From 2010 to 2018, the increase in the proportion of women CMs and board members outpaced that of overall women members (p < 0.05). Women executives surpassed overall women members in 2018 (29% vs. 19%). A similar proportion of men and women "skipped ranks" to reach the Board/Executive (37% vs. 25%, p = 0.307). CONCLUSION: The proportion of women in leadership positions within SAGES is higher than in the overall membership. There were no gender differences in the advancement of CMs to leadership positions. While these data are encouraging, SAGES should continue to foster the advancement of women surgeons.

Simulation platforms to assess laparoscopic suturing skills: a scoping review.

BACKGROUND: Laparoscopic suturing (LS) has become a common technique used in a variety of advanced laparoscopic procedures. However, LS is a challenging skill to master, and many trainees may not be competent in performing LS at the end of their training. The purpose of this review is to identify simulation platforms available for assessment of LS skills, and determine the characteristics of the platforms and the LS skills that are targeted. METHODS: A scoping review was conducted between January 1997 and October 2018 for full-text articles. The search was done in various databases. Only articles written in English or French were included. Additional studies were identified through reference lists. The search terms included "laparoscopic suturing" and "clinical competence." RESULTS: Sixty-two studies were selected. The majority of the simulation platforms were box trainers with inanimate tissue, and targeted basic suturing and intracorporeal knot-tying techniques. Most of the validation came from internal structure (rater reliability) and relationship to other variables (compare training levels/case experience, and various metrics). Consequences were not addressed in any of the studies. CONCLUSION: We identified many types of simulation platforms that were used for assessing LS skills, with most being for assessment of basic skills. Platforms assessing the competence of trainees for advanced LS skills were limited. Therefore, future research should focus on development of LS tasks that better reflect the needs of the trainees.

Assessment of surgical performance of laparoscopic benign hiatal surgery: a systematic review.

BACKGROUND: Operative skills correlate with patient outcomes, yet at the completion of training or after learning a new procedure, these skills are rarely formally evaluated. There is interest in the use of summative video assessment of laparoscopic benign foregut and hiatal surgery (LFS). If this is to be used to determine competency, it must meet the robust criteria established for high-stakes assessments. The purpose of this review is to identify tools that have been used to assess performance of LFS and evaluate the available validity evidence for each instrument. METHODS: A systematic search was conducted up to July 2017. Eligible studies reported data on tools used to assess performance in the operating room during LFS. Two independent reviewers considered 1084 citations for eligibility. The characteristics and testing conditions of each assessment tool were recorded. Validity evidence was evaluated using five sources of validity (content, response process, internal structure, relationship to other variables, and consequences). RESULTS: There were six separate tools identified. Two tools were generic to laparoscopy, and four were specific to LFS [two specific to Nissen fundoplication (NF), one heller myotomy (HM), and one paraesophageal hernia repair (PEH)]. Overall, only one assessment was supported by moderate evidence while the others had limited or unknown evidence. Validity evidence was based mainly on internal structure (all tools reporting reliability and item analysis) and content (two studies referencing previous papers for tool development in the context of clinical assessment, and four listing items without specifying the development procedures). There was little or no evidence supporting test response process (one study reporting rater training), relationship to other variables (two comparing scores in subjects with different clinical experience), and consequences (no studies). Two tools were identified to have evidence for video assessment, specific to NF. CONCLUSION: There is limited evidence supporting the validity of assessment tools for laparoscopic foregut surgery. This precludes their use for summative video-based assessment to verify competency. Further research is needed to develop an assessment tool designed for this purpose.

A scoping review of assessment tools for laparoscopic suturing.

BACKGROUND: A needs assessment identified a gap in teaching and assessment of laparoscopic suturing (LS) skills. The purpose of this review is to identify assessment tools that were used to assess LS skills, to evaluate validity evidence available, and to provide guidance for selecting the right assessment tool for specific assessment conditions. METHODS: Bibliographic databases were searched till April 2017. Full-text articles were included if they reported on assessment tools used in the operating room/simulation to (1) assess procedures that require LS or (2) specifically assess LS skills. RESULTS: Forty-two tools were identified, of which 26 were used for assessing LS skills specifically and 26 for procedures that require LS. Tools had the most evidence in internal structure and relationship to other variables, and least in consequences. CONCLUSION: Through identification and evaluation of assessment tools, the results of this review could be used as a guideline when implementing assessment tools into training programs.

Development of a Model for the Acquisition and Assessment of Advanced Laparoscopic Suturing Skills Using an Automated Device.

BACKGROUND: Needs assessment identified a gap regarding laparoscopic suturing skills targeted in simulation. This study collected validity evidence for an advanced laparoscopic suturing task using an Endo StitchTM device. METHODS: Experienced (ES) and novice surgeons (NS) performed continuous suturing after watching an instructional video. Scores were based on time and accuracy, and Global Operative Assessment of Laparoscopic Surgery. Data are shown as medians [25th-75th percentiles] (ES vs NS). Interrater reliability was calculated using intraclass correlation coefficients (confidence interval). RESULTS: Seventeen participants were enrolled. Experienced surgeons had significantly greater task (980 [964-999] vs 666 [391-711], P = .0035) and Global Operative Assessment of Laparoscopic Surgery scores (25 [24-25] vs 14 [12-17], P = .0029). Interrater reliability for time and accuracy were 1.0 and 0.9 (0.74-0.96), respectively. All experienced surgeons agreed that the task was relevant to practice. CONCLUSION: This study provides validity evidence for the task as a measure of laparoscopic suturing skill using an automated suturing device. It could help trainees acquire the skills they need to better prepare for clinical learning.

Trends in the Fundamentals of Laparoscopic Surgery® (FLS) certification exam over the past 9 years.

BACKGROUND: The Fundamentals of Laparoscopic Surgery® (FLS) certification exam assesses both cognitive and manual skills, and has been administered for over a decade. The purpose of this study is to report results over the past 9 years of testing in order to identify trends over time and evaluate the need to update scoring practices. This is a quality initiative of the SAGES FLS committee. METHODS: A representative sample of FLS exam data from 2008 to 2016 was analyzed. The de-identified data included demographics and scores for the cognitive and manual tests. Standard descriptive statistics were used to compare trends over the years, training levels, and to assess the pass/fail rate. RESULTS: A total of 7232 FLS tests were analyzed [64% male, 6.4% junior (postgraduate year-PGY1-2), 84% senior (PGY3-5), 2.8% fellows (PGY6), and 6.7% attending surgeons (PGY7)]. Specialties included 93% general surgery (GS), 6.2% gynecology, and 0.9% urology. The Pearson correlation between cognitive and manual scores was 0.09. For the cognitive exam, there was an increase in scores over the years, and the most junior residents scored the lowest. For the manual skills, there were marginal differences in scores over the years, and junior residents scored the highest. The odds ratio of PGY3+ passing was 1.8 (CI 1.2-2.8) times higher than that of a PGY1-2. The internal consistency between tasks on the manual skills exam was 0.73. If any one of the tasks was removed, the Cronbach's alpha dropped to between 0.65 and 0.71, depending on the task being removed. CONCLUSION: The cognitive and manual components of FLS test different aspects of laparoscopy and demonstrate evidence for reliability and validity. More experienced trainees have a higher likelihood of passing the exam and tend to perform better on the cognitive skills. Each component of the manual skills contributes to the exam and should continue to be part of the test.

Effectiveness of Telementoring in Surgery Compared With On-site Mentoring: A Systematic Review.

BACKGROUND: Mentorship is important but may not be feasible for distance learning. To bridge this gap, telementoring has emerged. The purpose of this systematic review was to evaluate the effectiveness of telementoring compared with on-site mentoring. METHODS: A search was done up to March 2015. Studies were included if they used telementoring between surgeons during a clinical encounter and if they compared on-site mentoring and telementoring. RESULTS: A total of 11 studies were included. All reported no difference in complication rates, and 9 (82%) reported similar operative times; 4 (36%) reported technical issues, which was 3% of the total number of cases in the 11 studies. No study reported on higher levels of evidence for effectiveness of telementoring as an educational intervention. CONCLUSION: Studies reported that telementoring is associated with similar complication rates and operative times compared with on-site mentoring. However, the level of evidence to support the effectiveness of telementoring as a training tool is limited. There is a need for studies that provide evidence for the equivalence of the effectiveness of telementoring as an educational intervention in comparison with on-site mentoring.

What are the Training Gaps for Acquiring Laparoscopic Suturing Skills?

INTRODUCTION: Advanced laparoscopic suturing is considered a challenging skill to acquire. The aim of this study was to investigate the learning process for advanced laparoscopic suturing in the operating room to understand the obstacles trainees face when trying to master the skill. METHODS: A qualitative methodology using semistructured interviews and field observations was used. Data were analyzed using a Grounded Theory approach. Participants were general surgery residents and surgeons with advanced minimally invasive surgery (MIS) experience. RESULTS: Ten MIS surgeons across different institutions and 15 local general surgery residents were interviewed. The semistructured interviews and field observations of 9 advanced MIS operations (27h) yielded the following 6 themes around the acquisition of laparoscopic suturing skills for residents: complexity, training misalignment, variability of opportunities, inconsistency of techniques, lack of feedback, and differing expectations. CONCLUSION: There are several unmet training needs around laparoscopic suturing skills. Training for advanced laparoscopic skills requires more emphasis on coaching and the development of advanced models. This study heralded the need to incorporate advanced laparoscopic skills into the surgical simulation curriculum.

Establishing meaningful benchmarks: the development of a formative feedback tool for advanced laparoscopic suturing.

BACKGROUND: Time and accuracy are commonly used metrics to assess laparoscopic skills in a simulated environment. However, they do not provide trainees with meaningful information about how to improve their skills. The objective of this study was to provide preliminary validity evidence for the creation a formative feedback tool (FFT) for advanced laparoscopic suturing skills. METHODS: Videotapes of senior surgical residents (PGY3-5; SRs) and minimally invasive surgeons (MIS) performing 3 advanced laparoscopic suturing tasks were analyzed: needle handling (NH), suturing under tension (UT), and continuous suturing (CS). A FFT was created based on a grounded theory analysis of interviews with MIS surgeons about the key technical aspects of each task. The FFT was used to assess the videotaped performances of SRs and MIS surgeons by two blinded independent raters. RESULTS: The FFT is composed of three parts: NH contains 10 items, UT 18, and CS 20. Each item was classified according to seven key surgical principles: depth perception, safety, bimanual dexterity, exposure, tissue handling, instrument manipulation, and forward planning. The videotaped performance of SR and MIS surgeons was graded on a 3-point Likert scale ("does well," "needs some improvement," and "does poorly") and scores were calculated as a sum of the points. ICCs for all three tasks were high (NH 0.90, UT 0.87, and CS 0.90). FFT score correlated strongly with combined time and accuracy measurements for UT (0.82, p < 0.01) and CS (0.81, p < 0.01), and moderately for NH (0.65, p < 0.01). MIS surgeons performed significantly better than SRs on UT (p = 0.02) and CS (p = 0.05), while scores on NH were similar (p = 0.57). CONCLUSIONS: A comprehensive tool for providing feedback about advanced laparoscopic suturing skills was developed. The FFT demonstrates evidence for validity as a measure of suturing skills and experience, and provides meaningful information to trainees about how to improve their skills and engage in more deliberate and efficient practice.

Perioperative feedback in surgical training: A systematic review.

BACKGROUND: Changes in surgical training have raised concerns about residents' operative exposure and preparedness for independent practice. One way of addressing this concern is by optimizing teaching and feedback in the operating room (OR). The objective of this study was to perform a systematic review on perioperative teaching and feedback. METHODS: A systematic literature search identified articles from 1994 to 2014 that addressed teaching, feedback, guidance, or debriefing in the perioperative period. Data was extracted according to ENTREQ guidelines, and a qualitative analysis was performed. RESULTS: Thematic analysis of the 26 included studies identified four major topics. Observation of teaching behaviors in the OR described current teaching practices. Identification of effective teaching strategies analyzed teaching behaviors, differentiating positive and negative teaching strategies. Perceptions of teaching behaviors described resident and attending satisfaction with teaching in the OR. Finally models for delivering structured feedback cited examples of feedback strategies and measured their effectiveness. CONCLUSIONS: This study provides an overview of perioperative teaching and feedback for surgical trainees and identifies a need for improved quality and quantity of structured feedback.

Biologic mesh for repair of ventral hernias in contaminated fields: long-term clinical and patient-reported outcomes.

BACKGROUND: Guidelines recommend biologic prosthetics for ventral hernia repair (VHR) in contaminated fields, yet long-term and patient-reported data are limited. We aimed to determine the long-term rate of hernia recurrence, and other clinical and patient-reported outcomes following the use of porcine small intestine submucosa (PSIS) for VHR in a contaminated field. METHODS: Consecutive patients undergoing open VHR with PSIS mesh in a contaminated field from 2004 to 2014 were prospectively evaluated for hernia recurrence and other post-operative complications. Multivariate logistic and Cox regression analyses identified predictors of hernia recurrence and surgical site infection. Patient-reported outcomes were evaluated using SF-36, Hernia-Related Quality-of-Life Survey (HerQLes) and Body Image Questionnaire instruments. RESULTS: Forty-six hernias were repaired in clean-contaminated [16 (35 %)], contaminated [11 (24 %)] and dirty [19 (41 %)] fields. Median follow-up was 47 months [interquartile range: 31-79] and all patients had greater than 12-month follow-up. Sixteen patients (35 %) were not re-examined. Incidence of surgical site events and surgical site infection were 43 % (n = 20) and 56 % (n = 25), respectively. American Society of Anesthesiologists score 3 or greater was an independent predictor of surgical site infection (odds ratio 5.34 [95 % confidence interval 1.01-41.80], p = 0.04). Hernia recurrence occurred in 61 % (n = 28) with a median time to diagnosis of 16 months [interquartile range 8-26]. After bridged repair, 16 of 18 patients (89 %) recurred, compared to 12 of 28 (43 %) when fascia was approximated (p < 0.01). Bridged repair was an independent predictor of recurrence (odds ratio 10.67 [95 % confidence interval 2.42-76.08], p < 0.01). Patients with recurrences had significantly worse scores on the SF-36 mental health component and self-perceived body image, whereas HerQLes scores were similar. CONCLUSIONS: Hernia recurrences and wound infections are high with the use of biologic PSIS mesh in contaminated surgical fields. Careful consideration is warranted using this approach.

Measuring intra-operative decision-making during laparoscopic cholecystectomy: validity evidence for a novel interactive Web-based assessment tool.

BACKGROUND: Errors in judgment during laparoscopic cholecystectomy can lead to bile duct injuries and other complications. Despite correlations between outcomes, expertise and advanced cognitive skills, current methods to evaluate these skills remain subjective, rater- and situation-dependent and non-systematic. The purpose of this study was to develop objective metrics using a Web-based platform and to obtain validity evidence for their assessment of decision-making during laparoscopic cholecystectomy. METHODS: An interactive online learning platform was developed ( www.thinklikeasurgeon.com ). Trainees and surgeons from six institutions completed a 12-item assessment, developed based on a cognitive task analysis. Five items required subjects to draw their answer on the surgical field, and accuracy scores were calculated based on an algorithm derived from experts' responses ("visual concordance test", VCT). Test-retest reliability, internal consistency, and correlation with self-reported experience, Global Operative Assessment of Laparoscopic Skills (GOALS) score and Objective Performance Rating Scale (OPRS) score were calculated. Questionnaires were administered to evaluate the platform's usability, feasibility and educational value. RESULTS: Thirty-nine subjects (17 surgeons, 22 trainees) participated. There was high test-retest reliability (intraclass correlation coefficient = 0.95; n = 10) and internal consistency (Cronbach's α = 0.87). The assessment demonstrated significant differences between novices, intermediates and experts in total score (p < 0.01) and VCT score (p < 0.01). There was high correlation between total case number and total score (ρ = 0.83, p < 0.01) and between total case number and VCT (ρ = 0.82, p < 0.01), and moderate to high correlations between total score and GOALS (ρ = 0.66, p = 0.05), VCT and GOALS (ρ = 0.83, p < 0.01), total score and OPRS (ρ = 0.67, p = 0.04), and VCT and OPRS (ρ = 0.78, p = 0.01). Most subjects agreed or strongly agreed that the platform and assessment was easy to use [n = 29 (78 %)], facilitates learning intra-operative decision-making [n = 28 (81 %)], and should be integrated into surgical training [n = 28 (76 %)]. CONCLUSION: This study provides preliminary validity evidence for a novel interactive platform to objectively assess decision-making during laparoscopic cholecystectomy.

Don't fix it if it isn't broken: a survey of preparedness for practice among graduates of Fellowship Council-accredited fellowships.

BACKGROUND: General surgery residency may not adequately prepare residents for independent practice. It is unclear; however, if non-ACGME-accredited fellowships are better meeting training needs. The purpose of this mixed-method study was to determine perceived preparedness for practice and to identify gaps in fellowship training. METHODS: A survey was developed using an iterative qualitative methodology based on interviews and focus groups of graduated fellows and program directors. Five central themes emerged and were used as a framework: professional development, job marketability, autonomy, networking, and practice management. The survey was then circulated by email to fellows who graduated from Fellowship Council (FC)-accredited programs within the past 3 years. RESULTS: Of 201 respondents (response rate = 41 %), 95 and 97 % were highly satisfied with their operative and non-operative experiences; 83 % acquired jobs aligned with their skills and expectations, while 17 % sought additional training after fellowship. Respondents who intended to learn a given procedure felt competent after fellowship to perform 51(85 %) of the 60 procedures listed. They would have liked more experience in advanced therapeutic endoscopy, complex and revisional bariatric surgery, and uncommon laparoscopic procedures such as esophagectomy, adrenalectomy, and common bile duct exploration. Thirty-one percent expressed the desire for more autonomy in the management of complications. Educational gaps existed mostly in areas of coding and billing (42 %), hiring administrative staff (42 %), and managing insurance issues (34 %). CONCLUSIONS: FC-accredited fellowships seem to adequately prepare surgeons for independent practice and bridge training gaps after residency. Graduates are highly satisfied with the individualized training experience and acquire desired jobs aligned with their career goals.

Psychometric properties of the Global Operative Assessment of Laparoscopic Skills (GOALS) using item response theory.

BACKGROUND: The extent to which each item assessed using the Global Operative Assessment of Laparoscopic Skills (GOALS) contributes to the total score remains unknown. The purpose of this study was to evaluate the level of difficulty and discriminative ability of each of the 5 GOALS items using item response theory (IRT). METHODS: A total of 396 GOALS assessments for a variety of laparoscopic procedures over a 12-year time period were included. Threshold parameters of item difficulty and discrimination power were estimated for each item using IRT. RESULTS: The higher slope parameters seen with "bimanual dexterity" and "efficiency" are indicative of greater discriminative ability than "depth perception", "tissue handling", and "autonomy". CONCLUSIONS: IRT psychometric analysis indicates that the 5 GOALS items do not demonstrate uniform difficulty and discriminative power, suggesting that they should not be scored equally. "Bimanual dexterity" and "efficiency" seem to have stronger discrimination. Weighted scores based on these findings could improve the accuracy of assessing individual laparoscopic skills.

Multicenter proficiency benchmarks for advanced laparoscopic suturing tasks.

BACKGROUND: Advanced laparoscopic suturing (LS) tasks were developed based on a needs assessment. Initial validity evidence has been shown. The purpose of this multicenter study was to determine expert proficiency benchmarks for these tasks. METHODS: 6 tasks were included: needle handling (NH), offset-camera forehand suturing (OF), offset-camera backhand suturing (OB), confined space suturing (CF), suturing under tension (UT), and continuous suturing (CS). Minimally invasive surgeons experienced in LS completed the tasks twice. Mean time and median accuracy scores were used to establish the benchmarks. RESULTS: Seventeen MIS surgeons enrolled, from 7 academic centers. Mean (95% CI) time in seconds to complete each task was: NH 169 (149-189), OF 158 (134-181), OB 189 (154-224), CF 181 (156-205), UT 379 (334-423), and CS 416 (354-477). Very few errors in accuracy were made by experts in each of the tasks. CONCLUSIONS: Time- and accuracy-based proficiency benchmarks for 6 advanced LS tasks were established. These benchmarks will be included in an advanced laparoscopic surgery curriculum currently under development.