Self-assessment, and not continuous training, improves basic open suturing skills.

BACKGROUND: To develop and maintain suturing skills, clinical exposure is important. When clinical exposure cannot be guaranteed, an adequate training schedule for suturing skills is required. This study evaluates the effect of continuous training, 'reflection before practice' and self-assessment on basic open suturing skills. METHODS: Medical students performed four basic suturing tasks on a simulation set up before ('pre-test') and after their surgical rotation ('after-test'). Participants were divided in three groups; the 'clinical exposure group' (n = 44) had clinical exposure during their rotation only, the 'continuous training group' (n = 16) completed a suturing interval training during their rotation and the 'self-assessment group' (n = 16) also completed a suturing interval training, but with the use of reflection before practice and self-assessment. Parameters measured by a tracking system during the suturing tasks and a calculated 'composite score' were compared between groups and test-moments. RESULTS: A significantly better composite score was found at the after-test compared to the pre-test for all groups for all basic suturing tasks (0.001 ≤ p ≤ 0.049). The self-assessment group scored better at the pre-test than the other two groups for all tasks, except for 'knot tying by hand' (0.004 ≤ p ≤ 0.063). However, this group did not score better at the after-test for all tasks, compared to the other two groups. This resulted in a smaller delta of time ('transcutaneous suture', p = 0.013), distance ('Donati suture' and 'intracutaneous suture', 0.005 ≤ p ≤ 0.009) or composite score (all tasks, except for knot tying by hand, 0.007 ≤ p ≤ 0.061) in the self-assessment group. CONCLUSION: Reflection before practice and self-assessment during continuous training of basic open suturing tasks, may improve surgical skills at the start of the learning curve.

Concurrent validity of objective assessment by finger tracking for open surgical suturing training.

BACKGROUND: The aim of this study is to evaluate whether the parameters 'time' and 'distance', measured by SurgTrac, correlate with the assessment of the same skills by blinded experts. METHODS: Basic open suturing tasks were executed by medical students. SurgTrac software measured objective parameters by tracking fingers. The executed tasks were recorded by a tablet and additionally assessed by a blinded expert with a Competency Assessment Tool-form (CAT-form). A Pearson's correlation was used to investigate the correlation between the parameters and the outcomes of the expert assessment. RESULTS: A strong correlation between the measured parameters of SurgTrac and the expert-assessment was found for knot tying by hand (r â€‹= â€‹-0.703) and vertical mattress suture (r â€‹= â€‹-0.644) and a moderate correlation for transcutaneous suture (r â€‹= â€‹-0.555) and intracutaneous suture (r â€‹= â€‹-0.451). CONCLUSION: The use of finger tracking by SurgTrac showed a good concurrent validity for the basic open suturing tasks knot tying by hand, transcutaneous suture and vertical mattress suture.

Objective assessment for open surgical suturing training by finger tracking can discriminate novices from experts.

It is difficult, time consuming and expensive to assess manual skills in open surgery. The aim of this study is to investigate the construct validity of a low-cost, easily accessible tracking technique for basic open suturing tasks. Medical master students, surgical residents, and surgeons at the Radboud University Medical Center were recruited between September 2020 until September 2021. The participants were divided, according to experience, in a novice group (≤10 sutures performed) and an expert group (>50 sutures performed). For objective tracking, a tablet with SurgTrac software was used, which tracked a blue and a red tag placed on respectively their left and right index finger. The participants executed four basic tasks on a suturing model: 1) knot tying by hand, 2) transcutaneous suturing with an instrument knot, 3) 'Donati' (vertical mattress suture) with an instrument knot and 4) continuous intracutaneous suturing without a knot. In total 76 participants were included: 57 novices and 19 experts. All four tasks showed significant differences between the novice group and expert group for the parameters time (p<0.001), distance (p<0.001 for Task 1, 2 and 3 and p=0.034 for Task 4) and smoothness (p<0.001). Additionally, Task 3 showed a significant difference for the parameter handedness (p=0.006) and Task 4 for speed (p=0.033). Tracking index finger movements using SurgTrac software on a tablet while executing basic open suturing skills on a simulator shows excellent construct validity for time, distance and motion smoothness in all four suturing tasks.

Assessment of Minimally Invasive Suturing Skills: Is Instrument Tracking an Accurate Prediction?

Introduction: Minimally invasive surgery (MIS) suturing demands advanced surgical skills. Therefore, it is important these skills are adequately trained and assessed. Assessment and feedback can consist of judgments and scores of expert observers or objective parameters using instrument tracking. The aim of this study was to determine to what extent objective parameters correspond to expert assessment. Methods: Participants performed an intracorporeal suturing task on the EoSim simulator repeatedly (maximum 20 repetitions) during training. The best discriminating parameters, which previously shown construct validation, were combined into a composite score, using regression analysis. All videos were blinded and assessed by 2 independent reviewers using the validated laparoscopic suturing competency assessment tool (LS-CAT). These scores were compared with the composite score. Results: A 100 videos of 16 trainees, during separate points on their learning curve, and 8 experts were used. The parameters "time" and "distance" were statistically significantly correlated with all LS-CAT domains. The composite score (calculated from "time" and "distance") showed improvement between the first and the last knot (57% versus 94%, P < .001). Also the LS-CAT score improved (28 versus 17, P < .001). However, the correlation of the composite score with the LS-CAT score was weak (R: 0.351), with an accuracy of 55/100 when pooling the outcomes based on inadequate, adequate, or good performance. Conclusion: Instrument tracking parameters (using Surgtrac) could give an indication of the skill level, however, it missed important elements, essential for reliable assessment. Therefore, expert assessment remains superior to determine the skill level in MIS suturing skills.

Are Self-Assessment and Peer Assessment of Added Value in Training Complex Pediatric Surgical Skills?

INTRODUCTION: Self-assessment aids "reflection-before-practice," which is expected to result in a better understanding of one's strengths and weaknesses and consequently a better overall performance. This is, however, rarely used in surgical training. This study aims to evaluate the correlation between self-, peer-, and expert assessment on surgical skills of pediatric surgical trainees. MATERIALS AND METHODS: A competency assessment tool for the posterior sagittal anorectoplasty (CAT-PSARP) was previously developed and validated. During international hands-on pediatric colorectal workshops in 2019 and 2020, participants practiced the PSARP on an inanimate anorectal malformation model. They were assisted by a peer and assessed by two independent expert observers, using the CAT-PSARP. After the training session, both self- and peer assessment were completed, using the same CAT-PSARP. RESULTS: A total of 79 participants were included. No correlation was found between the overall CAT-PSARP scores of the expert observers and the self-assessment (r = 0.179, p = 0.116), while a weak correlation was found between experts and peer assessment (r = 0.317, p = 0.006). When comparing the self-assessment scores with peer assessment, a moderate correlation was found for the overall performance score (r = 0.495, p < 0.001). Additionally, the participants who were first to perform the procedure scored significantly better than those who trained second on the overall performance (mean 27.2 vs. 24.4, p < 0.001). CONCLUSION: Participants, peers, and experts seemed to have a unique view on the performance during training because there was little correlation between outcomes of the trainees and the experts. Self-assessment may be useful for reflection during training; however, expert assessment seems to be essential for assessment of surgical skills.

Take-Home Laparoscopy Simulators in Pediatric Surgery: Is More Expensive Better?

Background: To increase complex minimally invasive skills (MIS), frequent training outside the clinical setting is of uttermost importance. This study compares two low-cost pediatric MIS simulators, which can easily be used preclinically. Materials and Methods: The LaparoscopyBoxx is a portable simulator without a tracking system, with costs ranging from €90 to €315. The EoSim simulator has a built-in camera and tracking system and costs range from €780 to €1800. During several pediatric surgical conferences and workshops (January 2017-December 2018), participants were asked to use both simulators. Afterward, they completed a questionnaire regarding their opinion on realism and didactic value, scored on a five-point Likert scale. Results: A total of 50 participants (24 experts and 25 target group, one unknown) evaluated one or both simulators. Both simulators scored well on the questionnaire. The LaparoscopyBoxx scored significantly better regarding the "on screen representation of the instrument actions" (mean 4.2 versus 3.5, P = .001), "training tool for pediatric surgery" (mean 4.4 versus 3.9, P = .005), and "appealing take-home simulator" (mean 4.6 versus 4.0, P = .002). Conclusion: The simulators tested in this study were both regarded an appealing take-home simulator. The LaparoscopyBoxx scored significantly better than the EoSim, even though this is a low budget simulator without tracking capabilities.

"Reflection-Before-Practice" Improves Self-Assessment and End-Performance in Laparoscopic Surgical Skills Training.

OBJECTIVE: To establish whether a systematized approach to self-assessment in a laparoscopic surgical skills course improves accordance between expert- and self-assessment. DESIGN: A systematic training course in self-assessment using Competency Assessment Tool was introduced into the normal course of evaluation within a Laparoscopic Surgical Skills training course for the test group (n = 30). Differences between these and a control group (n = 30) who did not receive the additional training were assessed. SETTING: Catharina Hospital, Eindhoven, The Netherlands (n = 27), and GSL Medical College, Rajahmundry, India (n = 33). PARTICIPANTS: Sixty postgraduate year 2 and 3 surgical residents who attended the 2-day Laparoscopic Surgical Skills grade 1 level 1 curriculum were invited to participate. RESULTS: The test group (n = 30) showed better accordance between expert- and self-assessment (difference of 1.5, standard deviation [SD] = 0.2 versus 3.83, SD = 0.6, p = 0.009) as well as half the number (7 versus 14) of cases of overreporting. Furthermore, the test group also showed higher overall mean performance (mean = 38.1, SD = 0.7 versus mean = 31.8, SD = 1.0, p < 0.001) than the control group (n = 30). The systematic approach to self-assessment can be viewed as responsible for this and can be seen as "reflection-before-practice" within the framework of reflective practice as defined by Donald Schon. CONCLUSION: Our results suggest that "reflection-before-practice" in implementing self-assessment is an important step in the development of surgical skills, yielding both better understanding of one's strengths and weaknesses and also improving overall performance.

The reliability of multisource feedback in competency-based assessment programs: the effects of multiple occasions and assessor groups.

PURPOSE: Residency programs around the world use multisource feedback (MSF) to evaluate learners' performance. Studies of the reliability of MSF show mixed results. This study aimed to identify the reliability of MSF as practiced across occasions with varying numbers of assessors from different professional groups (physicians and nonphysicians) and the effect on the reliability of the assessment for different competencies when completed by both groups. METHOD: The authors collected data from 2008 to 2012 from electronically completed MSF questionnaires. In total, 428 residents completed 586 MSF occasions, and 5,020 assessors provided feedback. The authors used generalizability theory to analyze the reliability of MSF for multiple occasions, different competencies, and varying numbers of assessors and assessor groups across multiple occasions. RESULTS: A reliability coefficient of 0.800 can be achieved with two MSF occasions completed by at least 10 assessors per group or with three MSF occasions completed by 5 assessors per group. Nonphysicians' scores for the "Scholar" and "Health advocate" competencies and physicians' scores for the "Health advocate" competency had a negative effect on the composite reliability. CONCLUSIONS: A feasible number of assessors per MSF occasion can reliably assess residents' performance. Scores from a single occasion should be interpreted cautiously. However, every occasion can provide valuable feedback for learning. This research confirms that the (unique) characteristics of different assessor groups should be considered when interpreting MSF results. Reliability seems to be influenced by the included assessor groups and competencies. These findings will enhance the utility of MSF during residency training.