The effect of a three-dimensional instructional video on performance of a spatially complex procedure in surgical residents in relation to their visual-spatial abilities.

BACKGROUND: The effect of three-dimensional (3D) vs. two-dimensional (2D) video on performance of a spatially complex procedure and perceived cognitive load were examined among residents in relation to their visual-spatial abilities (VSA). METHODS: In a randomized controlled trial, 108 surgical residents performed a 5-Flap Z-plasty on a simulation model after watching the instructional video either in a 3D or 2D mode. Outcomes included perceived cognitive load measured by NASA-TLX questionnaire, task performance assessed using Observational Clinical Human Reliability Analysis and the percentage of achieved safe lengthening of the scar. RESULTS: No significant differences were found between groups. However, when accounted for VSA, safe lengthening was achieved significantly more often in the 3D group and only among individuals with high VSA (OR = 6.67, 95%CI: 1.23-35.9, p = .027). CONCLUSIONS: Overall, 3D instructional videos are as effective as 2D videos. However, they can be effectively used to enhance learning in high VSA residents.

Global versus task-specific postoperative feedback in surgical procedure learning.

BACKGROUND: Task-specific checklists and global rating scales are both recommended assessment tools to provide constructive feedback on surgical performance. This study evaluated the most effective feedback tool by comparing the effects of the Observational Clinical Human Reliability Analysis (OCHRA) and the Objective Structured Assessment of Technical Skills (OSATS) on surgical performance in relation to the visual-spatial ability of the learners. METHODS: In a randomized controlled trial, medical students were allocated to either the OCHRA (n = 25) or OSATS (n = 25) feedback group. Visual-spatial ability was measured by a Mental Rotation Test. Participants performed an open inguinal hernia repair procedure on a simulation model twice. Feedback was provided after the first procedure. Improvement in performance was evaluated blindly using a global rating scale (performance score) and hand-motion analysis (time and path length). RESULTS: Mean improvement in performance score was not significantly different between the OCHRA and OSATS feedback groups (P = .100). However, mean improvement in time (371.0 ± 223.4 vs 274.6 ± 341.6; P = .027) and path length (53.5 ± 42.4 vs 34.7 ± 39.0; P = .046) was significantly greater in the OCHRA feedback group. When stratified by mental rotation test scores, the greater improvement in time (P = .032) and path length (P = .053) was observed only among individuals with low visual-spatial abilities. CONCLUSION: A task-specific (OCHRA) feedback is more effective in improving surgical skills in terms of time and path length in novices compared to a global rating scale (OSATS). The effects of a task-specific feedback are present mostly in individuals with lower visual-spatial abilities.

Identifying the relationship between postoperative urinary continence and residual urethra stump measurements in robot assisted radical prostatectomy patients.

OBJECTIVE: To investigate the feasibility of urethral stump length and width measurements in recorded videos of robot assisted radical prostatectomy procedures using the Kinovea software and to assess if these measurements could be used as predictors of postoperative urinary continence. METHODS: Fifty-three patients were selected from an institutional database of 1400 cases and included in the study. All videos were analysed using the computer software 'Kinovea'. All measurements were performed using the inserted bladder catheter as a reference point. RESULTS: The reference point (bladder catheter) was available in 33 out of 53 patients. The median surgical urethral length (SUL) was significantly higher in the continent group (1050 vs. 1294 mm, p = 0.018). The urethral width measurements did not show a difference between the groups. In order to validate the Kinovea software as an accurate tool for the measurement of the urethral stump length and width results were correlated with the magnetic resonance imaging measurements of the urethra. CONCLUSIONS: The results of this study showed a significantly longer median SUL incontinent patients.

One Step at a Time: Step by Step Versus Continuous Video-Based Learning to Prepare Medical Students for Performing Surgical Procedures.

OBJECTIVE: The objective of this study was to compare the effects of cognitive load and surgical performance in medical students that performed the open inguinal hernia repair after preparation with step-by-step video-demonstration versus continuous video-demonstration. Hypothetically, the step-by-step group will perceive lower extraneous load during the preparation of the surgical procedure compared to the continuous group. Subsequently, fewer errors will be made in the surgical performance assessment by the step-by-step group, resulting in better surgical performance. DESIGN: In this prospective study, participants were randomly assigned to the step-by-step or continuous video-demonstration. They completed questionnaires regarding perceived cognitive load during preparation (10-point Likert scale). Their surgical performance was assessed on a simulation hernia model using the Observational Clinical Human Reliability Assessment. SETTING: Erasmus University Medical Center, Rotterdam, the Netherlands. PARTICIPANTS: Participants included medical students who were enrolled in extracurricular anatomy courses. RESULTS: Forty-three students participated; 23 students in the step-by-step group and 20 in the continuous group. As expected, the step-by-step group perceived a lower extraneous cognitive load (2.92 ± 1.21) compared to the continuous group (3.91 ± 1.67, p = 0.030). The surgical performance was not statistically significantly different between both groups; however, in subanalyses on a selection of students that prepared for 1 to 2 hours, the step-by-step group made less procedural errors, 1.67 ± 1.11, compared to the continuous group, 3.06 ± 1.91, p = 0.018. CONCLUSIONS: Our results suggest that preparation using step-by-step video-based learning results in lower extraneous cognitive load and subsequently fewer procedural errors during the surgical performance. For learning purposes, demonstration videos of surgical procedures should be presented in a segmented format.

Analysis of the video motion tracking system "Kinovea" to assess surgical movements during robot-assisted radical prostatectomy.

BACKGROUNDS: Robot-assisted surgery facilitated the possibility to evaluate the surgeon's skills by recording and evaluating the robot surgical images. The aim of this study was to investigate the possibility of using a computer programme (Kinovea) for objective assessment of surgical movements in previously recorded in existing robot-assisted radical prostatectomy (RARP) videos. METHODS: Twelve entire RARP procedures were analysed by a trained researcher using the computer programme "Kinovea" to perform semi-automated assessment of surgical movements. RESULTS: Data analysis showed Kinovea was on average able to automatically assess only 22% of the total surgical duration per video of the robot-assisted surgery. On average, it lasted 4 hours of continued monitoring by the researcher to assess one RARP using Kinovea. CONCLUSION: Although we proved it is technically possible to use the Kinovea system in retrospective analysis of surgical movement in robot-assisted surgery, the acquired data do not give a comprehensive enough analysis of the video to be used in skills assessment.

A Prospective, Observational, Multicentre Study Concerning Nontechnical Skills in Robot-assisted Radical Cystectomy Versus Open Radical Cystectomy.

INTRODUCTION AND HYPOTHESES: valuation of surgical skills, both technical and nontechnical, is possible through observations and video analysis. Besides technical failures, adverse outcomes in surgery can also be related to hampered communication, moderate teamwork, lack of leadership, and loss of situational awareness. Even though some surgeons are convinced about nontechnical skills being an important part of their professionalisation, there is paucity of data about a possible relationship between nontechnical skills and surgical outcome. In robot-assisted surgery, the surgeon sits behind the console and is at a remote position from the surgical field and team, making communication more important than in open surgery and conventional laparoscopy. A lack of structured research makes it difficult to assess the value of the different analysis methods for nontechnical skills, particularly in robot-assisted surgery. Our hypothesis includes the following: (1) introduction of robot-assisted surgery leads to an initial decay in nontechnical skills behaviour during the learning curve of the team, (2) nontechnical skills behaviour is more explicitly expressed in experienced robot-assisted surgery teams than in experienced open surgery teams, and (3) introduction of robot-assisted surgery leads to the development of different forms of nontechnical skills behaviour compared with open surgery. DESIGN: This study is a prospective, observational, multicentre, nonrandomised, case-control study including bladder cancer patients undergoing either an open radical cystectomy or a robot-assisted radical cystectomy at the Catharina Hospital Eindhoven, the Netherlands, or at the Netherlands Cancer Institute, Antoni van Leeuwenhoek Hospital Amsterdam. All patients are eligible for inclusion; there are no exclusion criteria. The Catharina Hospital Eindhoven, the Netherlands, performs on average 35 radical cystectomies a year. The Netherlands Cancer Institute, Antoni van Leeuwenhoek Hospital Amsterdam, performs on average 100 radical cystectomies a year. PROTOCOL OVERVIEW: The choice of treatment is at the discretion of the patient and the surgeon. Patient results will be obtained prospectively. Pathology results as well as complications occurring within 90 d following surgery will be registered. Surgical complications will be registered according to the Clavien-Dindo system. MEASUREMENTS: Nontechnical skills will be observed using five different methods: (1) NOTSS: Nontechnical Skills for Surgeons; (2) Oxford NOTECHS II: a modified theatre team nontechnical skills scoring system; (3) OTAS: Observational Teamwork Assessment for Surgery; (4) Interpersonal and Cognitive Assessment for Robotic Surgery (ICARS): evaluation of nontechnical skills in robotic surgery; and (5) analysis of human factors. Technical skills in robot-assisted radical cystectomy will be analysed using two different methods: (1) GEARS: Global Evaluative Assessment of Robotic Skill and (2) GERT: Generic Error Rating Tool. SAFETY CRITERIA AND REPORTING: Formal ethical approval has been provided by Medical research Ethics Committees United (MEC-U), The Netherlands (reference number W19.048). We hope to present the results of this study to the scientific community at conferences and in peer-reviewed journals. STATISTICAL ANALYSIS: Frequency statistics will be calculated for patient demographical data, and a Shapiro-Wilk test with p > 0.05 will be used to define normal distribution. Univariate analysis will be conducted to test for statistically significant differences in observation scores between open radical cystectomy and robot-assisted radical cystectomy cohorts across all variables, using independent sample t tests and Mann-Whitney U testing, as appropriate. A variable-selection strategy will be used to create multivariate models. Binary logistic regression will be conducted to calculate odds ratios and 95% confidence intervals for significant predictors on univariate analysis and clinically relevant covariates. Statistical significance is set at p < 0.05 based on a two-tailed comparison. SUMMARY: This study uses a structured approach to the analysis of nontechnical skills using extracorporeal videos of both open radical cystectomy and robot-assisted radical cystectomy surgeries, in order to obtain detailed data on nontechnical skills during open and minimally invasive surgeries. The results of this study could possibly be used to develop team-training programmes, specifically for the introduction of the surgical robot in relation to changes in nontechnical skills. Additional analysis of technical skills using the intracorporeal footage of the surgical robot will be used to elucidate the role of surgical skills and surgical events in nontechnical skills.

A review to identify key perspectives in PBL meta-analyses and reviews: trends, gaps and future research directions.

In the past 50 years, the original McMaster PBL model has been implemented, experimented, revised, and modified, and is still evolving. Yet, the development of PBL is not a series of success stories, but rather a journey of experiments, failures and lessons learned. In this paper, we analyzed the meta-analyses and systematic reviews on PBL from 1992 to present as they provide a focused lens on the PBL research in the past 5 decades. We identified three major waves in the PBL research development, analyzed their impact on PBL research and practice, and offered suggestions of research gaps and future directions for the field. The first wave of PBL research (polarization: 1990-mid 2000) focused on answering the question "Does PBL work?" and the outcomes. The results were conflicting. The researchers took polarizing positions and debated over the merits of PBL throughout this wave. However, the contradictory results and the debates in fact pushed the researchers to look harder for new directions to solve the puzzle. These efforts resulted in the second wave (from outcomes to process: mid 2000-mid 2010) that focused on the question "How does PBL work?" The second wave of PBL research targeted at investigating the effects of implementation constituents, such as assessment formats or single versus curriculum wide implementations. The third wave (specialization: mid 2010 and onward) of PBL research focused on "How does PBL work in different specific contexts?" These research widened our perspectives by expanding our understanding of how PBL manifests itself in different contexts. Given the diversification of PBL and more hybrid PBL models, we suggest "Why does PBL with particular implementation characteristics for specific outcomes work or not work in the condition where it is implemented?" to be the question to answer in the next wave of PBL research.

Reversal of the hanging protocol of Contrast Enhanced Mammography leads to similar diagnostic performance yet decreased reading times.

OBJECTIVES: Contrast-enhanced mammography (CEM) was found superior to Full-Field Digital Mammography (FFDM) for breast cancer detection. Current hanging protocols show low-energy (LE, similar to FFDM) images first, followed by recombined (RC) images. However, evidence regarding which hanging protocol leads to the most efficient reading process and highest diagnostic performance is lacking. This study investigates the effects of hanging-protocol ordering on the reading process and diagnostic performance of breast radiologists using eye-tracking methodology. Furthermore, it investigates differences in reading processes and diagnostic performance between LE, RC and FFDM images. MATERIALS AND METHODS: Twenty-seven breast radiologists were randomized into three reading groups: LE-RC (commonly used hangings), RC-LE (reversed hangings) and FFDM. Thirty cases (nine malignant) were used. Fixation count, net dwell time and time-to-first fixation on malignancies as visual search measures were registered by the eye-tracker. Reading time per image was measured. Participants clicked on suspicious lesions to determine sensitivity and specificity. Area-under-the-ROC-curve (AUC) values were calculated. RESULTS: RC-LE scored identical on visual search measures, t(16)= -1.45, p = .17 or higher-p values, decreased reading time with 31%, t(16)= -2.20, p = .04, while scoring similar diagnostic performance compared to LE-RC, t(13.2) = -1.39, p - .20 or higher p-values. The reading process was more efficient on RC compared to LE. Diagnostic performance of CEM was superior to FFDM; F (2,26) = 16.1, p < .001. Average reading time did not differ between the three groups, F (2,25) = 3.15, p = .06. CONCLUSION: The reversed CEM hanging protocol (RC-LE) scored similar on diagnostic performance compared to LE-RC, while reading time was a third faster. Abnormalities were interpreted quicker on RC images. A RC-LE hanging protocol is therefore recommended for clinical practice and training. Diagnostic performance of CEM was (again) superior to FFDM.

The value of a 1-day multidisciplinary robot surgery training for novice robot surgeons.

INTRODUCTION: To fulfil the need for a basic level of competence in robotic surgery (Brinkman et al., Surg Endosc Other Interv Tech 31(1):281-287, 2017; Dutch Health inspectorate (Inspectie voor de gezondheidszorg), Insufficient carefulness at the introduction of surgical robots (in Dutch: Onvoldoende zorgvuldigheid bij introductie van operatierobots), Igz, Utrecht, 2010), the NIVEL (Netherlands Institute for Healthcare Research) developed the 'Basic proficiency requirements for the safe use of robotic surgery' (BPR). Based on the BPR a 1-day robotic surgery training was organised to answer the following research questions: (1) Are novice robot surgeons able to accurately self-assess their knowledge and dexterity skills? (2) Is it possible to include the teaching of all BPRs in a 1-day training? MATERIALS AND METHODS: Based on the BPR, a robot surgery course was developed for residents and specialists (surgery, gynaecology and urology). In preparation, the participants completed an online e-module. The 1-day training consisted of a practical part on robot set-up, a theoretical section, and hands-on exercises on virtual reality robot simulators. Multiple online questionnaire was filled out by the participants at the end of the training to evaluate the perceived educational value of the course and to self-assess the degree to which BPRs were reached. RESULTS: 20 participants completed the training during the conference of the Dutch Association for Endoscopic Surgery (NVEC) in 2017. Participants indicated nearly all competency requirements were mastered at the end of the training. The competency requirements not mastered were, however, critical requirements for the safe use of the surgical robot. Skill simulation results show a majority of participants are unable to reach a proficient simulation score in basic skill simulation exercises. CONCLUSION: Results show novice robot surgeons are too positive in the self-assessment of their own dexterity skills after a 1-day training. Self-assessment revealed uncertainty of the obtained knowledge level on requirements for the safe use of the surgical robot. Basic courses on robotic training should inform trainees about their results to enhance learning and inform them of their competence levels.

Development and validation of the TOCO-TURBT tool: a summative assessment tool that measures surgical competency in transurethral resection of bladder tumour.

BACKGROUND: The current shift towards competency-based residency training has increased the need for objective assessment of skills. In this study, we developed and validated an assessment tool that measures technical and non-technical competency in transurethral resection of bladder tumour (TURBT). METHODS: The 'Test Objective Competency' (TOCO)-TURBT tool was designed by means of cognitive task analysis (CTA), which included expert consensus. The tool consists of 51 items, divided into 3 phases: preparatory (n = 15), procedural (n = 21), and completion (n = 15). For validation of the TOCO-TURBT tool, 2 TURBT procedures were performed and videotaped by 25 urologists and 51 residents in a simulated setting. The participants' degree of competence was assessed by a panel of eight independent expert urologists using the TOCO-TURBT tool. Each procedure was assessed by two raters. Feasibility, acceptability and content validity were evaluated by means of a quantitative cross-sectional survey. Regression analyses were performed to assess the strength of the relation between experience and test scores (construct validity). Reliability was analysed by generalizability theory. RESULTS: The majority of assessors and urologists indicated the TOCO-TURBT tool to be a valid assessment of competency and would support the implementation of the TOCO-TURBT assessment as a certification method for residents. Construct validity was clearly established for all outcome measures of the procedural phase (all r > 0.5, p < 0.01). Generalizability-theory analysis showed high reliability (coefficient Phi ≥ 0.8) when using the format of two assessors and two cases. CONCLUSIONS: This study provides first evidence that the TOCO-TURBT tool is a feasible, valid and reliable assessment tool for measuring competency in TURBT. The tool has the potential to be used for future certification of competencies for residents and urologists. The methodology of CTA might be valuable in the development of assessment tools in other areas of clinical practice.

The Simbla TURBT Simulator in Urological Residency Training: From Needs Analysis to Validation.

OBJECTIVE: To investigate the value of the physical Simbla Transurethral Resection of a Bladder Tumor (TURBT) simulator as an educational tool within urological residency training, by means of a training needs analysis (TNA) and assessment of its feasibility, acceptability, and face, content, and construct validity. METHODS: To analyze the training needs for TURBT, procedural steps and pitfalls were identified and the TNA was completed during an expert consensus meeting. Participants (n = 76) were divided into three groups based on their experience in TURBT: novices, intermediates, and experts. Participants performed two standardized TURBT procedures on the simulator. Face validity and content validity, as well as feasibility and acceptability, were assessed with a quantitative survey. Construct validity was assessed by comparing the performance of novices, intermediates, and experts on resection time, quality of tumor resection, and overall performance. RESULTS: Of the 21 procedural steps and 17 pitfalls defined in TNA, 13 steps and 8 pitfalls were covered by the Simbla. Participants rated the Simbla's overall realism (face validity) with a score of 8 of 10 (range 6-9). The simulator was judged to be most useful (content validity) for learning eye-hand coordination: score 8 (6-10). All aspects regarding realism and usefulness were rated above the acceptability threshold of 6/10. Intermediates (100%) and experts (96%) considered the Simbla to be a useful educational tool within the urological curriculum. Resection time was longer for novices than for experts (p < 0.05; construct validity). In addition, the overall performance of novices was rated lower compared with intermediates and experts, and novices showed more irradical resections and bladder perforations (all p < 0.05). CONCLUSIONS: The Simbla TURBT simulator is a valid, feasible, and acceptable educational tool for training procedural skills and may be implemented in the urological curriculum to complement learning in clinical practice. TNA is valuable in defining training objectives and evaluating the educational value of a simulator.

Patient Safety Risks of Basic Urological Procedures Performed by Junior and Senior Residents.

OBJECTIVE: To investigate the current performance of urological residents regarding basic urological procedures in relation to patient safety issues and the identification of specific training needs. DESIGN: Observational data of 146 urethrocystoscopies (UCSs), 27 transrectal ultrasounds of the prostate (TRUSs), 38 transrectal ultrasound-guided prostatic biopsies (TRUSPs), and 30 transurethral resections of bladder tumor (TURBTs) were collected. Performance was evaluated using scoring lists including details on completeness of procedural steps, level of independence, time, and the incidence of unintended events. The causal factors contributing to the unintended events were identified by 2 expert urologists and classified according to the recognized PRISMA method. SETTING: This study was performed in 5 teaching hospitals in the Netherlands. PARTICIPANTS: We included 11 junior residents and 5 senior residents in urology in the final study cohort. RESULTS: Senior residents showed a lower degree of completeness in material usage than junior residents did during UCS (p < 0.01) and in preparation, material usage, and procedure during TRUSP (all p < 0.05). In UCS and TURBT, senior residents received significantly less feedback than junior residents did (both p < 0.01). Incidence of unintended events for junior vs senior residents was 11% and 4% in UCS, 0% and 7% in transrectal ultrasound of the prostate, 36% and 62% in TRUSP, and 41% and 23% in TURBT, respectively. Overall, unintended events were mainly caused by human factors, in particular, verification and skills-based issues. CONCLUSION: Present performance of basic urological procedures involves a high percentage of unintended events, especially in TRUSP and TURBT, which are mainly caused by human factors and are a potential threat for patient safety. Junior residents are less independent but more thorough in the performance of UCS and TRUSP than senior residents are. Targeted skills training including assessment should be implemented before privileges for independent practice are granted to reduce the incidence of unintended events and optimize patient safety.

Expertise under the microscope: processing histopathological slides.

OBJECTIVES: Although the obvious goal of training in clinical pathology is to bring forth capable diagnosticians, developmental stages and their characteristics are unknown. This study therefore aims to find expertise-related differences in the processing of histopathological slides using a combination of eye tracking data and verbal data. METHODS: Participants in this study were 13 clinical pathologists (experts), 12 pathology residents (intermediates) and 13 medical students (novices). They diagnosed 10 microscopic images of colon tissue for 2 seconds. Eye movements, the given diagnoses, and the vocabulary used in post hoc verbal explanations were registered. Eye movements were analysed according to changes over trial time and the processing of diagnostically relevant areas. The content analysis of verbal data was based on a categorisation system developed from the literature. RESULTS: Although experts and intermediates showed equal levels of diagnostic accuracy, their visual and cognitive processing differed. Whereas experts relied on their first findings and checked the image further for other abnormalities, intermediates tended to double-check their first findings. In their explanations, experts focused on the typicality of the tissue, whereas intermediates mainly mentioned many specific pathologies. Novices looked less often at the relevant areas and were incomplete, incorrect and inconclusive in their explanations. Their diagnostic accuracy was correspondingly poor. CONCLUSIONS: This study indicates that in the case of intermediates and experts, different visual and cognitive strategies can result in equal levels of diagnostic accuracy. Lessons for training underline the relevance of the distinction between normal and abnormal tissue for novices, especially when the mental rotation of 2-D images is required. Intermediates need to be trained to see deviations in abnormalities. Feedback and an educational design that is specific to these developmental stages might improve training.

Designing simulator-based training: an approach integrating cognitive task analysis and four-component instructional design.

Most studies of simulator-based surgical skills training have focused on the acquisition of psychomotor skills, but surgical procedures are complex tasks requiring both psychomotor and cognitive skills. As skills training is modelled on expert performance consisting partly of unconscious automatic processes that experts are not always able to explicate, simulator developers should collaborate with educational experts and physicians in developing efficient and effective training programmes. This article presents an approach to designing simulator-based skill training comprising cognitive task analysis integrated with instructional design according to the four-component/instructional design model. This theory-driven approach is illustrated by a description of how it was used in the development of simulator-based training for the nephrostomy procedure.