Laypersons versus experienced surgeons in assessing simulated robot-assisted radical prostatectomy.

BACKGROUND: Feedback is important for surgical trainees but it can be biased and time-consuming. We examined crowd-sourced assessment as an alternative to experienced surgeons' assessment of robot-assisted radical prostatectomy (RARP). METHODS: We used video recordings (n = 45) of three RARP modules on the RobotiX, Simbionix simulator from a previous study in a blinded comparative assessment study. A group of crowd workers (CWs) and two experienced RARP surgeons (ESs) evaluated all videos with the modified Global Evaluative Assessment of Robotic Surgery (mGEARS). RESULTS: One hundred forty-nine CWs performed 1490 video ratings. Internal consistency reliability was high (0.94). Inter-rater reliability and test-retest reliability were low for CWs (0.29 and 0.39) and moderate for ESs (0.61 and 0.68). In an Analysis of Variance (ANOVA) test, CWs could not discriminate between the skill level of the surgeons (p = 0.03-0.89), whereas ES could (p = 0.034). CONCLUSION: We found very low agreement between the assessments of CWs and ESs when they assessed robot-assisted radical prostatectomies. As opposed to ESs, CWs could not discriminate between surgical experience using the mGEARS ratings or when asked if they wanted the surgeons to perform their robotic surgery.

Kidney stone disease.

Kidney stone disease is rapidly increasing with a strong relationship to metabolic syndrome. This review gives a brief overview of the current state and current treatment modalities. Increasing use of CT and ultrasound scans leads to increased diagnosis of asymptomatic kidney stones, which rarely require treatment. The trend in stone treatment goes towards endoscopic lithotripsy which together with ESWL enables a personalised approach. Obstructive stones with infection require urgent intervention to reduce mortality. Increased fluid intake, dietary changes as well as potassium citrate supplements are the most important elements in stone prevention in the common idiopathic stone disease.

Surgical experience is predictive for bladder tumour resection quality.

OBJECTIVES: To assess the resection quality of transurethral bladder tumour resection (TURBT) and the association to surgeon experience depending on the presence of detrusor muscle. METHODS: A retrospective study on 640 TURBT procedures performed at Zealand University Hospital, Denmark, from 1 January 2015 - 31 December 2016. Data included patient characteristics, procedure type, surgeon category, supervisor presence, surgical report data, pathological data, complications data and recurrence data. Analysis was performed using simple and multiple logistic regression on the association between surgeon experience and the presence of detrusor muscle in resected tissue from TURBT. RESULTS: Supervised junior residents had significant lower detrusor muscle presence (73%) compared with consultants (83%) (OR = 0.4, 95% CI = 0.21-0.83). Limitations were the retrospective design and the diversity of included TURBT. CONCLUSIONS: It was found that surgical experience predicts detrusor muscle presence and supervised junior residents performing TURBT on patients resulted in less detrusor muscle than consultants.

National Implementation of Simulator Training Improves Transurethral Resection of Bladder Tumours in Patients.

Background: Transurethral resection of bladder tumours (TURBT) is the initial diagnostic treatment for patients with bladder cancer. TURBT is not an easy procedure to master and simulator training may play a role in improving the learning curve. Objective: To implement a national training programme for simulation-based mastery learning in TURBT and explore operating theatre performance after training. Design setting and participants: From June 2019 to March 2021, 31 doctors at urology departments in Denmark performed two pretraining TURBT procedures on patients, followed by proficiency-based mastery learning on a virtual reality simulator and then two post-training TURBTs on patients. Outcome measurements and statistical analyses: Operating theatre performances were video-recorded and assessed by two independent, blinded raters using the Objective Structured Assessment for Transurethral Resection of Bladder Tumours Skills (OSATURBS) assessment tool. Paired-sample t tests were used to compare pretraining and post-training analyses and independent t tests for between-group comparisons. This trial is registered at ClinicalTrials.gov as NCT03864302. Results and limitations: Before training, novices had significantly lower performance scores in comparison to those with intermediate experience (p = 0.017) and experienced doctors (p < 0.001). After training, novices significantly improved their clinical performance score (from 11.4 to 17.1; p = 0.049, n = 10). Those with intermediate experience and experienced doctors did not benefit significantly from simulator training (p = 0.9 and p = 0.8, respectively). Conclusions: Novices improved their TURBT performance in the operating theatre after completing a proficiency-based training programme on a virtual reality simulator. Patient summary: We trained surgeons in an operation to remove bladder tumours using a virtual reality simulator. Novice doctors improved their performance significantly after the training, but the training effects for more experienced doctors were minimal. Therefore, we suggest the introduction of mandatory simulator training in the residency programme for urologists.

Validation of a Novel Assessment Tool Identifying Proficiency in Transurethral Bladder Tumor Resection: The OSATURBS Assessment Tool.

Background: Competence in transurethral resection of bladder tumors (TURB) is critical in bladder cancer management and should be ensured before independent practice. Objective: To develop an assessment tool for TURB and explore validity evidence in a clinical context. Design, Setting, and Participants: From July 2019 to March 2021, a total of 33 volunteer doctors from three hospitals were included after exemption from the regional ethics committee (REG-008-2018). Participants performed two TURB procedures on patients with bladder tumors. A newly developed assessment tool (Objective Structured Assessment for Transurethral Resection of Bladder Tumors Skills, OSATURBS) was used for direct observation assessment (DOA), self-assessment (SA), and blinded video assessment (VA). Outcome Measurements and Statistical Analysis: Cronbach's alpha and Pearson's r were calculated for across items internal consistency reliability, inter-rater reliability, and test-retest reliability. Correlation between OSATURBS scores and the operative experience was calculated with Pearson's r and a pass/fail score was established. Differences in assessment scores were explored with paired t-test and independent samples t-test. Results and Limitations: The internal consistency reliability across items Cronbach's alpha was 0.94 (n = 260, p < 0.001). Inter-rater reliability was 0.80 (n = 64, p < 0.001). Test-retest correlation was high, r = 0.71 (n = 32, p < 0.001). Relationship with TURB experience was high, r = 0.71 (n = 32, p < 0.001). Pass/fail score was 19 points. DOAs were strongly correlated with video ratings (r = 0.85, p < 0.001) but with a significant social bias with lower scores for inexperienced and higher scores for experienced participants. Participants tended to overestimate their own performances. Conclusions: OSATURBS tool for TURB can be used for assessment of surgical proficiency in the clinical setting. DOA and SA are biased, and blinded VA of TURB performances is advised. Clinical Trials NCT03864302.

Validation of a Novel Simulation-Based Test in Robot-Assisted Radical Prostatectomy.

Purpose: To investigate validity evidence for a simulator-based test in robot-assisted radical prostatectomy (RARP). Materials and Methods: The test consisted of three modules on the RobotiX Mentor VR-simulator: Bladder Neck Dissection, Neurovascular Bundle Dissection, and Ureterovesical Anastomosis. Validity evidence was investigated by using Messick's framework by including doctors with different RARP experience: novices (who had assisted for RARP), intermediates (robotic surgeons, but not RARP surgeons), or experienced (RARP surgeons). The simulator metrics were analyzed, and Cronbach's alpha and generalizability theory were used to explore reliability. Intergroup comparisons were done with mixed-model, repeated measurement analysis of variance and the correlation between the number of robotic procedures and the mean test score were examined. A pass/fail score was established by using the contrasting groups' method. Results: Ten novices, 11 intermediates, and 6 experienced RARP surgeons were included. Six metrics could discriminate between groups and showed acceptable internal consistency reliability, Cronbach's alpha = 0.49, p < 0.001. Test-retest reliability was 0.75, 0.85, and 0.90 for one, two, and three repetitions of tests, respectively. Six metrics were combined into a simulator score that could discriminate between all three groups, p = 0.002, p < 0.001, and p = 0.029 for novices vs intermediates, novices vs experienced, and intermediates vs experienced, respectively. Total number of robotic operations and the mean score of the three repetitions were significantly correlated, Pearson's r = 0.74, p < 0.001. Conclusion: This study provides validity evidence for a simulator-based test in RARP. We determined a pass/fail level that can be used to ensure competency before proceeding to supervised clinical training.

Simulation-based training for flexible cystoscopy - A randomized trial comparing two approaches.

BACKGROUND: Simulation-based training allows trainees to experiment during training and end-of-training tests could increase motivation and retention. The aim of this trial was to determine if a simulation-based training program including directed self-regulated learning and post-testing improved clinical outcomes compared to a traditional simulation-based training program. METHODS: A randomized trial was conducted involving 32 participants without prior experience in endoscopic procedures. The intervention group practiced independently in a simulation centre and got a post-test whereas the control group received traditional instructions and demonstrations before being allowed to practice. Three weeks after the intervention the participants performed cystoscopies on two consecutive patients. Clinical performance was assessed using a global rating scale (GRS) with established evidence of validity. Independent samples t-test, Cronbach's α, Pearson's r, and paired samples t-test were used for statistical analysis. RESULTS: Twenty-five participants performed two cystoscopies on patients. There was no significant difference between the two study groups with respect to mean GRS of performance (p = 0.63, 95 % CI; -2.4-3.9). The internal consistency of the global rating scale was high, Cronbach's α = 0.91. Participants from both study groups demonstrated significant improvement between the first and second clinical procedures (p = 0.004, 95 % CI, 0.8-3.5). Eight (32%) and 15 (60%) participants demonstrated adequate clinical skills in their first and second procedure, respectively. CONCLUSIONS: No significant differences were found on the clinical transfer when comparing the two programs. Neither of our training programs was able to ensure consistent, competent performance on patients and this finding could serve as an important argument for simulation-based mastery learning where all training continues until a pre-defined level of proficiency is met. TRIAL REGISTRATIONS: The trial was submitted before enrolment of participants to the Regional Scientific Ethics Committee of the Capital Region which established that ethical approval was not necessary (H-4-2014-122). The trial was registered at Clinicaltrials.gov (NCT02411747).

Development and validation of a simulator-based test in transurethral resection of bladder tumours (TURBEST).

Objectives: A prospective observational trial to develop and gather validity evidence using Messick's framework for a simulator-based test in TURB. Methods: Forty-nine doctors were recruited from urology departments (Herlev/Gentofte University Hospital, Rigshospitalet Copenhagen University Hospital and Zealand University Hospital Roskilde) and enrolled from April to September 2018. The TURB Mentor™ virtual reality (VR) simulator was assessed at an expert meeting selecting clinically relevant cases and metrics. Test sessions were done on identical simulators at two university hospitals in Denmark. All participants performed three TURB procedures on the VR simulator. Simulator metrics were analysed with analysis of variance (ANOVA) and metrics with the ability to discriminate between groups were combined in a total simulator score. Finally, a pass/fail score was identified using the contrasting groups' method.Results: Eleven simulator metrics were found eligible and four had significant discrimination ability between competency levels: resected pathology (%) (p = 0.008); cutting in bladder wall (n) (p = 0.004); time (s) (p = 0.034); and inspection of the bladder wall (%) (p = 0.002). The internal structure of the total simulator score [(resected pathology*inspection of the bladder wall)/time] was high with the intraclass correlation coefficient, Cronbach's alpha = 0.85. The mean total simulator score was significantly lower in the novice group than in the intermediate, 15.9 and 25.6, respectively (mean difference = 9.7, p = 0.011) and experienced group, 30.6 (mean difference = 14.7, p < 0.001). A pass/fail score of 22 was identified.Conclusion: We found validity evidence for a newly developed VR simulator-based test and establised a pass/fail score identifying surgical skills in TURB. The TURBEST test can be used in a proficiency-based TURB simulator training programme for accreditation prior to supervised procedures on patients.

Hand-Assisted Laparoscopic Partial Nephrectomy for Large Renal Carcinoma with Succinate Dehydrogenase Deficiency.

Background: Germline mutations in succinate dehydrogenase (SDH) are associated with multifocal cancers: pituitary gland tumors, pheochromocytomas, paragangliomas, gastrointestinal stromal tumors, and renal-cell carcinomas (RCCs). SDH-deficient renal-cell carcinoma (SDH-RCC) was first identified in 2004 as an inherited kidney cancer with mutations in the SDH gene. SDH consists of A, B, C, and D units. Mutation in the SDHB gene is the most common mutation in SDH-deficient RCCs. Case Presentation: We report a case of a 51-year-old healthy man diagnosed with SDHB germline mutation and RCCs. Positron emission tomography/computed tomography (PET/CT) showed a 12 cm tumor in the upper pole of the left kidney. A hand-assisted laparoscopic partial nephrectomy was performed and the histopathology of the tumor showed SDH-deficient RCC with clear surgical margins. Six months after the initial presentation, the patient had a slightly impaired renal function and was disease-free on PET/CT. Conclusion: Patients with SDH-deficient RCC are at risk of multifocal kidney tumors and should be offered lifelong follow-up. To preserve the renal function, nephron-sparing surgery is the choice of treatment when feasible, regardless of tumor size.

Direct Observation vs. Video-Based Assessment in Flexible Cystoscopy.

OBJECTIVE: Direct observation in assessment of clinical skills is prone to bias, demands the observer to be present at a certain location at a specific time, and is time-consuming. Video-based assessment could remove the risk of bias, increase flexibility, and reduce the time spent on assessment. This study investigated if video-based assessment was a reliable tool for cystoscopy and if direct observers were prone to bias compared with video-raters. DESIGN: This study was a blinded observational trial. Twenty medical students and 9 urologists were recorded during 2 cystoscopies and rated by a direct observer and subsequently by 2 blinded video-raters on a global rating scale (GRS) for cystoscopy. Both intrarater and interrater reliability were explored. Furthermore, direct observer bias was explored by a paired samples t-test. RESULTS: Intrarater reliability calculated by Pearson's r was 0.86. Interrater reliability was 0.74 for single measure and 0.85 for average measures. A hawk-dove effect was seen between the 2 raters. Direct observer bias was detected when comparing direct observer scores to the assessment by an independent video-rater (p < 0.001). CONCLUSION: This study found that video-based assessment was a reliable tool for cystoscopy with 2 video-raters. There was a significant bias when comparing direct observation with blinded video-based assessment.

Nonspecialist Raters Can Provide Reliable Assessments of Procedural Skills.

BACKGROUND: Competency-based learning has become a crucial component in medical education. Despite the advantages of competency-based learning, there are still challenges that need to be addressed. Currently, the common perception is that specialist assessment is needed for evaluating procedural skills which is difficult owing to the limited availability of faculty time. The aim of this study was to explore the validity of assessments of video recorded procedures performed by nonspecialist raters. METHODS: This study was a blinded observational trial. Twenty-three novices (senior medical students) and 9 experienced doctors were video recorded while each performing 2 flexible cystoscopies on patients. The recordings were anonymized and placed in random order and then rated by 2 experienced cystoscopists (specialist raters) and 2 medical students (nonspecialist raters). Flexible cystoscopy was chosen as it is a simple procedural skill that is crucial to master in a resident urology program. RESULTS: The internal consistency of assessments was high, Cronbach's α = 0.93 and 0.95 for nonspecialist and specialist raters, respectively (p < 0.001 for both correlations). The interrater reliability was significant (p < 0.001) with a Pearson's correlation of 0.77 for the nonspecialists and 0.75 for the specialists. The test-retest reliability showed the biggest difference between the 2 groups, 0.59 and 0.38 for the nonspecialist raters and the specialist raters, respectively (p < 0.001). CONCLUSION: Our study suggests that nonspecialist raters can provide reliable and valid assessments of video recorded cystoscopies. This could make mastery learning and competency-based education more feasible.

Expert Involvement and Adherence to Medical Evidence in Medical Mobile Phone Apps: A Systematic Review.

BACKGROUND: Both clinicians and patients use medical mobile phone apps. Anyone can publish medical apps, which leads to contents with variable quality that may have a serious impact on human lives. We herein provide an overview of the prevalence of expert involvement in app development and whether or not app contents adhere to current medical evidence. OBJECTIVE: To systematically review studies evaluating expert involvement or adherence of app content to medical evidence in medical mobile phone apps. METHODS: We systematically searched 3 databases (PubMed, The Cochrane Library, and EMBASE), and included studies evaluating expert involvement or adherence of app content to medical evidence in medical mobile phone apps. Two authors performed data extraction independently. Qualitative analysis of the included studies was performed. RESULTS: Based on inclusion criteria, 52 studies were included in this review. These studies assessed a total of 6520 apps. Studies dealt with a variety of medical specialties and topics. As much as 28 studies assessed expert involvement, which was found in 9-67% of the assessed apps. Thirty studies (including 6 studies that also assessed expert involvement) assessed adherence of app content to current medical evidence. Thirteen studies found that 10-87% of the assessed apps adhered fully to the compared evidence (published studies, recommendations, and guidelines). Seventeen studies found that none of the assessed apps (n=2237) adhered fully to the compared evidence. CONCLUSIONS: Most medical mobile phone apps lack expert involvement and do not adhere to relevant medical evidence.