Virtual Reality Warm-up Before Robot-assisted Surgery: A Randomized Controlled Trial.

TRIAL DESIGN: This was a randomized controlled trial. BACKGROUND: Intraoperative errors correlate with surgeon skill and skill declines with intervals of inactivity. The goals of this research were to identify the optimal virtual reality (VR) warm-up curriculum to prime a surgeon's technical skill and validate benefit in the operating room. MATERIALS AND METHODS: Surgeons were randomized to receive six trial sessions of a designated set of VR modules on the da Vinci Skills Simulator to identify optimal VR warm-up curricula to prime technical skill. After performing their curricula, warm-up effect was assessed based on performance on a criterion task. The optimal warm-up curriculum was chosen from the group with the best task time and video review-based technical skill. Robot-assisted surgery-experienced surgeons were then recruited to either receive or not receive warm-up before surgery. Skill in the first 15 min of surgery was assessed by blinded surgeon and crowdworker review as well as tool motion metrics. The intervention was performing VR warm-up before human robot-assisted surgery. Warm-up effect was measured using objective performance metrics and video review using the Global Evaluative Assessment of Robotic Skills tool. Linear mixed effects models with a random intercept for each surgeon and nonparametric modified Friedman tests were used for analysis. RESULTS: The group performing only a Running Suture task on the simulator was on average 31.3 s faster than groups performing other simulation tasks and had the highest Global Evaluative Assessment of Robotic Skills scores from 41 surgeons who participated. This was chosen as the optimal curriculum. Thereafter, 34 surgeons completed 347 surgeries with corresponding video and tool motion data. No statistically significant differences in skill were observed with the warm-up intervention. CONCLUSIONS: We conclude that a robotic VR warm-up before performing the early stages of surgery does not impact the technical skill of the surgeon.

Comparison of Natural Language Processing Rules-based and Machine-learning Systems to Identify Lumbar Spine Imaging Findings Related to Low Back Pain.

RATIONALE AND OBJECTIVES: To evaluate a natural language processing (NLP) system built with open-source tools for identification of lumbar spine imaging findings related to low back pain on magnetic resonance and x-ray radiology reports from four health systems. MATERIALS AND METHODS: We used a limited data set (de-identified except for dates) sampled from lumbar spine imaging reports of a prospectively assembled cohort of adults. From N = 178,333 reports, we randomly selected N = 871 to form a reference-standard dataset, consisting of N = 413 x-ray reports and N = 458 MR reports. Using standardized criteria, four spine experts annotated the presence of 26 findings, where 71 reports were annotated by all four experts and 800 were each annotated by two experts. We calculated inter-rater agreement and finding prevalence from annotated data. We randomly split the annotated data into development (80%) and testing (20%) sets. We developed an NLP system from both rule-based and machine-learned models. We validated the system using accuracy metrics such as sensitivity, specificity, and area under the receiver operating characteristic curve (AUC). RESULTS: The multirater annotated dataset achieved inter-rater agreement of Cohen's kappa > 0.60 (substantial agreement) for 25 of 26 findings, with finding prevalence ranging from 3% to 89%. In the testing sample, rule-based and machine-learned predictions both had comparable average specificity (0.97 and 0.95, respectively). The machine-learned approach had a higher average sensitivity (0.94, compared to 0.83 for rules-based), and a higher overall AUC (0.98, compared to 0.90 for rules-based). CONCLUSIONS: Our NLP system performed well in identifying the 26 lumbar spine findings, as benchmarked by reference-standard annotation by medical experts. Machine-learned models provided substantial gains in model sensitivity with slight loss of specificity, and overall higher AUC.

Using Natural Language Processing of Free-Text Radiology Reports to Identify Type 1 Modic Endplate Changes.

Electronic medical record (EMR) systems provide easy access to radiology reports and offer great potential to support quality improvement efforts and clinical research. Harnessing the full potential of the EMR requires scalable approaches such as natural language processing (NLP) to convert text into variables used for evaluation or analysis. Our goal was to determine the feasibility of using NLP to identify patients with Type 1 Modic endplate changes using clinical reports of magnetic resonance (MR) imaging examinations of the spine. Identifying patients with Type 1 Modic change who may be eligible for clinical trials is important as these findings may be important targets for intervention. Four annotators identified all reports that contained Type 1 Modic change, using N = 458 randomly selected lumbar spine MR reports. We then implemented a rule-based NLP algorithm in Java using regular expressions. The prevalence of Type 1 Modic change in the annotated dataset was 10%. Results were recall (sensitivity) 35/50 = 0.70 (95% confidence interval (C.I.) 0.52-0.82), specificity 404/408 = 0.99 (0.97-1.0), precision (positive predictive value) 35/39 = 0.90 (0.75-0.97), negative predictive value 404/419 = 0.96 (0.94-0.98), and F1-score 0.79 (0.43-1.0). Our evaluation shows the efficacy of rule-based NLP approach for identifying patients with Type 1 Modic change if the emphasis is on identifying only relevant cases with low concern regarding false negatives. As expected, our results show that specificity is higher than recall. This is due to the inherent difficulty of eliciting all possible keywords given the enormous variability of lumbar spine reporting, which decreases recall, while availability of good negation algorithms improves specificity.