Automatic Disease Annotation From Radiology Reports Using Artificial Intelligence Implemented by a Recurrent Neural Network.

OBJECTIVE: Radiology reports are rich resources for biomedical researchers. Before utilization of radiology reports, experts must manually review these reports to identify the categories. In fact, automatically categorizing electronic medical record (EMR) text with key annotation is difficult because it has a free-text format. To address these problems, we developed an automated system for disease annotation. MATERIALS AND METHODS: Reports of musculoskeletal radiography examinations performed from January 1, 2016, through December 31, 2016, were exported from the database of Hanyang University Medical Center. After sentences not written in English and sentences containing typos were excluded, 3032 sentences were included. We built a system that uses a recurrent neural network (RNN) to automatically identify fracture and nonfracture cases as a preliminary study. We trained and tested the system using orthopedic surgeon-classified reports. We evaluated the system for the number of layers in the following two ways: the word error rate of the output sentences and performance as a binary classifier using standard evaluation metrics including accuracy, precision, recall, and F1 score. RESULTS: The word error rate using Levenshtein distance showed the best performance in the three-layer model at 1.03%. The three-layer model also showed the highest overall performance with the highest precision (0.967), recall (0.967), accuracy (0.982), and F1 score (0.967). CONCLUSION: Our results indicate that the RNN-based system has the ability to classify important findings in radiology reports with a high F1 score. We expect that our system can be used in cohort construction such as for retrospective studies because it is efficient for analyzing a large amount of data.

Experimental study on restricting the robotic end-effector inside a lesion for safe telesurgery.

INTRODUCTION: Using an endoscopic telesurgical robot system (ETSRS), the authors propose a strategy for improving the safety of telesurgery by restricting the movement of an end-effector within a lesion. The strategy is validated by phantom model experiments. MATERIAL AND METHODS: The method focused on generation of force feedback and restriction of robotic end-effector movement of ETSRS based on a virtual wall. Collision detection and case classification procedures were used to determine whether the generation of force feedback or restricting the end-effector's movement was continued. The method was implemented in ETSRS and tested using a brain and tofu phantom. RESULTS: Force feedback was well generated proportional to a linear combination of the insertion depth and the velocity of the end-effector of the ETSRS from the surface of the predefined virtual wall. The movement of the end-effector was well limited inside the virtual wall by the method. The virtual wall update was sufficiently fast to check the current surgical situation. The control rate of the entire system was >30 fps so that the method showed acceptable performance in phantom experiments. CONCLUSION: The results show that the strategy allows for well controlled robotic end-effectors inside a predefined virtual wall by the robot itself and an operator through the signal and force feedback.

Construction and verification of a safety region for brain tumor removal with a telesurgical robot system.

BACKGROUND: The authors propose and verify a method for the construction of a safety region for minimally invasive brain tumor removal. The safety region is constructed to avoid damaging normal tissues through the movement of a robotic instrument during brain tumor surgery using a telesurgical robotic system and a small port. MATERIAL AND METHODS: 3 D boundaries of a tumor and a port were generated as a critical wall to avoid invading normal tissues through an image processing algorithm with consideration of a safe margin. Then, fast collision detection between the boundary and the robotic instrument was continuously performed to monitor the movement of the robotic instrument. RESULTS: An experiment was conducted using the prototype of a telesurgical robot system and a hemispherical phantom. A 3 D boundary was generated from the CT images of the phantom with a safe margin of 2.76 mm. The robotic instrument did not penetrate the boundary. CONCLUSION: The experimental result shows that our method can contribute toward safe brain tumor removal with robotic surgery.

Classification of femur fracture in pelvic X-ray images using meta-learned deep neural network.

In the medical field, various studies using artificial intelligence (AI) techniques have been attempted. Numerous attempts have been made to diagnose and classify diseases using image data. However, different forms of fracture exist, and inaccurate results have been confirmed depending on condition at the time of imaging, which is problematic. To overcome this limitation, we present an encoder-decoder structured neural network that utilizes radiology reports as ancillary information at training. This is a type of meta-learning method used to generate sufficiently adequate features for classification. The proposed model learns representation for classification from X-ray images and radiology reports simultaneously. When using a dataset of only 459 cases for algorithm training, the model achieved a favorable performance in a test dataset containing 227 cases (classification accuracy of 86.78% and classification F1 score of 0.867 for fracture or normal classification). This finding demonstrates the potential for deep learning to improve performance and accelerate application of AI in clinical practice.

Three-dimensional analysis of acetabular orientation using a semi-automated algorithm.

Understanding the morphology of the acetabulum is necessary for preoperative evaluation in hip surgery. The purpose of this study was to (1) establish a novel method for measuring three-dimensional (3D) acetabular orientation, (2) quantify the reliability of this method, and (3) describe relevant characteristics of three-dimensional (3D) acetabular orientation among normal Asian subjects. Computed tomography (CT) scans of the pelvis that had been performed for suspected non-musculoskeletal conditions were obtained from 200 subjects (60 males, 140 females). A novel method was developed to measure 3D acetabular orientation with a semi-automatically determined pelvic coordinate system based on the anterior pelvic plane (APP). To quantify the robustness of our method, we analyzed the results obtained from 20 patients at different times and with different raters and pelvic poses in the same CT volume. To determine morphological differences of the acetabulum by age and sex, we analyzed the parameters of 200 CT volumes. Each intraclass correlation coefficient (ICC) values for intra- and inter-observer reliability were over 0.975 and 0.945, demonstrating high reliability. Furthermore, agreement between the angles determined from the original volume and the rotated volume was nearly perfect (ICCs > 0.956). Multiple linear regression analysis with age and sex as covariates indicated that acetabular inclination was not significantly associated with age (p = 0.687) or sex (p = 0.09). There was also no evidence that acetabular anteversion was associated with age (p = 0.383) or sex (p = 0.53). Our method showed excellent reliability for determining acetabular orientation, as it is robust, fast, and easily applicable to larger populations. In addition, the results of the analysis of acetabular orientation by age and sex can be used as a reference in various diagnostic procedures in orthopedics.

A robust method to extract the anterior pelvic plane from CT volume independent of pelvic pose.

PURPOSE: The anterior pelvic plane (APP) is commonly used as a reference plane to assess acetabular orientation. However, conventional methods for determining the APP may not be accurate and are prone to user variability. To overcome these issues, we developed a robust method to accurately extract the APP independent of pelvic pose using three-dimensional pelvic computed tomography (CT). MATERIALS AND METHODS: Twenty-eight studies for suspected nonmusculoskeletal conditions were obtained. The APP was determined by four landmarks that were automatically extracted from user-defined regions of interest (ROIs) with compensation of pelvic pose. The APP defined from these landmarks was quantitatively compared to the APPs determined by an expert and an unskilled. Intraobserver reliability was measured to evaluate the time-interval variability. Finally, we evaluate the robustness of this method to patient posture using an arbitrarily rotated volume. The intraclass correlation coefficients (ICCs) were calculated to determine the interobserver and intraobserver reliabilities. RESULTS: The ICC values for the four landmarks and the APP were similar between the semiautomated method and expert determination (ICC >0.937). The ICC values for intraobserver reliability over time for our method were all 1, demonstrating high reliability. Furthermore, agreement between the parameters determined from the original volume and the rotated volume was nearly perfect. CONCLUSIONS: Our method is a useful measurement tool for the APP as it is robust, and the results were similar to an experienced surgeon's determination. Furthermore, it was independent to the direction of the CT slice and more robust than a measurement by an unskilled.

Real-time Triple-modal Photoacoustic, Ultrasound, and Magnetic Resonance Fusion Imaging of Humans.

Imaging that fuses multiple modes has become a useful tool for diagnosis and therapeutic monitoring. As a next step, real-time fusion imaging has attracted interest as for a tool to guide surgery. One widespread fusion imaging technique in surgery combines real-time ultrasound (US) imaging and pre-acquired magnetic resonance (MR) imaging. However, US imaging visualizes only structural information with relatively low contrast. Here, we present a photoacoustic (PA), US, and MR fusion imaging system which integrates a clinical PA and US imaging system with an optical tracking-based navigation sub-system. Through co-registration of pre-acquired MR and real-time PA/US images, overlaid PA, US, and MR images can be concurrently displayed in real time. We successfully acquired fusion images from a phantom and a blood vessel in a human forearm. This fusion imaging can complementarily delineate the morphological and vascular structure of tissues with good contrast and sensitivity, has a well-established user interface, and can be flexibly integrated with clinical environments. As a novel fusion imaging, the proposed triple-mode imaging can provide comprehensive image guidance in real time, and can potentially assist various surgeries.

Virtual wall-based haptic-guided teleoperated surgical robotic system for single-port brain tumor removal surgery.

This article presents a haptic-guided teleoperation for a tumor removal surgical robotic system, so-called a SIROMAN system. The system was developed in our previous work to make it possible to access tumor tissue, even those that seat deeply inside the brain, and to remove the tissue with full maneuverability. For a safe and accurate operation to remove only tumor tissue completely while minimizing damage to the normal tissue, a virtual wall-based haptic guidance together with a medical image-guided control is proposed and developed. The virtual wall is extracted from preoperative medical images, and the robot is controlled to restrict its motion within the virtual wall using haptic feedback. Coordinate transformation between sub-systems, a collision detection algorithm, and a haptic-guided teleoperation using a virtual wall are described in the context of using SIROMAN. A series of experiments using a simplified virtual wall are performed to evaluate the performance of virtual wall-based haptic-guided teleoperation. With haptic guidance, the accuracy of the robotic manipulator's trajectory is improved by 57% compared to one without. The tissue removal performance is also improved by 21% ( p < 0.05). The experiments show that virtual wall-based haptic guidance provides safer and more accurate tissue removal for single-port brain surgery.