Evaluation of how users interface with holographic augmented reality surgical scenes: Interactive planning MR-Guided prostate biopsies.

BACKGROUND: User interfaces play a vital role in the planning and execution of an interventional procedure. The objective of this study is to investigate the effect of using different user interfaces for planning transrectal robot-assisted MR-guided prostate biopsy (MRgPBx) in an augmented reality (AR) environment. METHOD: End-user studies were conducted by simulating an MRgPBx system with end- and side-firing modes. The information from the system to the operator was rendered on HoloLens as an output interface. Joystick, mouse/keyboard, and holographic menus were used as input interfaces to the system. RESULTS: The studies indicated that using a joystick improved the interactive capacity and enabled operator to plan MRgPBx in less time. It efficiently captures the operator's commands to manipulate the augmented environment representing the state of MRgPBx system. CONCLUSIONS: The study demonstrates an alternative to conventional input interfaces to interact and manipulate an AR environment within the context of MRgPBx planning.

End-user evaluation of software-generated intervention planning environment for transrectal magnetic resonance-guided prostate biopsies.

BACKGROUND: This study presents user evaluation studies to assess the effect of information rendered by an interventional planning software on the operator's ability to plan transrectal magnetic resonance (MR)-guided prostate biopsies using actuated robotic manipulators. METHODS: An intervention planning software was developed based on the clinical workflow followed for MR-guided transrectal prostate biopsies. The software was designed to interface with a generic virtual manipulator and simulate an intervention environment using 2D and 3D scenes. User studies were conducted with urologists using the developed software to plan virtual biopsies. RESULTS: User studies demonstrated that urologists with prior experience in using 3D software completed the planning less time. 3D scenes were required to control all degrees-of-freedom of the manipulator, while 2D scenes were sufficient for planar motion of the manipulator. CONCLUSIONS: The study provides insights on using 2D versus 3D environment from a urologist's perspective for different operational modes of MR-guided prostate biopsy systems.

The Upcoming Role for Nursing and Assistive Robotics: Opportunities and Challenges Ahead.

As an integral part of patient care, nursing is required to constantly adapt to changes in the healthcare system, as well as the wider financial and societal environment. Among the key factors driving these changes is the aging of population. Combined with an existing shortage of nursing and caregiving professionals, accommodating for the patients and elderly needs within hospitals, elderly-care facilities and at a home setting, becomes a societal challenge. Amongst the technological solutions that have evolved in response to these developments, nursing and assistive robotics claim a pivotal role. The objective of the present study is to provide an overview of today's landscape in nursing and assistive robotics, highlighting the benefits associated with adopting such solutions in standard clinical practice. At the same time, to identify existing challenges and limitations that essentially outline the area's future directions. Beyond technological innovation, the manuscript also investigates the end-users' angle, being a crucial parameter in the success of robotics solutions operating within a healthcare environment. In this direction, the results of a survey designed to capture the nursing professionals' perspective toward more informed robotics design and development are presented.

In vivo estimates of axonal stretch and 3D brain deformation during mild head impact.

The rapid deformation of brain tissue in response to head impact can lead to traumatic brain injury. In vivo measurements of brain deformation during non-injurious head impacts are necessary to understand the underlying mechanisms of traumatic brain injury and compare to computational models of brain biomechanics. Using tagged magnetic resonance imaging (MRI), we obtained measurements of three-dimensional strain tensors that resulted from a mild head impact after neck rotation or neck extension. Measurements of maximum principal strain (MPS) peaked shortly after impact, with maximal values of 0.019-0.053 that correlated strongly with peak angular velocity. Subject-specific patterns of MPS were spatially heterogeneous and consistent across subjects for the same motion, though regions of high deformation differed between motions. The largest MPS values were seen in the cortical gray matter and cerebral white matter for neck rotation and the brainstem and cerebellum for neck extension. Axonal fiber strain (Ef) was estimated by combining the strain tensor with diffusion tensor imaging data. As with MPS, patterns of Ef varied spatially within subjects, were similar across subjects within each motion, and showed group differences between motions. Values were highest and most strongly correlated with peak angular velocity in the corpus callosum for neck rotation and in the brainstem for neck extension. The different patterns of brain deformation between head motions highlight potential areas of greater risk of injury between motions at higher loading conditions. Additionally, these experimental measurements can be directly compared to predictions of generic or subject-specific computational models of traumatic brain injury.

A Platform Integrating Acquisition, Reconstruction, Visualization, and Manipulator Control Modules for MRI-Guided Interventions.

This work presents a platform that integrates a customized MRI data acquisition scheme with reconstruction and three-dimensional (3D) visualization modules along with a module for controlling an MRI-compatible robotic device to facilitate the performance of robot-assisted, MRI-guided interventional procedures. Using dynamically-acquired MRI data, the computational framework of the platform generates and updates a 3D model representing the area of the procedure (AoP). To image structures of interest in the AoP that do not reside inside the same or parallel slices, the MRI acquisition scheme was modified to collect a multi-slice set of intraoblique to each other slices; which are termed composing slices. Moreover, this approach interleaves the collection of the composing slices so the same k-space segments of all slices are collected during similar time instances. This time matching of the k-space segments results in spatial matching of the imaged objects in the individual composing slices. The composing slices were used to generate and update the 3D model of the AoP. The MRI acquisition scheme was evaluated with computer simulations and experimental studies. Computer simulations demonstrated that k-space segmentation and time-matched interleaved acquisition of these segments provide spatial matching of the structures imaged with composing slices. Experimental studies used the platform to image the maneuvering of an MRI-compatible manipulator that carried tubing filled with MRI contrast agent. In vivo experimental studies to image the abdomen and contrast enhanced heart on free-breathing subjects without cardiac triggering demonstrated spatial matching of imaged anatomies in the composing planes. The described interventional MRI framework could assist in performing real-time MRI-guided interventions.

Cardiac ultrasonography over 4G wireless networks using a tele-operated robot.

This Letter proposes an end-to-end mobile tele-echography platform using a portable robot for remote cardiac ultrasonography. Performance evaluation investigates the capacity of long-term evolution (LTE) wireless networks to facilitate responsive robot tele-manipulation and real-time ultrasound video streaming that qualifies for clinical practice. Within this context, a thorough video coding standards comparison for cardiac ultrasound applications is performed, using a data set of ten ultrasound videos. Both objective and subjective (clinical) video quality assessment demonstrate that H.264/AVC and high efficiency video coding standards can achieve diagnostically-lossless video quality at bitrates well within the LTE supported data rates. Most importantly, reduced latencies experienced throughout the live tele-echography sessions allow the medical expert to remotely operate the robot in a responsive manner, using the wirelessly communicated cardiac ultrasound video to reach a diagnosis. Based on preliminary results documented in this Letter, the proposed robotised tele-echography platform can provide for reliable, remote diagnosis, achieving comparable quality of experience levels with in-hospital ultrasound examinations.

Medical telerobotic systems: current status and future trends.

Teleoperated medical robotic systems allow procedures such as surgeries, treatments, and diagnoses to be conducted across short or long distances while utilizing wired and/or wireless communication networks. This study presents a systematic review of the relevant literature between the years 2004 and 2015, focusing on medical teleoperated robotic systems which have witnessed tremendous growth over the examined period. A thorough insight of telerobotics systems discussing design concepts, enabling technologies (namely robotic manipulation, telecommunications, and vision systems), and potential applications in clinical practice is provided, while existing limitations and future trends are also highlighted. A representative paradigm of the short-distance case is the da Vinci Surgical System which is described in order to highlight relevant issues. The long-distance telerobotics concept is exemplified through a case study on diagnostic ultrasound scanning. Moreover, the present review provides a classification into short- and long-distance telerobotic systems, depending on the distance from which they are operated. Telerobotic systems are further categorized with respect to their application field. For the reviewed systems are also examined their engineering characteristics and the employed robotics technology. The current status of the field, its significance, the potential, as well as the challenges that lie ahead are thoroughly discussed.

A novel, general-purpose, MR-compatible, manually actuated robotic manipulation system for minimally invasive interventions under direct MRI guidance.

BACKGROUND: Performing minimally invasive interventions under direct MRI guidance offers significant advantages. Required accessibility to the patient inside the MRI scanner is fairly limited, and employment of robotic assistance has been proposed. The development of MR-compatible robotic systems entails engineering challenges related to geometric constraints and the magnetic nature of the scanning environment. METHODS: A novel, general-purpose, MR-compatible robotic manipulation system has been developed for the performance of minimally invasive interventions inside a cylindrical scanner under direct MRI guidance. The system is endowed with five degrees of freedom (DOF), is characterized by a unique kinematics structure and is manually actuated. RESULTS: The prototype system was shown to exhibit the required MR-compatibility characteristics and a task-space positioning ability of approximately 5 mm. Needle targeting testing demonstrated a 93% success rate in acquiring a 5 mm spherical target. Phantom testing was performed inside a 3 T scanner and results are reported for an experimental study simulating MRI-guided, manipulator-assisted, MR arthrography. CONCLUSIONS: Robotic assistance provided by the developed manipulator may effectively facilitate the performance of various MRI-guided, minimally invasive interventions inside a cylindrical scanner.

An approach for preoperative planning and performance of MR-guided interventions demonstrated with a manual manipulator in a 1.5T MRI scanner.

PURPOSE: The aim of this work was to develop and test a general methodology for the planning and performance of robot-assisted, MR-guided interventions. This methodology also includes the employment of software tools with appropriately tailored routines to effectively exploit the capabilities of MRI and address the relevant spatial limitations. METHODS: The described methodology consists of: (1) patient-customized feasibility study that focuses on the geometric limitations imposed by the gantry, the robotic hardware, and interventional tools, as well as the patient; (2) stereotactic preoperative planning for initial positioning of the manipulator and alignment of its end-effector with a selected target; and (3) real-time, intraoperative tool tracking and monitoring of the actual intervention execution. Testing was performed inside a standard 1.5T MRI scanner in which the MR-compatible manipulator is deployed to provide the required access. RESULTS: A volunteer imaging study demonstrates the application of the feasibility stage. A phantom study on needle targeting is also presented, demonstrating the applicability and effectiveness of the proposed preoperative and intraoperative stages of the methodology. For this purpose, a manually actuated, MR-compatible robotic manipulation system was used to accurately acquire a prescribed target through alternative approaching paths. CONCLUSIONS: The methodology presented and experimentally examined allows the effective performance of MR-guided interventions. It is suitable for, but not restricted to, needle-targeting applications assisted by a robotic manipulation system, which can be deployed inside a cylindrical scanner to provide the required access to the patient facilitating real-time guidance and monitoring.

Deformation of the human brain induced by mild angular head acceleration.

Deformation of the human brain was measured in tagged magnetic resonance images (MRI) obtained dynamically during angular acceleration of the head. This study was undertaken to provide quantitative experimental data to illuminate the mechanics of traumatic brain injury (TBI). Mild angular acceleration was imparted to the skull of a human volunteer inside an MR scanner, using a custom MR-compatible device to constrain motion. A grid of MR "tag" lines was applied to the MR images via spatial modulation of magnetization (SPAMM) in a fast gradient echo imaging sequence. Images of the moving brain were obtained dynamically by synchronizing the imaging process with the motion of the head. Deformation of the brain was characterized quantitatively via Lagrangian strain. Consistent patterns of radial-circumferential shear strain occur in the brain, similar to those observed in models of a viscoelastic gel cylinder subjected to angular acceleration. Strain fields in the brain, however, are clearly mediated by the effects of heterogeneity, divisions between regions of the brain (such as the central fissure and central sulcus) and the brain's tethering and suspension system, including the dura mater, falx cerebri, and tentorium membranes.

Magnetic resonance-compatible robotic and mechatronics systems for image-guided interventions and rehabilitation: a review study.

The continuous technological progress of magnetic resonance imaging (MRI), as well as its widespread clinical use as a highly sensitive tool in diagnostics and advanced brain research, has brought a high demand for the development of magnetic resonance (MR)-compatible robotic/mechatronic systems. Revolutionary robots guided by real-time three-dimensional (3-D)-MRI allow reliable and precise minimally invasive interventions with relatively short recovery times. Dedicated robotic interfaces used in conjunction with fMRI allow neuroscientists to investigate the brain mechanisms of manipulation and motor learning, as well as to improve rehabilitation therapies. This paper gives an overview of the motivation, advantages, technical challenges, and existing prototypes for MR-compatible robotic/mechatronic devices.

Performance of interventions with manipulator-driven real-time MR guidance: implementation and initial in vitro tests.

The purpose of this work was to implement and assess the performance of interventions inside a cylindrical magnetic resonance imaging (MRI) scanner with an MR-compatible manipulator system and manipulator-driven real-time MR guidance. The interventional system is based on a seven degree-of-freedom MR-compatible manipulator, which offers man-in-the-loop control either with a graphical user interface or with a master/slave device. The position and the orientation of the interventional tool are sent to an MR scanner for a manipulator-driven dynamic update of the imaging plane to track, visualize and guide the motion of an end-effector. Studies on phantoms were performed with a cylindrical 1.5-T scanner using an operator-managed triggered gradient-recalled echo (GRE) or a computer-managed dynamic True Fast Imaging with Steady Precession (TrueFISP). Targets were acquired with an accuracy of 3.2 mm and with an in-plane path orientation of 2.5 degrees relative to the prescribed one. Path planning, including negotiation of obstacles and needle bending, was successfully performed. The signal-to-noise ratio (SNR) of TrueFISP was 25.3+/-2.1 when the manipulator was idle and was 18.6+/-2.4 during its operation. The SNR of triggered GRE (acquired only when the manipulator was idle) was 61.3+/-1.8. In conclusion, this study shows the feasibility of performing manually directed interventions inside cylindrical MR scanners with real-time MRI.

Fast and efficient radiological interventions via a graphical user interface commanded magnetic resonance compatible robotic device.

The graphical user interface for an MR compatible robotic device has the capability of displaying oblique MR slices in 2D and a 3D virtual environment along with the representation of the robotic arm in order to swiftly complete the intervention. Using the advantages of the MR modality the device saves time and effort, is safer for the medical staff and is more comfortable for the patient.

A prototype manipulator for magnetic resonance-guided interventions inside standard cylindrical magnetic resonance imaging scanners.

The aim of this work is to develop a remotely controlled manipulator to perform minimally invasive diagnostic and therapeutic interventions in the abdominal and thoracic cavities, with real-time magnetic resonance imaging (MRI) guidance inside clinical cylindrical MR scanners. The manipulator is composed of a three degree of freedom Cartesian motion system, which resides outside the gantry of the scanner, and serves as the holder and global positioner of a three degree of freedom arm which extends inside the gantry of the scanner At its distal end, the arm's end-effector can carry an interventional tool such as a biopsy needle, which can be advanced to a desired depth by means of a seventh degree of freedom. These seven degrees of freedom, provided by the entire assembly, offer extended manipulability to the device and a wide envelope of operation to the user, who can select a trajectory suitable for the procedure. The device is constructed of nonmagnetic and nonconductive fiberglass, and carbon fiber composite materials, to minimize artifacts and distortion on the MR images as well as eliminate effects on its operation from the high magnetic field and the fast switching magnetic field gradients used in MR imaging. A user interface was developed for man-in-the-loop control of the device using real-time MR images. The user interface fuses all sensor signals (MR and manipulator information) in a visualization, planning, and control command environment. Path planning is performed with graphical tools for setting the trajectory of insertion of the interventional tool using multislice and/or three dimensional MR images which are refreshed in real time. The device control is performed with an embedded computer which runs real-time control software. The manipulator compatibility with the MR environment and image-guided operation was tested on a 1.5 T MR scanner.