Automatic patient positioning based on robot rotational workspace for extended reality.

PURPOSE: Understanding the properties and aspects of the robotic system is essential to a successful medical intervention, as different capabilities and limits characterize each. Robot positioning is a crucial step in the surgical setup that ensures proper reachability to the desired port locations and facilitates docking procedures. This very demanding task requires much experience to master, especially with multiple trocars, increasing the barrier of entry for surgeons in training. METHODS: Previously, we demonstrated an Augmented Reality-based system to visualize the rotational workspace of the robotic system and proved it helps the surgical staff to optimize patient positioning for single-port interventions. In this work, we implemented a new algorithm to allow for an automatic, real-time robotic arm positioning for multiple ports. RESULTS: Our system, based on the rotational workspace data of the robotic arm and the set of trocar locations, can calculate the optimal position of the robotic arm in milliseconds for the positional and in seconds for the rotational workspace in virtual and augmented reality setups. CONCLUSIONS: Following the previous work, we extended our system to support multiple ports to cover a broader range of surgical procedures and introduced the automatic positioning component. Our solution can decrease the surgical setup time and eliminate the need to repositioning the robot mid-procedure and is suitable both for the preoperative planning step using VR and in the operating room-running on an AR headset.

Validation of collaborative cyberspace virtual reality oculometry enhanced with near real-time spatial audio.

Currently, most medical image data, such as optical coherence tomography (OCT) images, are displayed in two dimensions on a computer screen. Advances in computer information technology have contributed to the growing storage of these data in electronic form. However, the data are usually processed only locally on site. To overcome such hurdles, a cyberspace virtual reality (csVR) application was validated, in which interactive OCT data were presented simultaneously to geographically distant sites (Lucerne, London, and Barcelona) where three graders independently measured the ocular csVR OCT diameters. A total of 109 objects were measured, each three times, resulting in a total of 327 csVR measurements. A minor mean absolute difference of 5.3 µm was found among the 3 measurements of an object (standard deviation 4.2 µm, coefficient of variation 0.3% with respect to the mean object size). Despite the 5 h of online work, csVR was well tolerated and safe. Digital high-resolution OCT data can be remotely and collaboratively processed in csVR. With csVR, measurements and actions enhanced with spatial audio communication can be made consistently in near real time, even if the users are situated geographically far apart. The proposed visuo-auditory framework has the potential to further boost the convenience of digital medicine toward csVR precision and collaborative medicine.

Force quantification and simulation of pedicle screw tract palpation using direct visuo-haptic volume rendering.

PURPOSE: We present a feasibility study for the visuo-haptic simulation of pedicle screw tract palpation in virtual reality, using an approach that requires no manual processing or segmentation of the volumetric medical data set. METHODS: In a first experiment, we quantified the forces and torques present during the palpation of a pedicle screw tract in a real boar vertebra. We equipped a ball-tipped pedicle probe with a 6-axis force/torque sensor and a motion capture marker cluster. We simultaneously recorded the pose of the probe relative to the vertebra and measured the generated forces and torques during palpation. This allowed us replaying the recorded palpation movements in our simulator and to fine-tune the haptic rendering to approximate the measured forces and torques. In a second experiment, we asked two neurosurgeons to palpate a virtual version of the same vertebra in our simulator, while we logged the forces and torques sent to the haptic device. RESULTS: In the experiments with the real vertebra, the maximum measured force along the longitudinal axis of the probe was 7.78 N and the maximum measured bending torque was 0.13 Nm. In an offline simulation of the motion of the pedicle probe recorded during the palpation of a real pedicle screw tract, our approach generated forces and torques that were similar in magnitude and progression to the measured ones. When surgeons tested our simulator, the distributions of the computed forces and torques were similar to the measured ones; however, higher forces and torques occurred more frequently. CONCLUSIONS: We demonstrated the suitability of direct visual and haptic volume rendering to simulate a specific surgical procedure. Our approach of fine-tuning the simulation by measuring the forces and torques that are prevalent while palpating a real vertebra produced promising results.

Validation of virtual reality orbitometry bridges digital and physical worlds.

Clinical science and medical imaging technology are traditionally displayed in two dimensions (2D) on a computer monitor. In contrast, three-dimensional (3D) virtual reality (VR) expands the realm of 2D image visualization, enabling an immersive VR experience with unhindered spatial interaction by the user. Thus far, analysis of data extracted from VR applications was mainly qualitative. In this study, we enhance VR and provide evidence for quantitative VR research by validating digital VR display of computed tomography (CT) data of the orbit. Volumetric CT data were transferred and rendered into a VR environment. Subsequently, seven graders performed repeated and blinded diameter measurements. The intergrader variability of the measurements in VR was much lower compared to measurements in the physical world and measurements were reasonably consistent with their corresponding elements in the real context. The overall VR measurements were 5.49% higher. As such, this study attests the ability of VR to provide similar quantitative data alongside the added benefit of VR interfaces. VR entails a lot of potential for the future research in ophthalmology and beyond in any scientific field that uses three-dimensional data.