Visualization modality for augmented reality guidance of in-depth tumour enucleation procedures.

Recent research studies reported that the employment of wearable augmented reality (AR) systems such as head-mounted displays for the in situ visualisation of ultrasound (US) images can improve the outcomes of US-guided biopsies through reduced procedure completion times and improved accuracy. Here, the authors continue in the direction of recent developments and present the first AR system for guiding an in-depth tumour enucleation procedure under US guidance. The system features an innovative visualisation modality with cutting trajectories that 'sink' into the tissue according to the depth reached by the electric scalpel, tracked in real-time, and a virtual-to-virtual alignment between the scalpel's tip and the trajectory. The system has high accuracy in estimating the scalpel's tip position (mean depth error of 0.4 mm and mean radial error of 1.34 mm). Furthermore, we demonstrated with a preliminary user study that our system allowed us to successfully guide an in-depth tumour enucleation procedure (i.e. preserving the safety margin around the lesion).

Registration Sanity Check for AR-guided Surgical Interventions: Experience From Head and Face Surgery.

Achieving and maintaining proper image registration accuracy is an open challenge of image-guided surgery. This work explores and assesses the efficacy of a registration sanity check method for augmented reality-guided navigation (AR-RSC), based on the visual inspection of virtual 3D models of landmarks. We analyze the AR-RSC sensitivity and specificity by recruiting 36 subjects to assess the registration accuracy of a set of 114 AR images generated from camera images acquired during an AR-guided orthognathic intervention. Translational or rotational errors of known magnitude up to ±1.5 mm/±15.5°, were artificially added to the image set in order to simulate different registration errors. This study analyses the performance of AR-RSC when varying (1) the virtual models selected for misalignment evaluation (e. g., the model of brackets, incisor teeth, and gingival margins in our experiment), (2) the type (translation/rotation) of registration error, and (3) the level of user experience in using AR technologies. Results show that: 1) the sensitivity and specificity of the AR-RSC depends on the virtual models (globally, a median true positive rate of up to 79.2% was reached with brackets, and a median true negative rate of up to 64.3% with incisor teeth), 2) there are error components that are more difficult to identify visually, 3) the level of user experience does not affect the method. In conclusion, the proposed AR-RSC, tested also in the operating room, could represent an efficient method to monitor and optimize the registration accuracy during the intervention, but special attention should be paid to the selection of the AR data chosen for the visual inspection of the registration accuracy.

Preliminary Results of Preoperative Planning Using 3D Printing and Augmented Reality in Cryotherapy Treatment of Giant Cell Tumor of Bone-CRIO2AR Project.

Giant Cell Tumor of Bone is a benign tumor with high local aggressive expansion, which, in rare cases, spreads metastasis. Surgical treatment, which often consists of wide curettage to reduce recurrence risk, can lower the quality of life for those affected. Along with aggressive surgery, adjuvant intraoperative techniques have been implemented such as PMMA and cryotherapy. One of the most widely used cryotherapy techniques involves the use of probes to generate ice balls, which have been scientifically shown to have various impacts on the tumor. Although this has been acknowledged, no one has yet tested a way to accurately plan the positioning of cryotherapy probes before surgery, according to the research conducted by the authors. CRIO2AR is a randomized clinical prospective ongoing study by which it will be experimented via preoperative planning of ice probes placement using AR and 3D printing technologies. By studying a single clinical case with these technologies, the surgeon gains better awareness of patient's anatomy and tumor localization. Preliminary results are shown in the article. The first results are confirming that these technologies are applicable in clinical practice. Secondly, preoperative planning is proving to be reliable, easily replicable, and useful for the surgeon.

Analysis of the Robotic-Based In Situ Bioprinting Workflow for the Regeneration of Damaged Tissues through a Case Study.

This study aims to critically analyse the workflow of the in situ bioprinting procedure, presenting a simulated neurosurgical case study, based on a real traumatic event, for collecting quantitative data in support of this innovative approach. After a traumatic event involving the head, bone fragments may have to be removed and a replacement implant placed through a highly demanding surgical procedure in terms of surgeon dexterity. A promising alternative to the current surgical technique is the use of a robotic arm to deposit the biomaterials directly onto the damaged site of the patient following a planned curved surface, which can be designed pre-operatively. Here we achieved an accurate planning-patient registration through pre-operative fiducial markers positioned around the surgical area, reconstructed starting from computed tomography images. Exploiting the availability of multiple degrees of freedom for the regeneration of complex and also overhanging parts typical of anatomical defects, in this work the robotic platform IMAGObot was used to regenerate a cranial defect on a patient-specific phantom. The in situ bioprinting process was then successfully performed showing the great potential of this innovative technology in the field of cranial surgery. In particular, the accuracy of the deposition process was quantified, as well as the duration of the whole procedure was compared to a standard surgical practice. Further investigations include a biological characterisation over time of the printed construct as well as an in vitro and in vivo analysis of the proposed approach, to better analyse the biomaterial performances in terms of osteo-integration with the native tissue.

Head-Mounted Projector for Manual Precision Tasks: Performance Assessment.

The growing interest in augmented reality applications has led to an in-depth look at the performance of head-mounted displays and their testing in numerous domains. Other devices for augmenting the real world with virtual information are presented less frequently and usually focus on the description of the device rather than on its performance analysis. This is the case of projected augmented reality, which, compared to head-worn AR displays, offers the advantages of being simultaneously accessible by multiple users whilst preserving user awareness of the environment and feeling of immersion. This work provides a general evaluation of a custom-made head-mounted projector for the aid of precision manual tasks through an experimental protocol designed for investigating spatial and temporal registration and their combination. The results of the tests show that the accuracy (0.6±0.1 mm of spatial registration error) and motion-to-photon latency (113±12 ms) make the proposed solution suitable for guiding precision tasks.

Magic Leap 1 versus Microsoft HoloLens 2 for the Visualization of 3D Content Obtained from Radiological Images.

The adoption of extended reality solutions is growing rapidly in the healthcare world. Augmented reality (AR) and virtual reality (VR) interfaces can bring advantages in various medical-health sectors; it is thus not surprising that the medical MR market is among the fastest-growing ones. The present study reports on a comparison between two of the most popular MR head-mounted displays, Magic Leap 1 and Microsoft HoloLens 2, for the visualization of 3D medical imaging data. We evaluate the functionalities and performance of both devices through a user-study in which surgeons and residents assessed the visualization of 3D computer-generated anatomical models. The digital content is obtained through a dedicated medical imaging suite (Verima imaging suite) developed by the Italian start-up company (Witapp s.r.l.). According to our performance analysis in terms of frame rate, there are no significant differences between the two devices. The surgical staff expressed a clear preference for Magic Leap 1, particularly for the better visualization quality and the ease of interaction with the 3D virtual content. Nonetheless, even though the results of the questionnaire were slightly more positive for Magic Leap 1, the spatial understanding of the 3D anatomical model in terms of depth relations and spatial arrangement was positively evaluated for both devices.

Semiautonomous Robotic Manipulator for Minimally Invasive Aortic Valve Replacement.

Aortic valve surgery is the preferred procedure for replacing a damaged valve with an artificial one. The ValveTech robotic platform comprises a flexible articulated manipulator and surgical interface supporting the effective delivery of an artificial valve by teleoperation and endoscopic vision. This article presents our recent work on force-perceptive, safe, semiautonomous navigation of the ValveTech platform prior to valve implantation. First, we present a force observer that transfers forces from the manipulator body and tip to a haptic interface. Second, we demonstrate how hybrid forward/inverse mechanics, together with endoscopic visual servoing, lead to autonomous valve positioning. Benchtop experiments and an artificial phantom quantify the performance of the developed robot controller and navigator. Valves can be autonomously delivered with a 2.0±0.5 mm position error and a minimal misalignment of 3.4±0.9°. The hybrid force/shape observer (FSO) algorithm was able to predict distributed external forces on the articulated manipulator body with an average error of 0.09 N. FSO can also estimate loads on the tip with an average accuracy of 3.3%. The presented system can lead to better patient care, delivery outcome, and surgeon comfort during aortic valve surgery, without requiring sensorization of the robot tip, and therefore obviating miniaturization constraints.

Corrigendum: Optical See-Through Head-Mounted Displays With Short Focal Distance: Conditions for Mitigating Parallax-Related Registration Error.

[This corrects the article DOI: 10.3389/frobt.2020.572001.].

Errata to "Parallax Free Registration for Augmented Reality Optical See-Through Displays in the Peripersonal Space" [1] (DOI: 10.1109/TVCG.2020.3021534).

In the original article, there was a mistake in the content of Table 2 page 8 column 1 as published. The values of the mean and standard deviation of the virtual-to-real overlay error in visual angles, which are reported for different checkerboard distances, are to be corrected. Due to a typing error within the data analysis code, we mistakenly considered an erroneous value of the average angular resolution for the eye-replacement camera. This scale factor is used to pass from the original registration errors (expressed in pixel) to the angular registration errors (in arcmin). The value of the average angular resolution is $\approx 2.67$≈2.67 arcmin/pixel. The corrected Table 2 appears below.

Brain Tumor and Augmented Reality: New Technologies for the Future.

In recent years, huge progress has been made in the management of brain tumors, due to the availability of imaging devices, which provide fundamental anatomical and pathological information not only for diagnostic purposes [...].

Parallax Free Registration for Augmented Reality Optical See-Through Displays in the Peripersonal Space.

Egocentric augmented reality (AR) interfaces are quickly becoming a key asset for assisting high precision activities in the peripersonal space in several application fields. In these applications, accurate and robust registration of computer-generated information to the real scene is hard to achieve with traditional Optical See-Through (OST) displays given that it relies on the accurate calibration of the combined eye-display projection model. The calibration is required to efficiently estimate the projection parameters of the pinhole model that encapsulate the optical features of the display and whose values vary according to the position of the user's eye. In this article, we describe an approach that prevents any parallax-related AR misregistration at a pre-defined working distance in OST displays with infinity focus; our strategy relies on the use of a magnifier placed in front of the OST display, and features a proper parameterization of the virtual rendering camera achieved through a dedicated calibration procedure that accounts for the contribution of the magnifier. We model the registration error due to the viewpoint parallax outside the ideal working distance. Finally, we validate our strategy on a OST display, and we show that sub-millimetric registration accuracy can be achieved for working distances of ±100 mm around the focal length of the magnifier.

Augmented Reality-Assisted Craniotomy for Parasagittal and Convexity En Plaque Meningiomas and Custom-Made Cranio-Plasty: A Preliminary Laboratory Report.

BACKGROUND: This report discusses the utility of a wearable augmented reality platform in neurosurgery for parasagittal and convexity en plaque meningiomas with bone flap removal and custom-made cranioplasty. METHODS: A real patient with en plaque cranial vault meningioma with diffuse and extensive dural involvement, extracranial extension into the calvarium, and homogeneous contrast enhancement on gadolinium-enhanced T1-weighted MRI, was selected for this case study. A patient-specific manikin was designed starting with the segmentation of the patient's preoperative MRI images to simulate a craniotomy procedure. Surgical planning was performed according to the segmented anatomy, and customized bone flaps were designed accordingly. During the surgical simulation stage, the VOSTARS head-mounted display was used to accurately display the planned craniotomy trajectory over the manikin skull. The precision of the craniotomy was assessed based on the evaluation of previously prepared custom-made bone flaps. RESULTS: A bone flap with a radius 0.5 mm smaller than the radius of an ideal craniotomy fitted perfectly over the performed craniotomy, demonstrating an error of less than ±1 mm in the task execution. The results of this laboratory-based experiment suggest that the proposed augmented reality platform helps in simulating convexity en plaque meningioma resection and custom-made cranioplasty, as carefully planned in the preoperative phase. CONCLUSIONS: Augmented reality head-mounted displays have the potential to be a useful adjunct in tumor surgical resection, cranial vault lesion craniotomy and also skull base surgery, but more study with large series is needed.

In Situ Visualization for 3D Ultrasound-Guided Interventions with Augmented Reality Headset.

Augmented Reality (AR) headsets have become the most ergonomic and efficient visualization devices to support complex manual tasks performed under direct vision. Their ability to provide hands-free interaction with the augmented scene makes them perfect for manual procedures such as surgery. This study demonstrates the reliability of an AR head-mounted display (HMD), conceived for surgical guidance, in navigating in-depth high-precision manual tasks guided by a 3D ultrasound imaging system. The integration between the AR visualization system and the ultrasound imaging system provides the surgeon with real-time intra-operative information on unexposed soft tissues that are spatially registered with the surrounding anatomic structures. The efficacy of the AR guiding system was quantitatively assessed with an in vitro study simulating a biopsy intervention aimed at determining the level of accuracy achievable. In the experiments, 10 subjects were asked to perform the biopsy on four spherical lesions of decreasing sizes (10, 7, 5, and 3 mm). The experimental results showed that 80% of the subjects were able to successfully perform the biopsy on the 5 mm lesion, with a 2.5 mm system accuracy. The results confirmed that the proposed integrated system can be used for navigation during in-depth high-precision manual tasks.

Can Liquid Lenses Increase Depth of Field in Head Mounted Video See-Through Devices?

Wearable Video See-Through (VST) devices for Augmented Reality (AR) and for obtaining a Magnified View are taking hold in the medical and surgical fields. However, these devices are not yet usable in daily clinical practice, due to focusing problems and a limited depth of field. This study investigates the use of liquid-lens optics to create an autofocus system for wearable VST visors. The autofocus system is based on a Time of Flight (TOF) distance sensor and an active autofocus control system. The integrated autofocus system in the wearable VST viewers showed good potential in terms of providing rapid focus at various distances and a magnified view.

Hybrid Simulation and Planning Platform for Cryosurgery with Microsoft HoloLens.

Cryosurgery is a technique of growing popularity involving tissue ablation under controlled freezing. Technological advancement of devices along with surgical technique improvements have turned cryosurgery from an experimental to an established option for treating several diseases. However, cryosurgery is still limited by inaccurate planning based primarily on 2D visualization of the patient's preoperative images. Several works have been aimed at modelling cryoablation through heat transfer simulations; however, most software applications do not meet some key requirements for clinical routine use, such as high computational speed and user-friendliness. This work aims to develop an intuitive platform for anatomical understanding and pre-operative planning by integrating the information content of radiological images and cryoprobe specifications either in a 3D virtual environment (desktop application) or in a hybrid simulator, which exploits the potential of the 3D printing and augmented reality functionalities of Microsoft HoloLens. The proposed platform was preliminarily validated for the retrospective planning/simulation of two surgical cases. Results suggest that the platform is easy and quick to learn and could be used in clinical practice to improve anatomical understanding, to make surgical planning easier than the traditional method, and to strengthen the memorization of surgical planning.

Evaluation of a Wearable AR Platform for Guiding Complex Craniotomies in Neurosurgery.

Today, neuronavigation is widely used in daily clinical routine to perform safe and efficient surgery. Augmented reality (AR) interfaces can provide anatomical models and preoperative planning contextually blended with the real surgical scenario, overcoming the limitations of traditional neuronavigators. This study aims to demonstrate the reliability of a new-concept AR headset in navigating complex craniotomies. Moreover, we aim to prove the efficacy of a patient-specific template-based methodology for fast, non-invasive, and fully automatic planning-to-patient registration. The AR platform navigation performance was assessed with an in-vitro study whose goal was twofold: to measure the real-to-virtual 3D target visualization error (TVE), and assess the navigation accuracy through a user study involving 10 subjects in tracing a complex craniotomy. The feasibility of the template-based registration was preliminarily tested on a volunteer. The TVE mean and standard deviation were 1.3 and 0.6 mm. The results of the user study, over 30 traced craniotomies, showed that 97% of the trajectory length was traced within an error margin of 1.5 mm, and 92% within a margin of 1 mm. The in-vivo test confirmed the feasibility and reliability of the patient-specific template for registration. The proposed AR headset allows ergonomic and intuitive fruition of preoperative planning, and it can represent a valid option to support neurosurgical tasks.

Effectiveness and Safety of A Nutraceutical Formulation for the Treatment of Functional Dyspepsia in Primary Care.

BACKGROUND: Although FD may affect up to 10% of the general population, the therapy for FD is not standard. Recently, ginger-based food supplements have been proposed in order to restore FD symptoms. Our aim was to assess the efficacy of a new nutraceutical formulation containing extract of gingerol and thymus as a possible natural treatment in managing the symptoms of functional dyspepsia (FD). METHODS: We retrospectively analyzed the efficacy and safety profiles of a nutraceutical formulation containing Zingiber officinalis root extract and a standardized Thymus extract. It was administered as 1 ml/day twice a day for 90 days. Patients were assessed at baseline and after 1, 2 and 3 months of treatment, following a month of pharmacological washout by completing a questionnaire reporting the trend of the following symptoms: epigastric pain, epigastric heaviness, early satiety, belching, and regurgitation. Every symptom was assessed by a Visual Analogic Scale (VAS), ranging from 0= absence to 10= maximal severity. RESULTS: We enrolled 272 patients (99 males and 173 females; median IQR age 49.5, 36-64 yrs). Obesity (BMI>30) was present in 28 (12.5%) patients; smokers were 83 (30.5%); and comorbidities were present in 107 (39.3%) patients. Improvement of symptom scores during treatment and one month after its suspension was extremely significant (p<0.000). CONCLUSION: This large study found that nutraceutical formulation could be one of the tools for an empirical approach to treat patients with FD, especially when a non-conventional drug treatment is preferable for the patient and considered suitable by the physician.

ValveTech: A Novel Robotic Approach for Minimally Invasive Aortic Valve Replacement.

OBJECTIVE: Aortic valve disease is the most common heart disease in the elderly calling for replacement with an artificial valve. The presented surgical robot aims to provide a highly controllable instrument for efficient delivery of an artificial valve by the help of integrated endoscopic vision. METHODS: A robot (called ValveTech), intended for minimally invasive surgery (MIS) and consisting of a flexible cable driven manipulator, a passive arm, and a control unit has been designed and prototyped. The flexible manipulator has several features (e.g., stabilizing flaps, tiny cameras, dexterous introducer and custom cartridge) to help the proper valve placement. It provides 5 degrees of freedom for reaching the operative site via mini-thoracotomy; it adjusts the valve and expands it at the optimal position. The robot was evaluated by ten cardiac surgeons following a real surgical scenario in artificial chest simulator with an aortic mockup. Moreover, after each delivery, the expanded valve was evaluated objectively in comparison with the ideal position. RESULTS: The robot performances were evaluated positively by surgeons. The trials resulted in faster delivery and an average misalignment distance of 3.8 mm along the aorta axis; 16.3 degrees rotational angle around aorta axis and 8.8 degrees misalignment of the valve commissure plane to the ideal plane were measured. CONCLUSION: The trials successfully proved the proposed system for valve delivery under endoscopic vision. SIGNIFICANCE: The ValveTech robot can be an alternative solution for minimally invasive aortic valve surgery and improve the quality of the operation both for surgeons and patients.

The Wearable VOSTARS System for Augmented Reality-Guided Surgery: Preclinical Phantom Evaluation for High-Precision Maxillofacial Tasks.

BACKGROUND: In the context of guided surgery, augmented reality (AR) represents a groundbreaking improvement. The Video and Optical See-Through Augmented Reality Surgical System (VOSTARS) is a new AR wearable head-mounted display (HMD), recently developed as an advanced navigation tool for maxillofacial and plastic surgery and other non-endoscopic surgeries. In this study, we report results of phantom tests with VOSTARS aimed to evaluate its feasibility and accuracy in performing maxillofacial surgical tasks. METHODS: An early prototype of VOSTARS was used. Le Fort 1 osteotomy was selected as the experimental task to be performed under VOSTARS guidance. A dedicated set-up was prepared, including the design of a maxillofacial phantom, an ad hoc tracker anchored to the occlusal splint, and cutting templates for accuracy assessment. Both qualitative and quantitative assessments were carried out. RESULTS: VOSTARS, used in combination with the designed maxilla tracker, showed excellent tracking robustness under operating room lighting. Accuracy tests showed that 100% of Le Fort 1 trajectories were traced with an accuracy of ±1.0 mm, and on average, 88% of the trajectory's length was within ±0.5 mm accuracy. CONCLUSIONS: Our preliminary results suggest that the VOSTARS system can be a feasible and accurate solution for guiding maxillofacial surgical tasks, paving the way to its validation in clinical trials and for a wide spectrum of maxillofacial applications.

Definition of Proficiency Level by a Virtual Simulator as a First Step Toward a Curriculum on Fundamental Skills for Endovascular Aneurysm Repair (EVAR).

OBJECTIVE: At present, there is no proficiency-based curriculum for endovascular treatment of aortic aneurysm repair (EVAR) using virtual reality (VR) surgical simulators, whereas such curricula are available for the treatment of iliac and/or superficial femoral artery disease. The purpose of this work was to compute proficiency, defined by a benchmark level determined by the performance of experts, using a commercial VR simulator as a first step of a curriculum on EVAR. MATERIALS AND METHODS: Expert endovascular surgeons (with more than 150 EVAR cases as first operators) from 12 major Italian centers completed three cases of EVAR of increasing difficulty level 3 times each, using the Angio Mentor simulator (by Simbionix) and Gore devices. Proficiency level was based on performance of expert surgeons, as assessed by metrics from a VR simulator. RESULTS: The participating surgeons had a median of 20 years of experience and executed a median of 440 EVAR. For the 3 simulated cases, the following proficiency values were respectively obtained: total procedure time: 22 minutes 32 seconds, 23 minutes 05 seconds, and 20 minutes 32 seconds; total amount of contrast injected: 85.16 mL, 89.97 mL, and 98.01 mL total fluoroscopy time: 10 minutes 39 seconds, 12 minutes 22 seconds, and 10 minutes 17 seconds; time to contralateral gate cannulation: 5 minutes 51 seconds, 7 minutes 09 seconds, and 3 minutes 32 seconds. CONCLUSIONS: We computed proficiency levels for 3 simulated cases of EVAR using a VR simulator. Our next step is to determine whether surgical residents can reach this level. Translational research will then be required to assess the impact of such training on real patients.