Registration Sanity Check for AR-guided Surgical Interventions: Experience From Head and Face Surgery.

Achieving and maintaining proper image registration accuracy is an open challenge of image-guided surgery. This work explores and assesses the efficacy of a registration sanity check method for augmented reality-guided navigation (AR-RSC), based on the visual inspection of virtual 3D models of landmarks. We analyze the AR-RSC sensitivity and specificity by recruiting 36 subjects to assess the registration accuracy of a set of 114 AR images generated from camera images acquired during an AR-guided orthognathic intervention. Translational or rotational errors of known magnitude up to ±1.5 mm/±15.5°, were artificially added to the image set in order to simulate different registration errors. This study analyses the performance of AR-RSC when varying (1) the virtual models selected for misalignment evaluation (e. g., the model of brackets, incisor teeth, and gingival margins in our experiment), (2) the type (translation/rotation) of registration error, and (3) the level of user experience in using AR technologies. Results show that: 1) the sensitivity and specificity of the AR-RSC depends on the virtual models (globally, a median true positive rate of up to 79.2% was reached with brackets, and a median true negative rate of up to 64.3% with incisor teeth), 2) there are error components that are more difficult to identify visually, 3) the level of user experience does not affect the method. In conclusion, the proposed AR-RSC, tested also in the operating room, could represent an efficient method to monitor and optimize the registration accuracy during the intervention, but special attention should be paid to the selection of the AR data chosen for the visual inspection of the registration accuracy.

Design, Fabrication, and Preliminary Validation of Patient-Specific Spine Section Phantoms for Use in Training Spine Surgeons Outside the Operating Room/Theatre.

Pedicle screw fixation (PSF) demands rigorous training to mitigate the risk of severe neurovascular complications arising from screw misplacement. This paper introduces a patient-specific phantom designed for PSF training, extending a portion of the learning process beyond the confines of the surgical room. Six phantoms of the thoracolumbar region were fabricated from radiological datasets, combining 3D printing and casting techniques. The phantoms were employed in three training sessions by a fifth-year resident who performed full training on all six phantoms; he/she placed a total of 57 pedicle screws. Analysis of the learning curve, focusing on time per screw and positioning accuracy, revealed attainment of an asymptotic performance level (around 3 min per screw) after 40 screws. The phantom's efficacy was evaluated by three experts and six residents, each inserting a minimum of four screws. Initial assessments confirmed face, content, and construct validity, affirming the patient-specific phantoms as a valuable training resource. These proposed phantoms exhibit great promise as an essential tool in surgical training as they exhibited a demonstrable learning effect on the PSF technique. This study lays the foundation for further exploration and underscores the potential impact of these patient-specific phantoms on the future of spinal surgical education.

Preliminary Results of Preoperative Planning Using 3D Printing and Augmented Reality in Cryotherapy Treatment of Giant Cell Tumor of Bone-CRIO2AR Project.

Giant Cell Tumor of Bone is a benign tumor with high local aggressive expansion, which, in rare cases, spreads metastasis. Surgical treatment, which often consists of wide curettage to reduce recurrence risk, can lower the quality of life for those affected. Along with aggressive surgery, adjuvant intraoperative techniques have been implemented such as PMMA and cryotherapy. One of the most widely used cryotherapy techniques involves the use of probes to generate ice balls, which have been scientifically shown to have various impacts on the tumor. Although this has been acknowledged, no one has yet tested a way to accurately plan the positioning of cryotherapy probes before surgery, according to the research conducted by the authors. CRIO2AR is a randomized clinical prospective ongoing study by which it will be experimented via preoperative planning of ice probes placement using AR and 3D printing technologies. By studying a single clinical case with these technologies, the surgeon gains better awareness of patient's anatomy and tumor localization. Preliminary results are shown in the article. The first results are confirming that these technologies are applicable in clinical practice. Secondly, preoperative planning is proving to be reliable, easily replicable, and useful for the surgeon.

Bioengineering, augmented reality, and robotic surgery in vascular surgery: A literature review.

Biomedical engineering integrates a variety of applied sciences with life sciences to improve human health and reduce the invasiveness of surgical procedures. Technological advances, achieved through biomedical engineering, have contributed to significant improvements in the field of vascular and endovascular surgery. This paper aims to review the most cutting-edge technologies of the last decade involving the use of augmented reality devices and robotic systems in vascular surgery, highlighting benefits and limitations. Accordingly, two distinct literature surveys were conducted through the PubMed database: the first review provides a comprehensive assessment of augmented reality technologies, including the different techniques available for the visualization of virtual content (11 papers revised); the second review collects studies with bioengineering content that highlight the research trend in robotic vascular surgery, excluding works focused only on the clinical use of commercially available robotic systems (15 papers revised). Technological flow is constant and further advances in imaging techniques and hardware components will inevitably bring new tools for a clinical translation of innovative therapeutic strategies in vascular surgery.

Brain Tumor and Augmented Reality: New Technologies for the Future.

In recent years, huge progress has been made in the management of brain tumors, due to the availability of imaging devices, which provide fundamental anatomical and pathological information not only for diagnostic purposes [...].

Can Liquid Lenses Increase Depth of Field in Head Mounted Video See-Through Devices?

Wearable Video See-Through (VST) devices for Augmented Reality (AR) and for obtaining a Magnified View are taking hold in the medical and surgical fields. However, these devices are not yet usable in daily clinical practice, due to focusing problems and a limited depth of field. This study investigates the use of liquid-lens optics to create an autofocus system for wearable VST visors. The autofocus system is based on a Time of Flight (TOF) distance sensor and an active autofocus control system. The integrated autofocus system in the wearable VST viewers showed good potential in terms of providing rapid focus at various distances and a magnified view.

The Wearable VOSTARS System for Augmented Reality-Guided Surgery: Preclinical Phantom Evaluation for High-Precision Maxillofacial Tasks.

BACKGROUND: In the context of guided surgery, augmented reality (AR) represents a groundbreaking improvement. The Video and Optical See-Through Augmented Reality Surgical System (VOSTARS) is a new AR wearable head-mounted display (HMD), recently developed as an advanced navigation tool for maxillofacial and plastic surgery and other non-endoscopic surgeries. In this study, we report results of phantom tests with VOSTARS aimed to evaluate its feasibility and accuracy in performing maxillofacial surgical tasks. METHODS: An early prototype of VOSTARS was used. Le Fort 1 osteotomy was selected as the experimental task to be performed under VOSTARS guidance. A dedicated set-up was prepared, including the design of a maxillofacial phantom, an ad hoc tracker anchored to the occlusal splint, and cutting templates for accuracy assessment. Both qualitative and quantitative assessments were carried out. RESULTS: VOSTARS, used in combination with the designed maxilla tracker, showed excellent tracking robustness under operating room lighting. Accuracy tests showed that 100% of Le Fort 1 trajectories were traced with an accuracy of ±1.0 mm, and on average, 88% of the trajectory's length was within ±0.5 mm accuracy. CONCLUSIONS: Our preliminary results suggest that the VOSTARS system can be a feasible and accurate solution for guiding maxillofacial surgical tasks, paving the way to its validation in clinical trials and for a wide spectrum of maxillofacial applications.

Interactive serious game for shoulder rehabilitation based on real-time hand tracking.

BACKGROUND: Virtual reality is becoming popular in the rehabilitation field thanks to the several advantages it can offer to patients and physicians. Indeed, serious games can: motivate and engage the patient; offer different levels of challenge and difficulty based on the patient baseline, and integrate objective measures of the patient's performance during each rehabilitation session. OBJECTIVE: We designed and implemented a serious game for shoulder rehabilitation based on real-time hand tracking. The aim was to maintain the medical benefits of traditional rehabilitation, while reducing human resources and costs and facilitating active patient participation. METHODS: Our software application provides the user with a shoulder horizontal adduction exercise. This exercise takes place in a 2D interactive game environment, controlled by hand movements on a desk pad. The hardware includes a standard desktop computer and screen, and the Leap Motion Controller: a hand tracking system. Changing the desk pad material allows the physiotherapist to vary the friction between the user hand and the supporting surface. RESULTS: Fourteen healthy volunteers and six rehabilitation experts tested our serious game. The results showed that the application is attractive, ergonomic and clinically useful. CONCLUSION: Despite promising results, clinical validation is necessary to demonstrate the efficacy of the serious game.

Wearable Augmented Reality Platform for Aiding Complex 3D Trajectory Tracing.

Augmented reality (AR) Head-Mounted Displays (HMDs) are emerging as the most efficient output medium to support manual tasks performed under direct vision. Despite that, technological and human-factor limitations still hinder their routine use for aiding high-precision manual tasks in the peripersonal space. To overcome such limitations, in this work, we show the results of a user study aimed to validate qualitatively and quantitatively a recently developed AR platform specifically conceived for guiding complex 3D trajectory tracing tasks. The AR platform comprises a new-concept AR video see-through (VST) HMD and a dedicated software framework for the effective deployment of the AR application. In the experiments, the subjects were asked to perform 3D trajectory tracing tasks on 3D-printed replica of planar structures or more elaborated bony anatomies. The accuracy of the trajectories traced by the subjects was evaluated by using templates designed ad hoc to match the surface of the phantoms. The quantitative results suggest that the AR platform could be used to guide high-precision tasks: on average more than 94% of the traced trajectories stayed within an error margin lower than 1 mm. The results confirm that the proposed AR platform will boost the profitable adoption of AR HMDs to guide high precision manual tasks in the peripersonal space.

Perceptual Limits of Optical See-Through Visors for Augmented Reality Guidance of Manual Tasks.

OBJECTIVE: The focal length of available optical see-through (OST) head-mounted displays (HMDs) is at least 2 m; therefore, during manual tasks, the user eye cannot keep in focus both the virtual and real content at the same time. Another perceptual limitation is related to the vergence-accommodation conflict, the latter being present in binocular vision only. This paper investigates the effect of incorrect focus cues on the user performance, visual comfort, and workload during the execution of augmented reality (AR)-guided manual task with one of the most advanced OST HMD, the Microsoft HoloLens. METHODS: An experimental study was designed to investigate the performance of 20 subjects in a connect-the-dots task, with and without the use of AR. The following tests were planned: AR-guided monocular and binocular, and naked-eye monocular and binocular. Each trial was analyzed to evaluate the accuracy in connecting dots. NASA Task Load Index and Likert questionnaires were used to assess the workload and the visual comfort. RESULTS: No statistically significant differences were found in the workload, and in the perceived comfort between the AR-guided binocular and monocular test. User performances were significantly better during the naked eye tests. No statistically significant differences in performances were found in the monocular and binocular tests. The maximum error in AR tests was 5.9 mm. CONCLUSION: Even if there is a growing interest in using commercial OST HMD, for guiding high-precision manual tasks, attention should be paid to the limitations of the available technology not designed for the peripersonal space.

Computed-tomography image segmentation and 3D-reconstruction of the female pelvis for the preoperative planning of sacrocolpopexy: preliminary data.

BACKGROUND: Minimally-invasive sacrocolpopexy is the gold standard procedure for advanced apical prolapse. Nonetheless, sacrocolpopexy has potential serious complications leading many surgeons to avoid this excellent surgical procedure. To overcome these limitations, preoperative planning with 3D models of the female pelvis is proposed. The aim of the study is to evaluate the feasibility of pelvic anatomy reconstruction with the ITK-SNAP software and highlight its potential benefits in this intervention. METHODS: Thirty patient-specific 3D models of the female pelvis were created using ITK-SNAP and the EndoCAS Segmentation Pipeline extension for image segmentation: contrast-enhanced computed tomography (CE-CT) data sets of women who underwent examinations for reasons other than prolapse were used. The distances of pelvic structures from the sacral promontory were standardised and measured, and correlations among these distances were evaluated with Spearman's correlation coefficient. RESULTS: Pelvic anatomy reconstruction was feasible for all CE-CT data sets. A statistically significant correlation was found between the distances of the cava bifurcation and common iliac vessels from the sacral promontory. An area for proximal mesh attachment was defined: it is free from the passage of iliac vessels in 97.5% of cases. A significant statistical correlation was found between the distances of the midpoint of the bispinous diameter and the uterine cervix from the sacral promontory; a process of linear regression showed that the latter measure can be estimated by multiplying the first one by 0.86. CONCLUSIONS: Pre-surgical 3D reconstructions of the female pelvis using ITK-SNAP could help achieve widespread use of sacrocolpopexy: further comparative studies are needed to evaluate the outcomes with and without their use.

A tele-ultrasonographic platform to collect specialist second opinion in less specialized hospitals.

In non-urban scenarios: rural areas or small cities, there is often a limited access to specialistic healthcare due to the inherent challenges associated with recruitment, retention, and access to healthcare professionals. Telemedicine is an economical and effective way to address this problem. In this research, we developed a framework for real-time communication during ultrasound examination that combines interaction via standard video conference protocols and basic AR functionalities (commercial) and a custom-developed application. The tele-ultrasonographic platform has been installed in a rural hospital in the Tuscan Apennines, and was tested on 12 patients. The study explores the utility of the system from the local and remote clinician perspectives. The results obtained provide valuable insight: the platform and the telemedicine paradigm can reduce the costs related to the necessity to move critical patients when there is a need for a specialist second opinion. Moreover, the possibility of having an expert guiding and commenting on the fly the diagnostic examination has also a didactic power, and thus allows the local less specialized clinicians to grow in competencies over time.

Face, content, and construct validity of a simulator for training in endovascular procedures.

Aim: In recent years the interest in structured training programs in endovascular surgical procedures has increased. In this study we assess face, content, and construct validity of a simulator to teach basic skills of endovascular surgery. Material and methods: A cohort of 21 medical students, 26 residents, and 14 expert surgeons participated in the study. Experts assessed face and content validity. Then, they executed four tasks once, while medical students and residents were allowed two attempts to reach a five-minute threshold under expert supervision. Medical students and residents repeated the same exercises during a second session plus three new additional ones, without expert supervision. Results: The simulator was rated as good by experts (four out of five on a Likert scale) in terms of realism (face validity) and usefulness as training tool for the training of basic skills (content). For construct validity, experts outperformed with a statistically significant difference (p < .05) medical students and residents in all tasks, except cannulation of upper mesenteric (p = .053). Differences between novices and intermediates persisted in the second session in the same four tasks and in the three additional ones, with statistically significant difference (p < .05) in the last four exercises. Conclusions: This study showed face, content, and construct validity of BEST simulator.

A new head-mounted display-based augmented reality system in neurosurgical oncology: a study on phantom.

PURPOSE: Benefits of minimally invasive neurosurgery mandate the development of ergonomic paradigms for neuronavigation. Augmented Reality (AR) systems can overcome the shortcomings of commercial neuronavigators. The aim of this work is to apply a novel AR system, based on a head-mounted stereoscopic video see-through display, as an aid in complex neurological lesion targeting. Effectiveness was investigated on a newly designed patient-specific head mannequin featuring an anatomically realistic brain phantom with embedded synthetically created tumors and eloquent areas. MATERIALS AND METHODS: A two-phase evaluation process was adopted in a simulated small tumor resection adjacent to Broca's area. Phase I involved nine subjects without neurosurgical training in performing spatial judgment tasks. In Phase II, three surgeons were involved in assessing the effectiveness of the AR-neuronavigator in performing brain tumor targeting on a patient-specific head phantom. RESULTS: Phase I revealed the ability of the AR scene to evoke depth perception under different visualization modalities. Phase II confirmed the potentialities of the AR-neuronavigator in aiding the determination of the optimal surgical access to the surgical target. CONCLUSIONS: The AR-neuronavigator is intuitive, easy-to-use, and provides three-dimensional augmented information in a perceptually-correct way. The system proved to be effective in guiding skin incision, craniotomy, and lesion targeting. The preliminary results encourage a structured study to prove clinical effectiveness. Moreover, our testing platform might be used to facilitate training in brain tumour resection procedures.

Augmented reality in neurosurgery: a systematic review.

Neuronavigation has become an essential neurosurgical tool in pursuing minimal invasiveness and maximal safety, even though it has several technical limitations. Augmented reality (AR) neuronavigation is a significant advance, providing a real-time updated 3D virtual model of anatomical details, overlaid on the real surgical field. Currently, only a few AR systems have been tested in a clinical setting. The aim is to review such devices. We performed a PubMed search of reports restricted to human studies of in vivo applications of AR in any neurosurgical procedure using the search terms "Augmented reality" and "Neurosurgery." Eligibility assessment was performed independently by two reviewers in an unblinded standardized manner. The systems were qualitatively evaluated on the basis of the following: neurosurgical subspecialty of application, pathology of treated lesions and lesion locations, real data source, virtual data source, tracking modality, registration technique, visualization processing, display type, and perception location. Eighteen studies were included during the period 1996 to September 30, 2015. The AR systems were grouped by the real data source: microscope (8), hand- or head-held cameras (4), direct patient view (2), endoscope (1), and X-ray fluoroscopy (1) head-mounted display (1). A total of 195 lesions were treated: 75 (38.46 %) were neoplastic, 77 (39.48 %) neurovascular, and 1 (0.51 %) hydrocephalus, and 42 (21.53 %) were undetermined. Current literature confirms that AR is a reliable and versatile tool when performing minimally invasive approaches in a wide range of neurosurgical diseases, although prospective randomized studies are not yet available and technical improvements are needed.

Augmented reality visualization of deformable tubular structures for surgical simulation.

BACKGROUND: Surgical simulation based on augmented reality (AR), mixing the benefits of physical and virtual simulation, represents a step forward in surgical training. However, available systems are unable to update the virtual anatomy following deformations impressed on actual anatomy. METHODS: A proof-of-concept solution is described providing AR visualization of hidden deformable tubular structures using nitinol tubes sensorized with electromagnetic sensors. This system was tested in vitro on a setup comprised of sensorized cystic, left and right hepatic, and proper hepatic arteries. In the trial session, the surgeon deformed the tubular structures with surgical forceps in 10 positions. RESULTS: The mean, standard deviation, and maximum misalignment between virtual and real arteries were 0.35, 0.22, and 0.99 mm, respectively. CONCLUSION: The alignment accuracy obtained demonstrates the feasibility of the approach, which can be adopted in advanced AR simulations, in particular as an aid to the identification and isolation of tubular structures. Copyright © 2015 John Wiley & Sons, Ltd.

Assessment of DICOM Viewers Capable of Loading Patient-specific 3D Models Obtained by Different Segmentation Platforms in the Operating Room.

Patient-specific 3D models obtained by the segmentation of volumetric diagnostic images play an increasingly important role in surgical planning. Surgeons use the virtual models reconstructed through segmentation to plan challenging surgeries. Many solutions exist for the different anatomical districts and surgical interventions. The possibility to bring the 3D virtual reconstructions with native radiological images in the operating room is essential for fostering the use of intraoperative planning. To the best of our knowledge, current DICOM viewers are not able to simultaneously connect to the picture archiving and communication system (PACS) and import 3D models generated by external platforms to allow a straight integration in the operating room. A total of 26 DICOM viewers were evaluated: 22 open source and four commercial. Two DICOM viewers can connect to PACS and import segmentations achieved by other applications: Synapse 3D® by Fujifilm and OsiriX by University of Geneva. We developed a software network that converts diffuse visual tool kit (VTK) format 3D model segmentations, obtained by any software platform, to a DICOM format that can be displayed using OsiriX or Synapse 3D. Both OsiriX and Synapse 3D were suitable for our purposes and had comparable performance. Although Synapse 3D loads native images and segmentations faster, the main benefits of OsiriX are its user-friendly loading of elaborated images and it being both free of charge and open source.