HoloLens 1 vs. HoloLens 2: Improvements in the New Model for Orthopedic Oncological Interventions.

This work analyzed the use of Microsoft HoloLens 2 in orthopedic oncological surgeries and compares it to its predecessor (Microsoft HoloLens 1). Specifically, we developed two equivalent applications, one for each device, and evaluated the augmented reality (AR) projection accuracy in an experimental scenario using phantoms based on two patients. We achieved automatic registration between virtual and real worlds using patient-specific surgical guides on each phantom. They contained a small adaptor for a 3D-printed AR marker, the characteristic patterns of which were easily recognized using both Microsoft HoloLens devices. The newest model improved the AR projection accuracy by almost 25%, and both of them yielded an RMSE below 3 mm. After ascertaining the enhancement of the second model in this aspect, we went a step further with Microsoft HoloLens 2 and tested it during the surgical intervention of one of the patients. During this experience, we collected the surgeons' feedback in terms of comfortability, usability, and ergonomics. Our goal was to estimate whether the improved technical features of the newest model facilitate its implementation in actual surgical scenarios. All of the results point to Microsoft HoloLens 2 being better in all the aspects affecting surgical interventions and support its use in future experiences.

UltrARsound: in situ visualization of live ultrasound images using HoloLens 2.

PURPOSE: Augmented Reality (AR) has the potential to simplify ultrasound (US) examinations which usually require a skilled and experienced sonographer to mentally align narrow 2D cross-sectional US images in the 3D anatomy of the patient. This work describes and evaluates a novel approach to track retroreflective spheres attached to the US probe using an inside-out technique with the AR glasses HoloLens 2. Finally, live US images are displayed in situ on the imaged anatomy. METHODS: The Unity application UltrARsound performs spatial tracking of the US probe and attached retroreflective markers using the depth camera integrated into the AR glasses-thus eliminating the need for an external tracking system. Additionally, a Kalman filter is implemented to improve the noisy measurements of the camera. US images are streamed wirelessly via the PLUS toolkit to HoloLens 2. The technical evaluation comprises static and dynamic tracking accuracy, frequency and latency of displayed images. RESULTS: Tracking is performed with a median accuracy of 1.98 mm/1.81[Formula: see text] for the static setting when using the Kalman filter. In a dynamic scenario, the median error was 2.81 mm/1.70[Formula: see text]. The tracking frequency is currently limited to 20 Hz. 83% of the displayed US images had a latency lower than 16 ms. CONCLUSIONS: In this work, we showed that spatial tracking of retroreflective spheres with the depth camera of HoloLens 2 is feasible, achieving a promising accuracy for in situ visualization of live US images. For tracking, no additional hardware nor modifications to HoloLens 2 are required making it a cheap and easy-to-use approach. Moreover, a minimal latency of displayed images enables a real-time perception for the sonographer.

Evaluation of optical tracking and augmented reality for needle navigation in sacral nerve stimulation.

BACKGROUND AND OBJECTIVE: Sacral nerve stimulation (SNS) is a minimally invasive procedure where an electrode lead is implanted through the sacral foramina to stimulate the nerve modulating colonic and urinary functions. One of the most crucial steps in SNS procedures is the placement of the tined lead close to the sacral nerve. However, needle insertion is very challenging for surgeons. Several x-ray projections are required to interpret the needle position correctly. In many cases, multiple punctures are needed, causing an increase in surgical time and patient's discomfort and pain. In this work we propose and evaluate two different navigation systems to guide electrode placement in SNS surgeries designed to reduce surgical time, minimize patient discomfort and improve surgical outcomes. METHODS: We developed, for the first alternative, an open-source navigation software to guide electrode placement by real-time needle tracking with an optical tracking system (OTS). In the second method, we present a smartphone-based AR application that displays virtual guidance elements directly on the affected area, using a 3D printed reference marker placed on the patient. This guidance facilitates needle insertion with a predefined trajectory. Both techniques were evaluated to determine which one obtained better results than the current surgical procedure. To compare the proposals with the clinical method, we developed an x-ray software tool that calculates a digitally reconstructed radiograph, simulating the fluoroscopy acquisitions during the procedure. Twelve physicians (inexperienced and experienced users) performed needle insertions through several specific targets to evaluate the alternative SNS guidance methods on a realistic patient-based phantom. RESULTS: With each navigation solution, we observed that users took less average time to complete each insertion (36.83 s and 44.43 s for the OTS and AR methods, respectively) and needed fewer average punctures to reach the target (1.23 and 1.96 for the OTS and AR methods respectively) than following the standard clinical method (189.28 s and 3.65 punctures). CONCLUSIONS: To conclude, we have shown two navigation alternatives that could improve surgical outcome by significantly reducing needle insertions, surgical time and patient's pain in SNS procedures. We believe that these solutions are feasible to train surgeons and even replace current SNS clinical procedures.

Longitudinal Association Between Muscle Loss and Mortality in Ever Smokers.

BACKGROUND: Body composition measures, specifically low weight or reduced muscle mass, are associated with mortality in COPD, but the effect of longitudinal body composition changes is undefined. RESEARCH QUESTION: Is the longitudinal loss of fat-free mass (FFM) associated with increased mortality, including in those with initially normal or elevated body composition metrics? STUDY DESIGN AND METHODS: Participants with complete data for at least one visit in the COPDGene study (n = 9,268) and the ECLIPSE study (n = 1,760) were included and monitored for 12 and 8 years, respectively. Pectoralis muscle area (PMA) was derived from thoracic CT scans and used as a proxy for FFM. A longitudinal mixed submodel for PMA and a Cox proportional hazards submodel for survival were fitted on a joint distribution, using a shared random intercept parameter and Markov chain Monte Carlo parameter estimation. RESULTS: Both cohorts demonstrated a left-shifted distribution of baseline FFM, not reflected in BMI, and an increase in all-cause mortality risk associated with longitudinal loss of PMA. For each 1-cm2 PMA loss, mortality increased 3.1% (95% CI, 2.4%-3.7%; P < .001) in COPDGene, and 2.4% (95% CI, 0.9%-4.0%; P < .001) in ECLIPSE. Increased mortality risk was independent of enrollment values for BMI and disease severity [BODE (body mass, airflow obstruction, dyspnea, and exercise capacity) index quartiles] and was significant even in participants with initially greater than average PMA. INTERPRETATION: Longitudinal loss of PMA is associated with increased all-cause mortality, regardless of BMI or initial muscle mass. Consideration of novel screening tests and further research into mechanisms contributing to muscle decline may improve risk stratification and identify novel therapeutic targets in ever smokers.

Augmented Reality as a Tool to Guide PSI Placement in Pelvic Tumor Resections.

Patient-specific instruments (PSIs) have become a valuable tool for osteotomy guidance in complex surgical scenarios such as pelvic tumor resection. They provide similar accuracy to surgical navigation systems but are generally more convenient and faster. However, their correct placement can become challenging in some anatomical regions, and it cannot be verified objectively during the intervention. Incorrect installations can result in high deviations from the planned osteotomy, increasing the risk of positive resection margins. In this work, we propose to use augmented reality (AR) to guide and verify PSIs placement. We designed an experiment to assess the accuracy provided by the system using a smartphone and the HoloLens 2 and compared the results with the conventional freehand method. The results showed significant differences, where AR guidance prevented high osteotomy deviations, reducing maximal deviation of 54.03 mm for freehand placements to less than 5 mm with AR guidance. The experiment was performed in two versions of a plastic three-dimensional (3D) printed phantom, one including a silicone layer to simulate tissue, providing more realism. We also studied how differences in shape and location of PSIs affect their accuracy, concluding that those with smaller sizes and a homogeneous target surface are more prone to errors. Our study presents promising results that prove AR's potential to overcome the present limitations of PSIs conveniently and effectively.

Combining Augmented Reality and 3D Printing to Improve Surgical Workflows in Orthopedic Oncology: Smartphone Application and Clinical Evaluation.

During the last decade, orthopedic oncology has experienced the benefits of computerized medical imaging to reduce human dependency, improving accuracy and clinical outcomes. However, traditional surgical navigation systems do not always adapt properly to this kind of interventions. Augmented reality (AR) and three-dimensional (3D) printing are technologies lately introduced in the surgical environment with promising results. Here we present an innovative solution combining 3D printing and AR in orthopedic oncological surgery. A new surgical workflow is proposed, including 3D printed models and a novel AR-based smartphone application (app). This app can display the patient's anatomy and the tumor's location. A 3D-printed reference marker, designed to fit in a unique position of the affected bone tissue, enables automatic registration. The system has been evaluated in terms of visualization accuracy and usability during the whole surgical workflow. Experiments on six realistic phantoms provided a visualization error below 3 mm. The AR system was tested in two clinical cases during surgical planning, patient communication, and surgical intervention. These results and the positive feedback obtained from surgeons and patients suggest that the combination of AR and 3D printing can improve efficacy, accuracy, and patients' experience.

Respiratory exacerbations are associated with muscle loss in current and former smokers.

OBJECTIVES: Muscle wasting is a recognised extra-pulmonary complication in chronic obstructive pulmonary disease and has been associated with increased risk of death. Acute respiratory exacerbations are associated with reduction of muscle function, but there is a paucity of data on their long-term effect. This study explores the relationship between acute respiratory exacerbations and long-term muscle loss using serial measurements of CT derived pectoralis muscle area (PMA). DESIGN AND SETTING: Participants were included from two prospective, longitudinal, observational, multicentre cohorts of ever-smokers with at least 10 pack-year history. PARTICIPANTS: The primary analysis included 1332 (of 2501) participants from Evaluation of COPD Longitudinally to Identify Predictive Surrogate Endpoints (ECLIPSE) and 4384 (of 10 198) participants from Genetic Epidemiology of COPD (COPDGene) who had complete data from their baseline and follow-up visits. INTERVENTIONS: PMA was measured on chest CT scans at two timepoints. Self-reported exacerbation data were collected from participants in both studies through the use of periodic longitudinal surveys. MAIN OUTCOME MEASURES: Age-related and excess muscle loss over time. RESULTS: Age, sex, race and body mass index were associated with baseline PMA. Participants experienced age-related decline at the upper end of reported normal ranges. In ECLIPSE, the exacerbation rate over time was associated with an excess muscle area loss of 1.3% (95% CI 0.6 to 1.9, p<0.001) over 3 years and in COPDGene with an excess muscle area loss of 2.1% (95% CI 1.2 to 2.8, p<0.001) over 5 years. Excess muscle area decline was absent in 273 individuals who participated in pulmonary rehabilitation. CONCLUSIONS: Exacerbations are associated with accelerated skeletal muscle loss. Each annual exacerbation was associated with the equivalent of 6 months of age-expected decline in muscle mass. Ameliorating exacerbation-associated muscle loss represents an important therapeutic target.

Computer-Assisted Dental Implant Placement Following Free Flap Reconstruction: Virtual Planning, CAD/CAM Templates, Dynamic Navigation and Augmented Reality.

Image-guided surgery, prosthetic-based virtual planning, 3D printing, and CAD/CAM technology are changing head and neck ablative and reconstructive surgical oncology. Due to quality-of-life improvement, dental implant rehabilitation could be considered in every patient treated with curative intent. Accurate implant placement is mandatory for prosthesis long-term stability and success in oncologic patients. We present a prospective study, with a novel workflow, comprising 11 patients reconstructed with free flaps and 56 osseointegrated implants placed in bone flaps or remnant jaws (iliac crest, fibula, radial forearm, anterolateral thigh). Starting from CT data and jaw plaster model scanning, virtual dental prosthesis was designed. Then prosthetically driven dental implacement was also virtually planned and transferred to the patient by means of intraoperative infrared optical navigation (first four patients), and a combination of conventional static teeth supported 3D-printed acrylic guide stent, intraoperative dynamic navigation, and augmented reality for final intraoperative verification (last 7 patients). Coronal, apical, and angular deviation between virtual surgical planning and final guided intraoperative position was measured on each implant. There is a clear learning curve for surgeons when applying guided methods. Initial only-navigated cases achieved low accuracy but were comparable to non-guided freehand positioning due to jig registration instability. Subsequent dynamic navigation cases combining highly stable acrylic static guides as reference and registration markers result in the highest accuracy with a 1-1.5-mm deviation at the insertion point. Smartphone-based augmented reality visualization is a valuable tool for intraoperative visualization and final verification, although it is still a difficult technique for guiding surgery. A fixed screw-retained ideal dental prosthesis was achieved in every case as virtually planned. Implant placement, the final step in free flap oncological reconstruction, could be accurately planned and placed with image-guided surgery, 3D printing, and CAD/CAM technology. The learning curve could be overcome with preclinical laboratory training, but virtually designed and 3D-printed tracer registration stability is crucial for accurate and predictable results. Applying these concepts to our difficult oncologic patient subgroup with deep anatomic alterations ended in comparable results as those reported in non-oncologic patients.

Surgical Navigation, Augmented Reality, and 3D Printing for Hard Palate Adenoid Cystic Carcinoma En-Bloc Resection: Case Report and Literature Review.

Adenoid Cystic Carcinoma is a rare and aggressive tumor representing less than 1% of head and neck cancers. This malignancy often arises from the minor salivary glands, being the palate its most common location. Surgical en-bloc resection with clear margins is the primary treatment. However, this location presents a limited line of sight and a high risk of injuries, making the surgical procedure challenging. In this context, technologies such as intraoperative navigation can become an effective tool, reducing morbidity and improving the safety and accuracy of the procedure. Although their use is extended in fields such as neurosurgery, their application in maxillofacial surgery has not been widely evidenced. One reason is the need to rigidly fixate a navigation reference to the patient, which often entails an invasive setup. In this work, we studied three alternative and less invasive setups using optical tracking, 3D printing and augmented reality. We evaluated their precision in a patient-specific phantom, obtaining errors below 1 mm. The optimum setup was finally applied in a clinical case, where the navigation software was used to guide the tumor resection. Points were collected along the surgical margins after resection and compared with the real ones identified in the postoperative CT. Distances of less than 2 mm were obtained in 90% of the samples. Moreover, the navigation provided confidence to the surgeons, who could then undertake a less invasive and more conservative approach. The postoperative CT scans showed adequate resection margins and confirmed that the patient is free of disease after two years of follow-up.

Combining Augmented Reality and 3D Printing to Display Patient Models on a Smartphone.

Augmented reality (AR) has great potential in education, training, and surgical guidance in the medical field. Its combination with three-dimensional (3D) printing (3DP) opens new possibilities in clinical applications. Although these technologies have grown exponentially in recent years, their adoption by physicians is still limited, since they require extensive knowledge of engineering and software development. Therefore, the purpose of this protocol is to describe a step-by-step methodology enabling inexperienced users to create a smartphone app, which combines AR and 3DP for the visualization of anatomical 3D models of patients with a 3D-printed reference marker. The protocol describes how to create 3D virtual models of a patient's anatomy derived from 3D medical images. It then explains how to perform positioning of the 3D models with respect to marker references. Also provided are instructions for how to 3D print the required tools and models. Finally, steps to deploy the app are provided. The protocol is based on free and multi-platform software and can be applied to any medical imaging modality or patient. An alternative approach is described to provide automatic registration between a 3D-printed model created from a patient's anatomy and the projected holograms. As an example, a clinical case of a patient suffering from distal leg sarcoma is provided to illustrate the methodology. It is expected that this protocol will accelerate the adoption of AR and 3DP technologies by medical professionals.

Augmented reality in computer-assisted interventions based on patient-specific 3D printed reference.

Augmented reality (AR) can be an interesting technology for clinical scenarios as an alternative to conventional surgical navigation. However, the registration between augmented data and real-world spaces is a limiting factor. In this study, the authors propose a method based on desktop three-dimensional (3D) printing to create patient-specific tools containing a visual pattern that enables automatic registration. This specific tool fits on the patient only in the location it was designed for, avoiding placement errors. This solution has been developed as a software application running on Microsoft HoloLens. The workflow was validated on a 3D printed phantom replicating the anatomy of a patient presenting an extraosseous Ewing's sarcoma, and then tested during the actual surgical intervention. The application allowed physicians to visualise the skin, bone and tumour location overlaid on the phantom and patient. This workflow could be extended to many clinical applications in the surgical field and also for training and simulation, in cases where hard body structures are involved. Although the authors have tested their workflow on AR head mounted display, they believe that a similar approach can be applied to other devices such as tablets or smartphones.