Enhancing Surgical Vision: Augmented Reality in Otolaryngology-Head and Neck Surgery.

Augmented reality (AR) technology has become widely established in otolaryngology-head and neck surgery. Over the past 20 years, numerous AR systems have been investigated and validated across the subspecialties, both in cadaveric and in live surgical studies. AR displays projected through head-mounted devices, microscopes, and endoscopes, most commonly, have demonstrated utility in preoperative planning, intraoperative guidance, and improvement of surgical decision-making. Specifically, they have demonstrated feasibility in guiding tumor margin resections, identifying critical structures intraoperatively, and displaying patient-specific virtual models derived from preoperative imaging, with millimetric accuracy. This review summarizes both established and emerging AR technologies, detailing how their systems work, what features they offer, and their clinical impact across otolaryngology subspecialties. As AR technology continues to advance, its integration holds promise for enhancing surgical precision, simulation training, and ultimately, improving patient outcomes.

The Future of Artificial Intelligence in Surgery.

Until recently, innovations in surgery were largely represented by extensions or augmentations of the surgeon's perception. This includes advancements such as the operating microscope, tumor fluorescence, intraoperative ultrasound, and minimally invasive surgical instrumentation. However, introducing artificial intelligence (AI) into the surgical disciplines represents a transformational event. Not only does AI contribute substantively to enhancing a surgeon's perception with such methodologies as three-dimensional anatomic overlays with augmented reality, AI-improved visualization for tumor resection, and AI-formatted endoscopic and robotic surgery guidance. What truly makes AI so different is that it also provides ways to augment the surgeon's cognition. By analyzing enormous databases, AI can offer new insights that can transform the operative environment in several ways. It can enable preoperative risk assessment and allow a better selection of candidates for procedures such as organ transplantation. AI can also increase the efficiency and throughput of operating rooms and staff and coordinate the utilization of critical resources such as intensive care unit beds and ventilators. Furthermore, AI is revolutionizing intraoperative guidance, improving the detection of cancers, permitting endovascular navigation, and ensuring the reduction in collateral damage to adjacent tissues during surgery (e.g., identification of parathyroid glands during thyroidectomy). AI is also transforming how we evaluate and assess surgical proficiency and trainees in postgraduate programs. It offers the potential for multiple, serial evaluations, using various scoring systems while remaining free from the biases that can plague human supervisors. The future of AI-driven surgery holds promising trends, including the globalization of surgical education, the miniaturization of instrumentation, and the increasing success of autonomous surgical robots. These advancements raise the prospect of deploying fully autonomous surgical robots in the near future into challenging environments such as the battlefield, disaster areas, and even extraplanetary exploration. In light of these transformative developments, it is clear that the future of surgery will belong to those who can most readily embrace and harness the power of AI.

Apple Vision Pro Augmented Reality-Assisted Minimally Invasive Surgical Treatment of Spinal Dural Arteriovenous Fistula.

This report outlines the innovative use of augmented reality (AR) in the surgical planning and treatment of a spinal dural arteriovenous fistula (dAVF) via a minimally invasive technique. AR technology by way of an Apple Vision Pro headset was employed to enhance preoperative visualization and understanding of the pathology, leading to successful surgical ligation of the AVF. This case describes a 56-year-old male presenting with progressive weakness and thoracic myelopathy who showed marked improvement postoperatively, highlighting AR's potential to improve surgical approach and outcomes.

Application of image recognition-based tracker-less augmented reality navigation system in a series of sawbone trials.

BACKGROUND: This study introduced an Augmented Reality (AR) navigation system to address limitations in conventional high tibial osteotomy (HTO). The objective was to enhance precision and efficiency in HTO procedures, overcoming challenges such as inconsistent postoperative alignment and potential neurovascular damage. METHODS: The AR-MR (Mixed Reality) navigation system, comprising HoloLens, Unity Engine, and Vuforia software, was employed for pre-clinical trials using tibial sawbone models. CT images generated 3D anatomical models, projected via HoloLens, allowing surgeons to interact through intuitive hand gestures. The critical procedure of target tracking, essential for aligning virtual and real objects, was facilitated by Vuforia's feature detection algorithm. RESULTS: In trials, the AR-MR system demonstrated significant reductions in both preoperative planning and intraoperative times compared to conventional navigation and metal 3D-printed surgical guides. The AR system, while exhibiting lower accuracy, exhibited efficiency, making it a promising option for HTO procedures. The preoperative planning time for the AR system was notably shorter (4 min) compared to conventional navigation (30.5 min) and metal guides (75.5 min). Intraoperative time for AR lasted 8.5 min, considerably faster than that of conventional navigation (31.5 min) and metal guides (10.5 min). CONCLUSIONS: The AR navigation system presents a transformative approach to HTO, offering a trade-off between accuracy and efficiency. Ongoing improvements, such as the incorporation of two-stage registration and pointing devices, could further enhance precision. While the system may be less accurate, its efficiency renders it a potential breakthrough in orthopedic surgery, particularly for reducing unnecessary harm and streamlining surgical procedures.

Exploration of the application of augmented reality technology for teaching spinal tumor's anatomy and surgical techniques.

Background: Augmented reality (AR) technology is gradually being applied in surgical teaching as an innovative teaching method. Developing innovative teaching methods to replicate clinical theory and practical teaching scenarios, simulate preoperative planning and training for bone tumor surgery, and offer enhanced training opportunities for young physicians to acquire and apply clinical knowledge is a crucial concern that impacts the advancement of the discipline and the educational standards for young orthopedic physicians. Objective: This study explores the application effect of augmented reality technology in anatomy teaching and surgical clinical teaching for spinal tumor. Methods: The method utilizes virtual reality and augmented reality technology to present a spinal tumor model and the surgical process of percutaneous vertebroplasty. We conducted a random selection of 12 students forming into the augmented reality teaching group and 13 students forming into the traditional teaching group among the 8-year medical students from Peking Union Medical College and Tsinghua University, ensuring that the age and learning stage of the students in both groups were similar. Two groups of students were taught using traditional teaching methods and augmented reality technology-assisted teaching methods, respectively. A questionnaire survey was conducted after class to assess the quality of course instruction, student motivation in learning, their proficiency in anatomical structures, their comprehension of spinal tumor growth and metastasis, and their understanding and proficiency in percutaneous vertebroplasty. Results: This study was the first to apply augmented reality technology in teaching, using spinal tumors and percutaneous vertebroplasty as examples, a head-mounted augmented reality device was used to create learning scenarios, presenting the complex three-dimensional spatial structure intuitively. The two groups of students differ significantly in their rating of teaching quality, enthusiasm for learning, knowledge of anatomical features, understanding of spinal trabecular structure, and understanding of steps in percutaneous vertebroplasty. The augmented reality technology-assisted teaching system demonstrates outstanding advantages. Conclusion: Augmented reality technology has great potential and broad prospects in teaching bone tumors, which can help improve the visualization, interactivity, and three-dimensional spatial sense of medical teaching in spinal tumor. The application and development prospects of using augmented reality technology for anatomy instruction, surgical teaching, and simulation training are extensive.

Augmented reality for endoscopic transsphenoidal surgery: evaluating design factors with neurosurgeons.

PURPOSE: This study investigates the potential utility of augmented reality (AR) in the endoscopic transsphenoidal approach (TSA). While previous research has addressed technical challenges in AR for TSA, this paper explores how design factors can improve AR for neurosurgeons from a human-centred design perspective. METHODS: Preliminary qualitative research involved observations of TSA procedures ( n = 2 ) and semi-structured interviews with neurosurgeons ( n = 4 ). These informed the design of an AR mockup, which was evaluated with neurosurgeons ( n = 3 ). An interactive low-fidelity prototype-the "AR-assisted Navigation for the TransSphenoidal Approach (ANTSA)"-was then developed in Unity 3D. A user study ( n = 4 ) evaluated the low-fidelity prototype of ANTSA through contextual interviews, providing feedback on design factors. RESULTS: AR visualisations may be beneficial in streamlining the sellar phase and reducing intraoperative errors such as excessive or inadequate exposure. Key design recommendations include a lean mesh rendering, an intuitive colour palette, and optional structure highlighting. CONCLUSION: This research presents user-centred design guidelines to improve sensemaking and surgical workflow in the sellar phase of TSA, with the goal of improving clinical outcomes. The specific improvements that AR could bring to the workflow are discussed along with surgeons' reservations and its possible application towards training less experienced physicians.

Mixed reality in interventional radiology: a focus on first clinical use of XR90 augmented reality-based visualization and navigation platform.

INTRODUCTION: Augmented reality (AR) and virtual reality (VR) are emerging tools in interventional radiology (IR), enhancing IR education, preprocedural planning, and intraprocedural guidance. AREAS COVERED: This review identifies current applications of AR/VR in IR, with a focus on studies that assess the clinical impact of AR/VR. We outline the relevant technology and assess current limitations and future directions in this space. We found that the use of AR in IR lags other surgical fields, and the majority of the data exists in case series or small-scale studies. Educational use of AR/VR improves learning anatomy, procedure steps, and procedural learning curves. Preprocedural use of AR/VR decreases procedure times, especially in complex procedures. Intraprocedural AR for live tracking is accurate within 5 mm live patients and has up to 0.75 mm in phantoms, offering decreased procedure time and radiation exposure. Challenges include cost, ergonomics, rapid segmentation, and organ motion. EXPERT OPINION: The use of AR/VR in interventional radiology may lead to safer and more efficient procedures. However, more data from larger studies is needed to better understand where AR/VR is confers the most benefit in interventional radiology clinical practice.

The future of telerehabilitation: embracing virtual reality and augmented reality innovations.

The integration of virtual reality (VR) and augmented reality (AR) into the telerehabilitation initiates a major change in the healthcare practice particularly in neurological and also orthopedic rehabilitation. This essay reflects the potential of the VR and AR in their capacity to create immersive, interactive environments that facilitate the recovery. The recent developments have illustrated the ability to enhance the patient engagement and outcomes, especially in tackling the complex motor and cognitive rehabilitation needs. The combination of artificial intelligence (AI) with VR and AR will bring the rehabilitation to the next level by enabling adaptive and responsive treatment programs provided through real-time feedback and predictive analytics. Nevertheless, the issues such as availability, cost, and digital gap among many others present huge obstacles to the mass adoption. This essay provides a very thorough review of the existing level of virtual reality and augmented reality in rehabilitation and examines the many potential gains, drawbacks, and future directions from a different perspective.

Bridging the digital and physical: The psychology of augmented reality.

By blurring the boundaries between digital and physical realities, Augmented Reality (AR) is transforming consumers' perceptions of themselves and their environments. This review demonstrates AR's capacity to influence psychology and behavior in profound ways. We begin by providing a concise introduction to AR, considering its technical, practical, and theoretical properties. Next, we showcase a multi-disciplinary set of recent studies that explore AR's impact on psychological processes and behavioral outcomes. We conclude by offering a selection of potential future research directions designed to deepen our understanding of the psychological and behavioral implications of AR experiences.

Utility of catheter-shaping using mixed-reality devices in cerebral aneurysm coil embolization.

BACKGROUND: Catheter shaping is vital in cerebral aneurysm coil embolization; however, understanding three-dimensional (3D) vascular structures on two-dimensional screens is challenging. Although 3D-printed vascular models are helpful, they demand time, effort, and sterility. This study explores whether mixed-reality (MR) devices displaying 3D computer graphics (3D-CG) can address these issues. METHODS: This study focused on magnetic resonance imaging (MRI) of seven cases of cerebral aneurysms. Head-mounted display (HMD) and spatial reality display (SRD) MR devices were used, and applications for 3D-CG display at a 1:1 scale and a 3D-CG control panel were developed. Catheters shaped using a 3D printer, HMD, and SRD were inserted into hollow models to assess their accessibility and positioning. RESULTS: The concordance rate of the 3D printer and HMD groups in terms of accessibility to the aneurysm was 71.4 %, while that of the 3D printer and SRD group was 85.7 %, and that of the HMD and SRD group was 85.7 %. The concordance rates of positioning in the 3D printer and HMD groups, 3D printer and SRD groups, and HMD and SRD groups were 85.7 %, 85.7 %, and 100 %, respectively. CONCLUSIONS: MR devices facilitate catheter shaping in cerebral aneurysm coil embolization and offer a time-efficient, precise, and sterile alternative to traditional 3D printing methods.

Exploring Augmented Reality Integration in Diagnostic Imaging: Myth or Reality?

This study delves into the transformative potential of integrating augmented reality (AR) within imaging technologies, shedding light on this evolving landscape. Through a comprehensive narrative review, this research uncovers a wealth of literature exploring the intersection between AR and medical imaging, highlighting its growing prominence in healthcare. AR's integration offers a host of potential opportunities to enhance surgical precision, bolster patient engagement, and customize medical interventions. Moreover, when combined with technologies like virtual reality (VR), artificial intelligence (AI), and robotics, AR opens up new avenues for innovation in clinical practice, education, and training. However, amidst these promising prospects lie numerous unanswered questions and areas ripe for exploration. This study emphasizes the need for rigorous research to elucidate the clinical efficacy of AR-integrated interventions, optimize surgical workflows, and address technological challenges. As the healthcare landscape continues to evolve, sustained research efforts are crucial to fully realizing AR's transformative impact in medical imaging. Systematic reviews on AR in healthcare also overlook regulatory and developmental factors, particularly in regard to medical devices. These include compliance with standards, safety regulations, risk management, clinical validation, and developmental processes. Addressing these aspects will provide a comprehensive understanding of the challenges and opportunities in integrating AR into clinical settings, informing stakeholders about crucial regulatory and developmental considerations for successful implementation. Moreover, navigating the regulatory approval process requires substantial financial resources and expertise, presenting barriers to entry for smaller innovators. Collaboration across disciplines and concerted efforts to overcome barriers will be essential in navigating this frontier and harnessing the potential of AR to revolutionize healthcare delivery.

Microscope-Based Augmented Reality: A New Approach in Intraoperative 3D Visualization in Microvascular Decompression?

Neurovascular compression (NVC) syndromes such as trigeminal neuralgia (TN) are causally treated with microvascular decompression (MVD). Semiautomatic segmentation of high-resolution magnetic resonance imaging (MRI) data and constructive interference in steady state (CISS)/time-of-flight (TOF) sequences are utilized for the three-dimensional (3D) visualization of underlying causative vessels at the root entry zones of the relevant cranial nerves. Augmented reality (AR) of neurovascular structures was introduced especially in the resection of brain tumors or aneurysmatic operations. In this report, the potential feasibility of the implementation of microscope-based AR into the intraoperative microsurgical set-up of MVD was investigated. This article recommends the preoperative evaluation of 3D visualization besides the microscopical view of the surgeon. The implementation of multiple imaging data by AR into the operating microscope may afflict the experienced surgeon's view, which should be examined prospectively.

Delineation of gastrointestinal tumors biopsies using a fluorescence lifetime imaging optical fiber probe.

Autofluorescence spectroscopy has emerged in recent years as a powerful tool to report label-free contrast between normal and diseased tissues, both in vivo and ex-vivo. We report the application of an instrument employing an optical fiber probe and capable of performing real-time autofluorescence lifetime imaging at a macroscopic scale, under bright background conditions. We validate and demonstrate the practicality of this technology to discriminate healthy against neoplastic tissue in freshly excised tumor biopsies. The capability of delineating tumor margins through processing the fluorescence decays in the phasors domain was demonstrated on four different types of cancer, highlighting the broad range of potential clinical applications for the proposed approach. The presented results suggest that our autofluorescence lifetime imaging probe, together with phasor analysis, can offer a real-time tool to observe lifetime contrast on tissues and, thus, is a suitable candidate for improving in situ tissue diagnostics during surgery.

Augmented reality-based software (Echo-QR) for guiding the echographic probe toward the acoustic window: a pilot study.

Introduction: With current technology, ultrasound imaging in remote conditions, such as the International Space Station, is performed with vocal guidance or using a teleoperated echograph controlled by an expert. Both methods require real-time communications between the novice operator and expert to locate the probe over the appropriate acoustic windows (AW). The current study presents the development and testing of a new augmented reality software (Echo-QR) that would allow novice operators (with no medical imaging background) to correctly locate the ultrasound probe over the AW of interest without expert assistance. Methods: On the first day of the study, the positions of the probe over the AWs were identified for each organ by an expert sonographer and saved in the Echo-QR software. On the second day, the novices independently performed the ultrasound investigation using the Echo-QR software to correctly position the probe over each organ's AW. Results: Using the Echo-QR software, novice operators found the AW in 73 (92%) of the 79 organs. The 2D images acquired by the novices "2D direct image" were acceptable for medical evaluation in 41% of the cases. However, when the "2D direct image" did not show the entire organ, a 3D capture of the volume below the probe was also performed, which allowed for the extraction of the appropriate 2D image "2D/3D image" for medical evaluation in 85% of the cases. Discussion: Therefore, in the absence of real-time communication between an isolated participant and an expert sonographer, novel software (Echo-QR) and automated 3D volume capture can be used to obtain images usable for ultrasound diagnostics.

Augmented Reality Assisted Intraoperative Navigation Increases Precision of Glenoid Inclination in Reverse Shoulder Arthroplasty.

BACKGROUND: Reverse Total Shoulder Arthroplasty (RTSA) is the standard of care for patients with glenohumeral osteoarthritis and rotator cuff deficiency. Preoperative RTSA planning based on medical images and patient-specific instruments has been established over the last decade. This study aims to determine the effects of using augmented reality assisted intraoperative navigation (ARIN) for baseplate positioning in RTSA compared to preoperative planning. It is hypothesized that ARIN will decrease deviation between preoperative planning and postoperative baseplate positioning. Moreover, ARIN will decrease deviation between the (senior) more (> 50 RTSA/year) and less experienced (junior) surgeon (5-10 RTSA/year). METHODS: Preoperative CT scans of sixteen fresh-frozen cadaveric shoulders were obtained. Baseplate placements were planned using a validated software. The data was then converted and uploaded to the augmented reality system (NextAR; Medacta International, Castel San Pietro, Switzerland). Each of the eight RTSAs were implanted by a senior and a junior surgeon, with four RTSAs utilizing ARIN and four without. A postoperative CT scan was performed in all cases. The scanned scapulae were segmented, and the preoperative scan was laid over the postoperative scapula by nearest iterative point cloud analysis. The deviation from the planned entry point and trajectory was calculated regarding the inclination, retroversion, medialization (reaming depth)/lateralization, antero-posterior position and superior/inferior position of the baseplate. Data are reported as mean ± standard deviation (SD) or mean and 95% confidence interval (CI). P values <0.05 were considered statistically significant. RESULTS: The use of ARIN yielded a reduction in the absolute difference between planned and obtained inclination from 9° (SD: 4°) to 3° (SD°:2) (p=0.011). Mean difference in planned-obtained inclination between surgeons was 3° in free-hand surgeries (95%CI: -4,10, p=0.578), while this difference reduced to 1° (95%CI: -6, 7, p=0.996) using ARIN. Retroversion, medialization (reaming depth)/lateralization, antero-posterior position and superior/inferior position of the baseplate were not affected by using ARIN. Surgical duration was increased using ARIN for both the senior (10 mins) and junior (18 mins) surgeon. CONCLUSIONS: The implementation of augmented reality assisted intraoperative navigation (ARIN) leads to greater accuracy of glenoid component placement, specifically with respect to inclination. Further studies have to verify if this increased accuracy is clinical important. Furthermore, ARIN allows less experienced surgeons to achieve a similar level of accuracy in component placement comparable to more experienced surgeons. However, the potential advantages of ARIN in RTSA are counterbalanced by an increase in operative time.

Physical loads on upper extremity muscles while interacting with virtual objects in an augmented reality context.

Augmented reality (AR) environments are emerging as prominent user interfaces and gathering significant attention. However, the associated physical strain on the users presents a considerable challenge. Within this background, this study explores the impact of movement distance (MD) and target-to-user distance (TTU) on the physical load during drag-and-drop (DND) tasks in an AR environment. To address this objective, a user experiment was conducted utilizing a 5× 5 within-subject design with MD (16, 32, 48, 64, and 80 cm) and TTU (40, 80, 120, 160, and 200 cm) as the variables. Physical load was assessed using normalized electromyography (NEMG) (%MVC) indicators of the upper extremity muscles and the physical item of NASA-Task load index (TLX). The results revealed significant variations in the physical load based on MD and TTU. Specifically, both the NEMG and subjective physical workload values increased with increasing MD. Moreover, NEMG increased with decreasing TTU, whereas the subjective physical workload scores increased with increasing TTU. Interaction effects of MD and TTU on NEMG were also significantly observed. These findings suggest that considering the MD and TTU when developing content for interacting with AR objects in AR environments could potentially alleviate user load.

An Iterative Participatory Design Approach to Develop Collaborative Augmented Reality Activities for Older Adults in Long-Term Care Facilities.

Over four million older adults living in long-term care (LTC) communities experience loneliness, adversely impacting their health. Increased contact with friends and family is an evidence-based intervention to reduce loneliness, but in-person visits are not always possible. Augmented Reality (AR)-based telepresence activities can offer viable alternatives with increased immersion and presence compared to video calls. However, its feasibility as an interaction technology for older adults is not known. In this paper, we detail the design of two dyadic collaborative AR activities that accommodate diminished physical and cognitive abilities of older adults. The findings include a general design framework based on an iterative participatory design focusing on preferred activities, modes of interaction, and overall AR experience of eight older adults, two family members, and five LTC staff. Results demonstrate the potential of collaborative AR as an effective means of interaction for older adults with their family, if designed to cater to their needs.

Multidimensional Encryption by Chip-Integrated Metasurfaces.

Facing the challenge of information security in the current era of information technology, optical encryption based on metasurfaces presents a promising solution to this issue. However, most metasurface-based encryption techniques rely on limited decoding keys and struggle to achieve multidimensional complex encryption. It hinders the progress of optical storage capacity and puts encryption security at a disclosing risk. Here, we propose and experimentally demonstrate a multidimensional encryption system based on chip-integrated metasurfaces that successfully incorporates the simultaneous manipulation of three-dimensional optical parameters, including wavelength, direction, and polarization. Hence, up to eight-channel augmented reality (AR) holograms are concealed by near- and far-field fused encryption, which can only be extracted by correctly providing the three-dimensional decoding keys and then vividly exhibit to the authorizer with low crosstalk, high definition, and no zero-order speckle noise. We envision that the miniature chip-integrated metasurface strategy for multidimensional encryption functionalities promises a feasible route toward the encryption capacity and information security enhancement of the anticounterfeiting performance and optically cryptographic storage.