Determination of rotation center and diameter of femoral heads using off-the-shelf augmented reality hardware for navigation.

In total hip arthroplasty (THA), determining the center of rotation (COR) and diameter of the hip joint (acetabulum and femoral head) is essential to restore patient biomechanics. This study investigates on-the-fly determination of hip COR and size, using off-the-shelf augmented reality (AR) hardware. An AR head-mounted device (HMD) was configured with inside-out infrared tracking enabling the determination of surface coordinates using a handheld stylus. Two investigators examined 10 prosthetic femoral heads and cups, and 10 human femurs. The HMD calculated the diameter and COR through sphere fitting. Results were compared to data obtained from either verified prosthetic geometry or post-hoc CT analysis. Repeated single-observer measurements showed a mean diameter error of 0.63 mm ± 0.48 mm for the prosthetic heads and 0.54 mm ± 0.39 mm for the cups. Inter-observer comparison yielded mean diameter errors of 0.28 mm ± 0.71 mm and 1.82 mm ± 1.42 mm for the heads and cups, respectively. Cadaver testing found a mean COR error of 3.09 mm ± 1.18 mm and a diameter error of 1.10 mm ± 0.90 mm. Intra- and inter-observer reliability averaged below 2 mm. AR-based surface mapping using HMD proved accurate and reliable in determining the diameter of THA components with promise in identifying COR and diameter of osteoarthritic femoral heads.

On-Device Execution of Deep Learning Models on HoloLens2 for Real-Time Augmented Reality Medical Applications.

The integration of Deep Learning (DL) models with the HoloLens2 Augmented Reality (AR) headset has enormous potential for real-time AR medical applications. Currently, most applications execute the models on an external server that communicates with the headset via Wi-Fi. This client-server architecture introduces undesirable delays and lacks reliability for real-time applications. However, due to HoloLens2's limited computation capabilities, running the DL model directly on the device and achieving real-time performances is not trivial. Therefore, this study has two primary objectives: (i) to systematically evaluate two popular frameworks to execute DL models on HoloLens2-Unity Barracuda and Windows Machine Learning (WinML)-using the inference time as the primary evaluation metric; (ii) to provide benchmark values for state-of-the-art DL models that can be integrated in different medical applications (e.g., Yolo and Unet models). In this study, we executed DL models with various complexities and analyzed inference times ranging from a few milliseconds to seconds. Our results show that Unity Barracuda is significantly faster than WinML (p-value < 0.005). With our findings, we sought to provide practical guidance and reference values for future studies aiming to develop single, portable AR systems for real-time medical assistance.

Exploring technology acceptance of head-mounted device-based augmented reality surgical navigation in orthopaedic surgery.

BACKGROUND: This study used the Unified Theory of Acceptance and Use of Technology (UTAUT) to investigate the acceptance of HMD-based AR surgical navigation. METHODS: An experiment was conducted in which participants drilled 12 predefined holes using freehand drilling, proprioceptive control, and AR assistance. Technology acceptance was assessed through a survey and non-participant observations. RESULTS: Participants' intention to use AR correlated (p < 0.05) with social influence (Spearman's rho (rs) = 0.599), perceived performance improvement (rs = 0.592) and attitude towards AR (rs = 0.542). CONCLUSIONS: While most participants acknowledged the potential of AR, they also highlighted persistent barriers to adoption, such as issues related to user-friendliness, time efficiency and device discomfort. To overcome these challenges, future AR surgical navigation systems should focus on enhancing surgical performance while minimising disruptions to workflows and operating times. Engaging orthopaedic surgeons in the development process can facilitate the creation of tailored solutions and accelerate adoption.

Neuro-oncological augmented reality planning for intracranial tumor resection.

Background: Before starting surgery for the resection of an intracranial tumor, its outlines are typically marked on the skin of the patient. This allows for the planning of the optimal skin incision, craniotomy, and angle of approach. Conventionally, the surgeon determines tumor borders using neuronavigation with a tracked pointer. However, interpretation errors can lead to important deviations, especially for deep-seated tumors, potentially resulting in a suboptimal approach with incomplete exposure. Augmented reality (AR) allows displaying of the tumor and critical structures directly on the patient, which can simplify and improve surgical preparation. Methods: We developed an AR-based workflow for intracranial tumor resection planning deployed on the Microsoft HoloLens II, which exploits the built-in infrared-camera for tracking the patient. We initially performed a phantom study to assess the accuracy of the registration and tracking. Following this, we evaluated the AR-based planning step in a prospective clinical study for patients undergoing resection of a brain tumor. This planning step was performed by 12 surgeons and trainees with varying degrees of experience. After patient registration, tumor outlines were marked on the patient's skin by different investigators, consecutively using a conventional neuronavigation system and an AR-based system. Their performance in both registration and delineation was measured in terms of accuracy and duration and compared. Results: During phantom testing, registration errors remained below 2.0 mm and 2.0° for both AR-based navigation and conventional neuronavigation, with no significant difference between both systems. In the prospective clinical trial, 20 patients underwent tumor resection planning. Registration accuracy was independent of user experience for both AR-based navigation and the commercial neuronavigation system. AR-guided tumor delineation was deemed superior in 65% of cases, equally good in 30% of cases, and inferior in 5% of cases when compared to the conventional navigation system. The overall planning time (AR = 119 ± 44 s, conventional = 187 ± 56 s) was significantly reduced through the adoption of the AR workflow (p < 0.001), with an average time reduction of 39%. Conclusion: By providing a more intuitive visualization of relevant data to the surgeon, AR navigation provides an accurate method for tumor resection planning that is quicker and more intuitive than conventional neuronavigation. Further research should focus on intraoperative implementations.

The effect of augmented reality on the accuracy and learning curve of external ventricular drain placement.

OBJECTIVE: The traditional freehand technique for external ventricular drain (EVD) placement is most frequently used, but remains the primary risk factor for inaccurate drain placement. As this procedure could benefit from image guidance, the authors set forth to demonstrate the impact of augmented-reality (AR) assistance on the accuracy and learning curve of EVD placement compared with the freehand technique. METHODS: Sixteen medical students performed a total of 128 EVD placements on a custom-made phantom head, both before and after receiving a standardized training session. They were guided by either the freehand technique or by AR, which provided an anatomical overlay and tailored guidance for EVD placement through inside-out infrared tracking. The outcome was quantified by the metric accuracy of EVD placement as well as by its clinical quality. RESULTS: The mean target error was significantly impacted by either AR (p = 0.003) or training (p = 0.02) in a direct comparison with the untrained freehand performance. Both untrained (11.9 ± 4.5 mm) and trained (12.2 ± 4.7 mm) AR performances were significantly better than the untrained freehand performance (19.9 ± 4.2 mm), which improved after training (13.5 ± 4.7 mm). The quality of EVD placement as assessed by the modified Kakarla scale (mKS) was significantly impacted by AR guidance (p = 0.005) but not by training (p = 0.07). Both untrained and trained AR performances (59.4% mKS grade 1 for both) were significantly better than the untrained freehand performance (25.0% mKS grade 1). Spatial aptitude testing revealed a correlation between perceptual ability and untrained AR-guided performance (r = 0.63). CONCLUSIONS: Compared with the freehand technique, AR guidance for EVD placement yielded a higher outcome accuracy and quality for procedure novices. With AR, untrained individuals performed as well as trained individuals, which indicates that AR guidance not only improved performance but also positively impacted the learning curve. Future efforts will focus on the translation and evaluation of AR for EVD placement in the clinical setting.

Augmented Reality-Assisted Neurosurgical Drain Placement (ARANED): Technical Note.

BACKGROUND: Many surgical procedures, such as placement of intracranial drains, are currently being performed blindly, relying on anatomical landmarks. As a result, accuracy results still have room for improvement. Neuronavigation could address this issue, but its application in an urgent setting is often impractical. Augmented reality (AR) provided through a head-worn device has the potential to tackle this problem, but its implementation should meet physicians' needs. METHODS: The Surgical Augmented Reality Assistance (SARA) project aims to develop an AR solution that is suitable for preoperative planning, intraoperative visualisation and navigational support in an everyday clinical setting, using a Microsoft HoloLens. RESULTS: Proprietary hardware and software adaptations and dedicated navigation algorithms are applied to the Microsoft HoloLens to optimise it specifically for neurosurgical navigation. This includes a pipeline with an additional set of advanced, semi-automated algorithms responsible for image processing, hologram-to-patient registration and intraoperative tracking using infrared depth-sensing. A smooth and efficient workflow while maintaining high accuracy is prioritised. The AR solution provides a fully integrated and completely mobile navigation setup. Initial preclinical and clinical validation tests applying the solution to intracranial drain placement are described. CONCLUSION: AR has the potential to vastly increase accuracy of everyday procedures that are frequently performed without image guidance, but could still benefit from navigational support, such as intracranial drain placements. Technical development should go hand in hand with preclinical and clinical validation in order to demonstrate improvements in accuracy and clinical outcomes.

Augmenting Microsoft's HoloLens with vuforia tracking for neuronavigation.

Major hurdles for Microsoft's HoloLens as a tool in medicine have been accessing tracking data, as well as a relatively high-localisation error of the displayed information; cumulatively resulting in its limited use and minimal quantification. The following work investigates the augmentation of HoloLens with the proprietary image processing SDK Vuforia, allowing integration of data from its front-facing RGB camera to provide more spatially stable holograms for neuronavigational use. Continuous camera tracking was able to maintain hologram registration with a mean perceived drift of 1.41 mm, as well as a mean sub 2-mm surface point localisation accuracy of 53%, all while allowing the researcher to walk about a test area. This represents a 68% improvement for the later and a 34% improvement for the former compared with a typical HoloLens deployment used as a control. Both represent a significant improvement on hologram stability given the current state-of-the-art, and to the best of the authors knowledge are the first example of quantified measurements when augmenting hologram stability using data from the RGB sensor.