Reducing residual forces in spinal fusion using a custom-built rod bending machine.

BACKGROUND AND OBJECTIVE: As part of spinal fusion surgery, shaping the rod implant to align with the anatomy is a tedious, error-prone, and time-consuming manual process. Inadequately contoured rod implants introduce stress on the screw-bone interface of the pedicle screws, potentially leading to screw loosening or even pull-out. METHODS: We propose the first fully automated solution to the rod bending problem by leveraging the advantages of augmented reality and robotics. Augmented reality not only enables the surgeons to intraoperatively digitize the screw positions but also provides a human-computer interface to the wirelessly integrated custom-built rod bending machine. Furthermore, we introduce custom-built test rigs to quantify per screw absolute tensile/compressive residual forces on the screw-bone interface. Besides residual forces, we have evaluated the required bending times and reducer engagements, and compared our method to the freehand gold standard. RESULTS: We achieved a significant reduction of the average absolute residual forces from for the freehand gold standard to (p=0.0015) using the bending machine. Moreover, our bending machine reduced the average time to instrumentation per screw from to . Reducer engagements per rod were significantly decreased from an average of 1.00±1.14 to 0.11±0.32 (p=0.0037). CONCLUSION: The combination of augmented reality and robotics has the potential to improve surgical outcomes while minimizing the dependency on individual surgeon skill and dexterity.

Augmented reality-guided pelvic osteotomy of Ganz: feasibility in cadavers.

INTRODUCTION: The periacetabular osteotomy is a technically demanding procedure with the goal to improve the osseous containment of the femoral head. The options for controlled execution of the osteotomies and verification of the acetabular reorientation are limited. With the assistance of augmented reality, new possibilities are emerging to guide this intervention. However, the scientific knowledge regarding AR navigation for PAO is sparse. METHODS: In this cadaveric study, we wanted to find out, if the execution of this complex procedure is feasible with AR guidance, quantify the accuracy of the execution of the three-dimensional plan, and find out what has to be done to proceed to real surgery. Therefore, an AR guidance for the PAO was developed and applied on 14 human hip cadavers. The guidance included performance of the four osteotomies and reorientation of the acetabular fragment. The osteotomy starting points, the orientation of the osteotomy planes, as well as the reorientation of the acetabular fragment were compared to the 3D planning. RESULTS: The mean 3D distance between planned and performed starting points was between 9 and 17 mm. The mean angle between planned and performed osteotomies was between 6° and 7°. The mean reorientation error between the planned and performed rotation of the acetabular fragment was between 2° and 11°. CONCLUSION: The planned correction can be achieved with promising accuracy and without serious errors. Further steps for a translation from the cadaver to the patient have been identified and must be addressed in future work.

Automatic registration with continuous pose updates for marker-less surgical navigation in spine surgery.

Established surgical navigation systems for pedicle screw placement have been proven to be accurate, but still reveal limitations in registration or surgical guidance. Registration of preoperative data to the intraoperative anatomy remains a time-consuming, error-prone task that includes exposure to harmful radiation. Surgical guidance through conventional displays has well-known drawbacks, as information cannot be presented in-situ and from the surgeon's perspective. Consequently, radiation-free and more automatic registration methods with subsequent surgeon-centric navigation feedback are desirable. In this work, we present a marker-less approach that automatically solves the registration problem for lumbar spinal fusion surgery in a radiation-free manner. A deep neural network was trained to segment the lumbar spine and simultaneously predict its orientation, yielding an initial pose for preoperative models, which then is refined for each vertebra individually and updated in real-time with GPU acceleration while handling surgeon occlusions. An intuitive surgical guidance is provided thanks to the integration into an augmented reality based navigation system. The registration method was verified on a public dataset with a median of 100% successful registrations, a median target registration error of 2.7 mm, a median screw trajectory error of 1.6°and a median screw entry point error of 2.3 mm. Additionally, the whole pipeline was validated in an ex-vivo surgery, yielding a 100% screw accuracy and a median target registration error of 1.0 mm. Our results meet clinical demands and emphasize the potential of RGB-D data for fully automatic registration approaches in combination with augmented reality guidance.

Factors affecting augmented reality head-mounted device performance in real OR.

PURPOSE: Over the last years, interest and efforts to implement augmented reality (AR) in orthopedic surgery through head-mounted devices (HMD) have increased. However, the majority of experiments were preclinical and within a controlled laboratory environment. The operating room (OR) is a more challenging environment with various confounding factors potentially affecting the performance of an AR-HMD. The aim of this study was to assess the performance of an AR-HMD in a real-life OR setting. METHODS: An established AR application using the HoloLens 2 HMD was tested in an OR and in a laboratory by two users. The accuracy of the hologram overlay, the time to complete the trial, the number of rejected registration attempts, the delay in live overlay of the hologram, and the number of completely failed runs were recorded. Further, different OR setting parameters (light condition, setting up partitions, movement of personnel, and anchor placement) were modified and compared. RESULTS: Time for full registration was higher with 48 s (IQR 24 s) in the OR versus 33 s (IQR 10 s) in the laboratory setting (p < 0.001). The other investigated parameters didn't differ significantly if an optimal OR setting was used. Within the OR, the strongest influence on performance of the AR-HMD was different light conditions with direct light illumination on the situs being the least favorable. CONCLUSION: AR-HMDs are affected by different OR setups. Standardization measures for better AR-HMD performance include avoiding direct light illumination on the situs, setting up partitions, and minimizing the movement of personnel.

Marker-free surgical navigation of rod bending using a stereo neural network and augmented reality in spinal fusion.

The instrumentation of spinal fusion surgeries includes pedicle screw placement and rod implantation. While several surgical navigation approaches have been proposed for pedicle screw placement, less attention has been devoted towards the guidance of patient-specific adaptation of the rod implant. We propose a marker-free and intuitive Augmented Reality (AR) approach to navigate the bending process required for rod implantation. A stereo neural network is trained from the stereo video streams of the Microsoft HoloLens in an end-to-end fashion to determine the location of corresponding pedicle screw heads. From the digitized screw head positions, the optimal rod shape is calculated, translated into a set of bending parameters, and used for guiding the surgeon with a novel navigation approach. In the AR-based navigation, the surgeon is guided step-by-step in the use of the surgical tools to achieve an optimal result. We have evaluated the performance of our method on human cadavers against two benchmark methods, namely conventional freehand bending and marker-based bending navigation in terms of bending time and rebending maneuvers. We achieved an average bending time of 231s with 0.6 rebending maneuvers per rod compared to 476s (3.5 rebendings) and 348s (1.1 rebendings) obtained by our freehand and marker-based benchmarks, respectively.

Augmented reality-navigated pedicle screw placement: a cadaveric pilot study.

PURPOSE: Augmented reality (AR) is an emerging technology with great potential for surgical navigation through its ability to provide 3D holographic projection of otherwise hidden anatomical information. This pilot cadaver study investigated the feasibility and accuracy of one of the first holographic navigation techniques for lumbar pedicle screw placement. METHODS: Lumbar computer tomography scans (CT) of two cadaver specimens and their reconstructed 3D models were used for pedicle screw trajectory planning. Planned trajectories and 3D models were subsequently uploaded to an AR head-mounted device. Randomly, k-wires were placed either into the left or the right pedicle of a vertebra (L1-5) with or without AR-navigation (by holographic projection of the planned trajectory). CT-scans were subsequently performed to assess accuracy of both techniques. RESULTS: A total of 18 k-wires could be placed (8 navigated, 10 free hand) by two experienced spine surgeons. In two vertebrae, the AR-navigation was aborted because the registration of the preoperative plan with the intraoperative anatomy was imprecise due to a technical failure. The average differences of the screw entry points between planning and execution were 4.74 ± 2.37 mm in the freehand technique and 5.99 ± 3.60 mm in the AR-navigated technique (p = 0.39). The average deviation from the planned trajectories was 11.21° ± 7.64° in the freehand technique and 5.88° ± 3.69° in the AR-navigated technique (p = 0.09). CONCLUSION: This pilot study demonstrates improved angular precision in one of the first AR-navigated pedicle screw placement studies worldwide. Technical shortcomings need to be eliminated before potential clinical applications.

SpineDepth: A Multi-Modal Data Collection Approach for Automatic Labelling and Intraoperative Spinal Shape Reconstruction Based on RGB-D Data.

Computer aided orthopedic surgery suffers from low clinical adoption, despite increased accuracy and patient safety. This can partly be attributed to cumbersome and often radiation intensive registration methods. Emerging RGB-D sensors combined with artificial intelligence data-driven methods have the potential to streamline these procedures. However, developing such methods requires vast amount of data. To this end, a multi-modal approach that enables acquisition of large clinical data, tailored to pedicle screw placement, using RGB-D sensors and a co-calibrated high-end optical tracking system was developed. The resulting dataset comprises RGB-D recordings of pedicle screw placement along with individually tracked ground truth poses and shapes of spine levels L1-L5 from ten cadaveric specimens. Besides a detailed description of our setup, quantitative and qualitative outcome measures are provided. We found a mean target registration error of 1.5 mm. The median deviation between measured and ground truth bone surface was 2.4 mm. In addition, a surgeon rated the overall alignment based on 10% random samples as 5.8 on a scale from 1 to 6. Generation of labeled RGB-D data for orthopedic interventions with satisfactory accuracy is feasible, and its publication shall promote future development of data-driven artificial intelligence methods for fast and reliable intraoperative registration.

Intraoperative hyperspectral label-free imaging: from system design to first-in-patient translation.

Despite advances in intraoperative surgical imaging, reliable discrimination of critical tissue during surgery remains challenging. As a result, decisions with potentially life-changing consequences for patients are still based on the surgeon's subjective visual assessment. Hyperspectral imaging (HSI) provides a promising solution for objective intraoperative tissue characterisation, with the advantages of being non-contact, non-ionising and non-invasive. However, while its potential to aid surgical decision-making has been investigated for a range of applications, to date no real-time intraoperative HSI (iHSI) system has been presented that follows critical design considerations to ensure a satisfactory integration into the surgical workflow. By establishing functional and technical requirements of an intraoperative system for surgery, we present an iHSI system design that allows for real-time wide-field HSI and responsive surgical guidance in a highly constrained operating theatre. Two systems exploiting state-of-the-art industrial HSI cameras, respectively using linescan and snapshot imaging technology, were designed and investigated by performing assessments against established design criteria and ex vivo tissue experiments. Finally, we report the use of our real-time iHSI system in a clinical feasibility case study as part of a spinal fusion surgery. Our results demonstrate seamless integration into existing surgical workflows.

Augmented Reality Navigated Sacral-Alar-Iliac Screw Insertion.

BACKGROUND: Sacral-alar-iliac (SAI) screws are increasingly used for lumbo-pelvic fixation procedures. Insertion of SAI screws is technically challenging, and surgeons often rely on costly and time-consuming navigation systems. We investigated the accuracy and precision of an augmented reality (AR)-based and commercially available head-mounted device requiring minimal infrastructure. METHODS: A pelvic sawbone model served to drill pilot holes of 80 SAI screw trajectories by 2 surgeons, randomly either freehand (FH) without any kind of navigation or with AR navigation. The number of primary pilot hole perforations, simulated screw perforation, minimal axis/outer cortical wall distance, true sagittal cranio-caudal inclination angle (tSCCIA), true axial medio-lateral angle, and maximal screw length (MSL) were measured and compared to predefined optimal values. RESULTS: In total, 1/40 (2.5%) of AR-navigated screw hole trajectories showed a perforation before passing the inferior gluteal line compared to 24/40 (60%) of FH screw hole trajectories (P < .05). The differences between FH- and AR-guided holes compared to optimal values were significant for tSCCIA with -10.8° ± 11.77° and MSL -65.29 ± 15 mm vs 55.04 ± 6.76 mm (P = .001). CONCLUSIONS: In this study, the additional anatomical information provided by the AR headset and the superimposed operative plan improved the precision of drilling pilot holes for SAI screws in a laboratory setting compared to the conventional FH technique. Further technical development and validation studies are currently being performed to investigate potential clinical benefits of the AR-based navigation approach described here. LEVEL OF EVIDENCE: 4.

Operator independent reliability of direct augmented reality navigated pedicle screw placement and rod bending.

BACKGROUND: AR based navigation of spine surgeries may not only provide accurate surgical execution but also operator independency by compensating for potential skill deficits. "Direct" AR-navigation, namely superposing trajectories on anatomy directly, have not been investigated regarding their accuracy and operator's dependence.Purpose of this study was to prove operator independent reliability and accuracy of both AR assisted pedicle screw navigation and AR assisted rod bending in a cadaver setting. METHODS: Two experienced spine surgeons and two biomedical engineers (laymen) performed independently from each other pedicle screw instrumentations from L1-L5 in a total of eight lumbar cadaver specimens (20 screws/operator) using a fluoroscopy-free AR based navigation method. Screw fitting rods from L1 to S2-Ala-Ileum were bent bilaterally using an AR based rod bending navigation method (4 rods/operator). Outcome measures were pedicle perforations, accuracy compared to preoperative plan, registration time, navigation time, total rod bending time and operator's satisfaction for these procedures. RESULTS: 97.5% of all screws were safely placed (<2 mm perforation), overall mean deviation from planned trajectory was 6.8±3.9°, deviation from planned entry point was 4±2.7 mm, registration time per vertebra was 2:25 min (00:56 to 10:00 min), navigation time per screw was 1:07 min (00:15 to 12:43 min) rod bending time per rod was 4:22 min (02:07 to 10:39 min), operator's satisfaction with AR based screw and rod navigation was 5.38±0.67 (1 to 6, 6 being the best rate). Comparison of surgeons and laymen revealed significant difference in navigation time (1:01 min; 00:15 to 3:00 min vs. 01:37 min; 00:23 to 12:43 min; p = 0.004, respectively) but not in pedicle perforation rate. CONCLUSIONS: Direct AR based screw and rod navigation using a surface digitization registration technique is reliable and independent of surgical experience. The accuracy of pedicle screw insertion in the lumbar spine is comparable with the current standard techniques.

HoloYolo: A proof-of-concept study for marker-less surgical navigation of spinal rod implants with augmented reality and on-device machine learning.

BACKGROUND: Existing surgical navigation approaches of the rod bending procedure in spinal fusion rely on optical tracking systems that determine the location of placed pedicle screws using a hand-held marker. METHODS: We propose a novel, marker-less surgical navigation proof-of-concept to bending rod implants. Our method combines augmented reality with on-device machine learning to generate and display a virtual template of the optimal rod shape without touching the instrumented anatomy. Performance was evaluated on lumbosacral spine phantoms against a pointer-based navigation benchmark approach and ground truth data obtained from computed tomography. RESULTS: Our method achieved a mean error of 1.83 ± 1.10 mm compared to 1.87 ± 1.31 mm measured in the marker-based approach, while only requiring 21.33 ± 8.80 s as opposed to 36.65 ± 7.49 s attained by the pointer-based method. CONCLUSION: Our results suggests that the combination of augmented reality and machine learning has the potential to replace conventional pointer-based navigation in the future.

Augmented reality for base plate component placement in reverse total shoulder arthroplasty: a feasibility study.

BACKGROUND: Accurate glenoid positioning in reverse total shoulder arthroplasty (RSA) is important to achieve satisfying functional outcome and prosthesis longevity. Optimal component placement can be challenging, especially in severe glenoid deformities. The use of patient-specific instruments (PSI) and 3D computer-assisted optical tracking navigation (NAV) are already established methods to improve surgical precision. Augmented reality technology (AR) promises similar results at low cost and ease of use. With AR, the planned component placement can be superimposed to the surgical situs and shown directly in the operating field using a head mounted display. We introduce a new navigation technique using AR via head mounted display for surgical navigation in this feasibility study, aiming to improve and enhance the surgical planning. METHODS: 3D surface models of ten human scapulae were printed from computed tomography (CT) data of cadaver scapulae. Guidewire positioning of the central back of the glenoid baseplate was planned with a dedicated computer software. A hologram of the planned guidewire with dynamic navigation was then projected onto the 3D-created models of the cadaver shoulders. The registration of the plan to the anatomy was realized by digitizing the glenoid surface and the base of the coracoid with optical tracking using a fiducial marker. After navigated placement of the central guidewires, another CT imaging was recorded, and the 3D model was superimposed with the preoperative planning to analyze the deviation from the planned and executed central guides trajectory and entry point. RESULTS: The mean deviation of the ten placed guidewires from the planned trajectory was 2.7° ± 1.3° (95% CI 1.9°; 3.6°). The mean deviation to the planned entry point of the ten placed guidewires measured 2.3 mm ± 1.1 mm (95% CI 1.5 mm; 3.1 mm). CONCLUSION: AR may be a promising new technology for highly precise surgical execution of 3D preoperative planning in RSA.

Augmented reality navigation for spinal pedicle screw instrumentation using intraoperative 3D imaging.

BACKGROUND CONTEXT: Due to recent developments in augmented reality with head-mounted devices, holograms of a surgical plan can be displayed directly in the surgeon's field of view. To the best of our knowledge, three dimensional (3D) intraoperative fluoroscopy has not been explored for the use with holographic navigation by head-mounted devices in spine surgery. PURPOSE: To evaluate the surgical accuracy of holographic pedicle screw navigation by head-mounted device using 3D intraoperative fluoroscopy. STUDY DESIGN: In this experimental cadaver study, the accuracy of surgical navigation using a head-mounted device was compared with navigation with a state-of-the-art pose-tracking system. METHODS: Three lumbar cadaver spines were embedded in nontransparent agar gel, leaving only commonly visible anatomy in sight. Intraoperative registration of preoperative planning was achieved by 3D fluoroscopy and fiducial markers attached to lumbar vertebrae. Trackable custom-made drill sleeve guides enabled real-time navigation. In total, 20 K-wires were navigated into lumbar pedicles using AR-navigation, 10 K-wires by the state-of-the-art pose-tracking system. 3D models obtained from postexperimental CT scans were used to measure surgical accuracy. MF is the founder and shareholder of Incremed AG, a Balgrist University Hospital start-up focusing on the development of innovative techniques for surgical executions. The other authors declare no conflict of interest concerning the contents of this study. No external funding was received for this study. RESULTS: No significant difference in accuracy was measured between AR-navigated drillings and the gold standard with pose-tracking system with mean translational errors between entry points (3D vector distance; p=.85) of 3.4±1.6 mm compared with 3.2±2.0 mm, and mean angular errors between trajectories (3D angle; p=.30) of 4.3°±2.3° compared with 3.5°±1.4°. CONCLUSIONS: In conclusion, holographic navigation by use of a head-mounted device achieve accuracy comparable to the gold standard of high-end pose-tracking systems. CLINICAL SIGNIFICANCE: These promising results could result in a new way of surgical navigation with minimal infrastructural requirements but now have to be confirmed in clinical studies.

Pedicle screw navigation using surface digitization on the Microsoft HoloLens.

PURPOSE: In spinal fusion surgery, imprecise placement of pedicle screws can result in poor surgical outcome or may seriously harm a patient. Patient-specific instruments and optical systems have been proposed for improving precision through surgical navigation compared to freehand insertion. However, existing solutions are expensive and cannot provide in situ visualizations. Recent technological advancement enabled the production of more powerful and precise optical see-through head-mounted displays for the mass market. The purpose of this laboratory study was to evaluate whether such a device is sufficiently precise for the navigation of lumbar pedicle screw placement. METHODS: A novel navigation method, tailored to run on the Microsoft HoloLens, was developed. It comprises capturing of the intraoperatively reachable surface of vertebrae to achieve registration and tool tracking with real-time visualizations without the need of intraoperative imaging. For both surface sampling and navigation, 3D printable parts, equipped with fiducial markers, were employed. Accuracy was evaluated within a self-built setup based on two phantoms of the lumbar spine. Computed tomography (CT) scans of the phantoms were acquired to carry out preoperative planning of screw trajectories in 3D. A surgeon placed the guiding wire for the pedicle screw bilaterally on ten vertebrae guided by the navigation method. Postoperative CT scans were acquired to compare trajectory orientation (3D angle) and screw insertion points (3D distance) with respect to the planning. RESULTS: The mean errors between planned and executed screw insertion were [Formula: see text] for the screw trajectory orientation and 2.77±1.46 mm for the insertion points. The mean time required for surface digitization was 125±27 s. CONCLUSIONS: First promising results under laboratory conditions indicate that precise lumbar pedicle screw insertion can be achieved by combining HoloLens with our proposed navigation method. As a next step, cadaver experiments need to be performed to confirm the precision on real patient anatomy.