Evaluating the feasibility of using augmented reality for tooth preparation.

OBJECTIVES: Tooth preparation is complicated because it requires the preparation of an abutment while simultaneously predicting the ideal shape of the tooth. This study aimed to develop and evaluate a system using augmented reality (AR) head-mounted displays (HMDs) that provide dynamic navigation capabilities for tooth preparation. METHODS: The proposed system utilizes optical see-through HMDs to overlay digital information onto the real world and enrich the user's environment. By integrating tracking algorithms and three-dimensional modeling, the system provides real-time visualization and navigation capabilities during tooth preparation by using two different visualization techniques. The experimental setup involved a comprehensive analysis of the distance to the surface and cross-sectional angles between the ideal and prepared teeth using three scenarios: traditional (without AR), overlay (AR-assisted visualization of the ideal prepared tooth), and cross-sectional (AR-assisted visualization with cross-sectional views and angular displays). RESULTS: A user study (N = 24) revealed that the cross-sectional approach was more effective for angle adjustment and reduced the occurrence of over-reduction. Additional questionnaires revealed that the AR-assisted approaches were perceived as less difficult, with the cross-sectional approach excelling in terms of performance. CONCLUSIONS: Visualization and navigation using cross-sectional approaches have the potential to support safer tooth preparation with less overreduction than traditional and overlay approaches do. The angular displays provided by the cross-sectional approach are considered helpful for tooth preparation. CLINICAL SIGNIFICANCE: The AR navigation system can assist dentists during tooth preparation and has the potential to enhance the accuracy and safety of prosthodontic treatment.

Autokinesis Reveals a Threshold for Perception of Visual Motion.

In natural viewing conditions, the brain can optimally integrate retinal and extraretinal signals to maintain a stable visual perception. These mechanisms, however, may fail in circumstances where extraction of a motion signal is less viable such as impoverished visual scenes. This can result in a phenomenon known as autokinesis in which one may experience apparent motion of a small visual stimulus in an otherwise completely dark environment. In this study, we examined the effect of autokinesis on visual perception of motion in human observers. We used a novel method with optical tracking in which the visual motion was reported manually by the observer. Experiment results show at lower speeds of motion, the perceived direction of motion was more aligned with the effect of autokinesis, whereas in the light or at higher speeds in the dark, it was more aligned with the actual direction of motion. These findings have important implications for understanding how the stability of visual representation in the brain can affect accurate perception of motion signals.

Towards Reducing Visual Workload in Surgical Navigation: Proof-of-concept of an Augmented Reality Haptic Guidance System.

The integration of navigation capabilities into the operating room has enabled surgeons take on more precise procedures guided by a pre-operative plan. Traditionally, navigation information based on this plan is presented using monitors in the surgical theater. But the monitors force the surgeon to frequently look away from the surgical area. Alternative technologies, such as augmented reality, have enabled surgeons to visualize navigation information in-situ. However, burdening the visual field with additional information can be distracting. In this work, we propose integrating haptic feedback into a surgical tool handle to enable surgical guidance capabilities. This property reduces the amount of visual information, freeing surgeons to maintain visual attention over the patient and the surgical site. To investigate the feasibility of this guidance paradigm we conducted a pilot study with six subjects. Participants traced paths, pinpointed locations and matched alignments with a mock surgical tool featuring a novel haptic handle. We collected quantitative data, tracking user's accuracy and time to completion as well as subjective cognitive load. Our results show that haptic feedback can guide participants using a tool to sub-millimeter and sub-degree accuracy with only little training. Participants were able to match a location with an average error of 0.82 mm, desired pivot alignments with an average error of 0.83° and desired rotations to 0.46°.

Synthetic data accelerates the development of generalizable learning-based algorithms for X-ray image analysis.

Artificial intelligence (AI) now enables automated interpretation of medical images. However, AI's potential use for interventional image analysis remains largely untapped. This is because the post hoc analysis of data collected during live procedures has fundamental and practical limitations, including ethical considerations, expense, scalability, data integrity and a lack of ground truth. Here we demonstrate that creating realistic simulated images from human models is a viable alternative and complement to large-scale in situ data collection. We show that training AI image analysis models on realistically synthesized data, combined with contemporary domain generalization techniques, results in machine learning models that on real data perform comparably to models trained on a precisely matched real data training set. We find that our model transfer paradigm for X-ray image analysis, which we refer to as SyntheX, can even outperform real-data-trained models due to the effectiveness of training on a larger dataset. SyntheX provides an opportunity to markedly accelerate the conception, design and evaluation of X-ray-based intelligent systems. In addition, SyntheX provides the opportunity to test novel instrumentation, design complementary surgical approaches, and envision novel techniques that improve outcomes, save time or mitigate human error, free from the ethical and practical considerations of live human data collection.

Feasibility Study of Using Augmented Mirrors for Alignment Task during Orthopaedic Procedures in Mixed Reality.

Accurate depth estimation poses a significant challenge in egocentric Augmented Reality (AR), particularly for precision-dependent tasks in the medical field, such as needle or tool insertions during percutaneous procedures. Augmented Mirrors (AMs) provide a unique solution to this problem by offering additional non-egocentric viewpoints that enhance spatial understanding of an AR scene. Despite the perceptual advantages of using AMs, their practical utility has yet to be thoroughly tested. In this work, we present results from a pilot study involving five participants tasked with simulating epidural injection procedures in an AR environment, both with and without the aid of an AM. Our findings indicate that using AM contributes to reducing mental effort while improving alignment accuracy. These results highlight the potential of AM as a powerful tool for AR-enabled medical procedures, setting the stage for future exploration involving medical professionals.

Pelphix: Surgical Phase Recognition from X-ray Images in Percutaneous Pelvic Fixation.

Surgical phase recognition (SPR) is a crucial element in the digital transformation of the modern operating theater. While SPR based on video sources is well-established, incorporation of interventional X-ray sequences has not yet been explored. This paper presents Pelphix, a first approach to SPR for X-ray-guided percutaneous pelvic fracture fixation, which models the procedure at four levels of granularity - corridor, activity, view, and frame value - simulating the pelvic fracture fixation workflow as a Markov process to provide fully annotated training data. Using added supervision from detection of bony corridors, tools, and anatomy, we learn image representations that are fed into a transformer model to regress surgical phases at the four granularity levels. Our approach demonstrates the feasibility of X-ray-based SPR, achieving an average accuracy of 99.2% on simulated sequences and 71.7% in cadaver across all granularity levels, with up to 84% accuracy for the target corridor in real data. This work constitutes the first step toward SPR for the X-ray domain, establishing an approach to categorizing phases in X-ray-guided surgery, simulating realistic image sequences to enable machine learning model development, and demonstrating that this approach is feasible for the analysis of real procedures. As X-ray-based SPR continues to mature, it will benefit procedures in orthopedic surgery, angiography, and interventional radiology by equipping intelligent surgical systems with situational awareness in the operating room.

Data-driven Shape Sensing of Continuum Dexterous Manipulators Using Embedded Capacitive Sensor.

We propose a novel inexpensive embedded capacitive sensor (ECS) for sensing the shape of Continuum Dexterous Manipulators (CDMs). Our approach addresses some limitations associated with the prevalent Fiber Bragg Grating (FBG) sensors, such as temperature sensitivity and high production costs. ECSs are calibrated using a vision-based system. The calibration of the ECS is performed by a recurrent neural network that uses the kinematic data collected from the vision-based system along with the uncalibrated data from ECSs. We evaluated the performance on a 3D printed prototype of a cable-driven CDM with multiple markers along its length. Using data from three ECSs along the length of the CDM, we computed the angle and position of its tip with respect to its base and compared the results to the measurements of the visual-based system. We found a 6.6% tip position error normalized to the length of the CDM. The work shows the early feasibility of using ECSs for shape sensing and feedback control of CDMs and discusses potential future improvements.

Design and Fabrication of a Fiber Bragg Grating Shape Sensor for Shape Reconstruction of a Continuum Manipulator.

Continuum dexterous manipulators (CDMs) are suitable for performing tasks in a constrained environment due to their high dexterity and maneuverability. Despite the inherent advantages of CDMs in minimally invasive surgery, real-time control of CDMs' shape during nonconstant curvature bending is still challenging. This study presents a novel approach for the design and fabrication of a large deflection fiber Bragg grating (FBG) shape sensor embedded within the lumens inside the walls of a CDM with a large instrument channel. The shape sensor consisted of two fibers, each with three FBG nodes. A shape-sensing model was introduced to reconstruct the centerline of the CDM based on FBG wavelengths. Different experiments, including shape sensor tests and CDM shape reconstruction tests, were conducted to assess the overall accuracy of the shape-sensing. The FBG sensor evaluation results revealed the linear curvature-wavelength relationship with the large curvature detection of 0.045 mm and a high wavelength shift of up to 5.50 nm at a 90° bending angle in both the bending directions. The CDM's shape reconstruction experiments in a free environment demonstrated the shape-tracking accuracy of 0.216 ± 0.126 mm for positive/negative deflections. Also, the CDM shape reconstruction error for three cases of bending with obstacles was observed to be 0.436 ± 0.370 mm for the proximal case, 0.485 ± 0.418 mm for the middle case, and 0.312 ± 0.261 mm for the distal case. This study indicates the adequate performance of the FBG sensor and the effectiveness of the model for tracking the shape of the large-deflection CDM with nonconstant-curvature bending for minimally invasive orthopedic applications.

Towards Visualizing Early-stage Osteonecrosis using Intraoperative Imaging Modalities.

Osteonecrosis of the Femoral Head (ONFH) is a progressive disease characterized by the death of bone cells due to the loss of blood supply. Early detection and treatment of this disease are vital in avoiding Total Hip Replacement. Early stages of ONFH can be diagnosed using Magnetic Resonance Imaging (MRI), commonly used intra-operative imaging modalities such as fluoroscopy frequently fail to depict the lesion. Therefore, increasing the difficulty of intra-operative localization of osteonecrosis. This work introduces a novel framework that enables the localization of necrotic lesions in Computed Tomography (CT) as a step toward localizing and visualizing necrotic lesions in intra-operative images. The proposed framework uses Deep Learning algorithms to enable automatic segmentation of femur, pelvis, and necrotic lesions in MRI. An additional step performs semi-automatic segmentation of these anatomies, excluding the necrotic lesions, in CT. A final step performs pairwise registration of the corresponding anatomies, allowing for the localization and visualization of the necrosis in CT. To investigate the feasibility of integrating the proposed framework in the surgical workflow, we conducted experiments on MRIs and CTs containing early-stage ONFH. Our results indicate that the proposed framework is able to segment the anatomical structures of interest and accurately register the femurs and pelvis of the corresponding volumes, allowing for the visualization and localization of the ONFH in CT and generated X-rays, which could enable intra-operative visualization of the necrotic lesions for surgical procedures such as core decompression of the femur.

Evaluation of a bone reinforcement technique using finite element analysis / Avaliação de uma técnica de reforço ósseo com o método de elementos finitos

ABSTRACT Objectives: To compare the results of a simulated fall on the greater trochanter in the proximal portion of a synthetic femur before and after femoral reinforcement with tricalcium phosphate bone cement (TP) and polymethyl methacrylate (PMMA), using finite element analysis (FEA). Methods: Using two synthetic proximal femurs, a FEA simulating a fall on the greater trochanter was performed, using the Bi-directional Evolutionary Structural Optimization (BESO) program. For this analysis, the femurs were filled with TP and PMMA after perforations were created in the trochanteric region and neck. The results were compared with the strength values obtained from testing the control specimen, a synthetic bone without reinforcement. Results: FEA showed a value of 600 N prior to reinforcement. After cementing with PMMA, the load increased by 57.5% (945 N), and by 53% (920 N) after cementing with TP. Conclusion: Synthetic femurs gained resistance to fracture-causing forces in a simulated fall on the trochanter after bone reinforcement with PMMA and TP. Level of Evidence III; Experimental study.

RESUMO Objetivos: Avaliar, com o método de elementos finitos (EF), os resultados obtidos com a simulação de queda sobre o trocanter maior, usando a porção proximal de um fêmur sintético, com a finalidade de comparar os valores obtidos antes e após técnica de reforço femoral com cimento de fosfato tricálcico (FT) e polimetilmetacrilato (PMMA). Métodos: Utilizando dois fêmures proximais sintéticos, foi realizada a análise de elementos finitos, simulando queda sobre o trocanter maior com o programa Bi-directional Evolutionary Structural Optimization (BESO). Para essa análise, os fêmures foram preenchidos, após a realização de pertuitos na região trocantérica e no colo, com FT e PMMA e os resultados foram comparados com a força obtida na análise do corpo de prova controle, osso sintético sem preenchimento. Resultados: Comparando a análise de elementos finitos antes do reforço femoral, obteve-se o valor de 600 N. Depois da cimentação com PMMA, foi observado um aumento na carga máxima da ordem de 57,5% (945 N) e de 53% (920 N) com o FT. Conclusão: Os fêmures sintéticos ganharam resistência aos fatores causadores de fratura em queda simulada sobre o trocanter depois do reforço ósseo com PMMA e cimento de FT. Nível de Evidência III; Estudo experimental.