First deployment of artificial intelligence recommendations in orthopedic surgery.

Scant research has delved into the non-clinical facets of artificial intelligence (AI), concentrating on leveraging data to enhance the efficiency of healthcare systems and operating rooms. Notably, there is a gap in the literature regarding the implementation and outcomes of AI solutions. The absence of published results demonstrating the practical application and effectiveness of AI in domains beyond clinical settings, particularly in the field of surgery, served as the impetus for our undertaking in this area. Within the realm of non-clinical strategies aimed at enhancing operating room efficiency, we characterize OR efficiency as the capacity to successfully perform four uncomplicated arthroplasty surgeries within an 8-h timeframe. This Community Case Study addresses this gap by presenting the results of incorporating AI recommendations at our clinical institute on 228 patient arthroplasty surgeries. The implementation of a prescriptive analytics system (PAS), utilizing supervised machine learning techniques, led to a significant improvement in the overall efficiency of the operating room, increasing it from 39 to 93%. This noteworthy achievement highlights the impact of AI in optimizing surgery workflows.

Leveraging machine learning and prescriptive analytics to improve operating room throughput.

Successful days are defined as days when four cases were completed before 3:45pm, and overtime hours are defined as time spent after 3:45pm. Based on these definitions and the 460 unsuccessful days isolated from the dataset, 465 hours, 22 minutes, and 30 seconds total overtime hours were calculated. To reduce the increasing wait lists for hip and knee surgeries, we aim to verify whether it is possible to add a 5th surgery, to the typical 4 arthroplasty surgery per day schedule, without adding extra overtime hours and cost at our clinical institution. To predict 5th cases, 301 successful days were isolated and used to fit linear regression models for each individual day. After using the models' predictions, it was determined that increasing performance to a 77% success rate can lead to approximately 35 extra cases per year, while performing optimally at a 100% success rate can translate to 56 extra cases per year at no extra cost. Overall, this shows the extent of resources wasted by overtime costs, and the potential for their use in reducing long wait times. Future work can explore optimal staffing procedures to account for these extra cases.

Use of multidisciplinary positive deviance seminars to improve efficiency in a high-volume arthroplasty practice: a pilot study.

BACKGROUND: Positive deviance (PD) seminars, which have shown excellent results in improving the quality of surgical practices, use individual performance feedback to identify team members who outperform their peers; the strategies from those with exemplary performance are used to improve team members' practices. Our study aimed to use the PD approach with arthroplasty surgeons and nurses to identify multidisciplinary strategies and recommendations to improve operating room (OR) efficiency. METHODS: We recruited 5 surgeons who performed high-volume primary arthroplasty and had participated in 4-joint rooms since 2012, and 29 nurses who had participated in 4-joint rooms and in at least 16 cases in our data set. Three 1-hour PD sessions were held in February and March 2021: 1 with surgeons, 1 with nurses, and 1 with both surgeons and nurses to select recommendations for implementation. The sessions were led by a member of the nonorthopedic surgical faculty who was familiar with the subjects discussed and with PD seminars. To determine the success of the recommendations, we compared OR efficiency before and after implementation. We defined success as performance of 4 joint procedures within 8 hours. RESULTS: Eleven recommendations were recorded from the session with nurses and 7 from the session with surgeons, of which 11 were selected for implementation. During the month after implementation, there were great improvements across all time intervals of surgical procedures, with the greatest improvements seen in mean anesthesia preparation time in the room (4.51 min [26.3%]), mean procedure duration (9.75 min [14.0%]) and mean anesthesia finish time (5.78 min [44.0%]) (all p < 0.001). The total time saved per day was 49.84 minutes; this led to a success rate of 69.0%, a relative increase of 73.8% from our 2012-2020 success rate of 39.7% (p < 0.001). CONCLUSION: The recommendations and increased motivation owing to the individualized feedback reduced time spent per case, allowing more days to finish on time. Positive deviance seminars offer an inexpensive, efficient and collegial means for process improvement in the OR.

Artificial intelligence-driven prescriptive model to optimize team efficiency in a high-volume primary arthroplasty practice.

PURPOSE: We aimed to improve OR efficiency using machine learning (ML) to find relevant metrics influencing surgery time success and team performance on efficiency to create a model which incorporated team, patient, and surgery-related factors. METHODS: From 2012 to 2020, five surgeons, 44 nurses, and 152 anesthesiologists participated in 1199 four joint days (4796 cases): 1461 THA, 1496 TKA, 652 HR, 242 UKA, and 945 others. Patients were 2461f:2335 m; age, 64.1; BMI, 29.93; and ASA, 2.45. Surgical Success was defined as completing four joints within an eight hour shift using one OR. Time data was recorded prospectively using Surgical Information Management Systems. Hospital records provided team, patient demographics, adverse events, and anesthetic. Data mining identified patterns and relationships in higher dimensions. Predictive analytics used ML ranking algorithm to identify important metrics and created decision tree models for benchmarks and success probability. RESULTS: Five variables predicted success: anaesthesia preparation time, surgical preparation time, time of procedure, anesthesia finish time, and type of joint replacement. The model determined success rate with accuracy of 72% and AUC = 0.72. Probability of success based on mean performance was 77-89% (mean-median) if APT 14-15 minutes, PT 68-70 minutes, AFT four to five minutes, and turnover 25-27 minutes. With the above benchmarks maintained, success rate was 59% if surgeon exceeded 71.5-minutes PT or 89% if 64-minutes procedure time or 66% when anesthesiologist spent 17-19.5 minutes on APT. CONCLUSION: AI-ML predicted OR success without increasing resources. Benchmarks track OR performance, demonstrate effects of strategic changes, guide decisions, and provide teamwork improvement opportunities.

Factors influencing delays and overtime during surgery: a descriptive analytics for high volume arthroplasty procedures.

The aim of this article is to analyze factors influencing delays and overtime during surgery. We utilized descriptive analytics and divided the factors into three levels. In level one, we analyzed each surgical metrics individually and how it may influence the Surgical Success Rate (SSR) of each operating day. In level two, we compared up to three metrics at once, and in level three, we analyzed four metrics to identify more complex patterns in data including correlations. Within each level, factors were categorized as patient, surgical team, and time specific. Retrospective data on 788 high volume arthroplasty procedures was compiled and analyzed from the 4-joint arthroplasty operating room at our institution. Results demonstrated that surgical team performance had the highest impact on SSR whereas patient metrics had the least influence on SSR. Additionally, beginning the surgical day on time has a prominent effect on the SSR. Finally, the experience of the surgeon had almost no impact on the SSR. In conclusion, we gathered a list of insights that can help influence the re-allocation of resources in daily clinical practice to offset inefficiencies in arthroplasty surgeries.

pix2xray: converting RGB images into X-rays using generative adversarial networks.

PURPOSE: We propose a novel methodology for generating synthetic X-rays from 2D RGB images. This method creates accurate simulations for use in non-diagnostic visualization problems where the only input comes from a generic camera. Traditional methods are restricted to using simulation algorithms on 3D computer models. To solve this problem, we propose a method of synthetic X-ray generation using conditional generative adversarial networks (CGANs). METHODS: We create a custom synthetic X-ray dataset generator to generate image triplets for X-ray images, pose images, and RGB images of natural hand poses sampled from the NYU hand pose dataset. This dataset is used to train two general-purpose CGAN networks, pix2pix and CycleGAN, as well as our novel architecture called pix2xray which expands upon the pix2pix architecture to include the hand pose into the network. RESULTS: Our results demonstrate that our pix2xray architecture outperforms both pix2pix and CycleGAN in producing higher-quality X-ray images. We measure higher similarity metrics in our approach, with pix2pix coming in second, and CycleGAN producing the worst results. Our network performs better in the difficult cases which involve high occlusion due to occluded poses or large rotations. CONCLUSION: Overall our work establishes a baseline that synthetic X-rays can be simulated using 2D RGB input. We establish the need for additional data such as the hand pose to produce clearer results and show that future research must focus on more specialized architectures to improve overall image clarity and structure.

Camera-augmented mobile C-arm (CamC): A feasibility study of augmented reality imaging in the operating room.

BACKGROUND: In orthopaedic trauma surgery, image-guided procedures are mostly based on fluoroscopy. The reduction of radiation exposure is an important goal. The purpose of this work was to investigate the impact of a camera-augmented mobile C-arm (CamC) on radiation exposure and the surgical workflow during a first clinical trial. METHODS: Applying a workflow-oriented approach, 10 general workflow steps were defined to compare the CamC to traditional C-arms. The surgeries included were arbitrarily identified and assigned to the study. The evaluation criteria were radiation exposure and operation time for each workflow step and the entire surgery. The evaluation protocol was designed and conducted in a single-centre study. RESULTS: The radiation exposure was remarkably reduced by 18 X-ray shots 46% using the CamC while keeping similar surgery times. CONCLUSIONS: The intuitiveness of the system, its easy integration into the surgical workflow, and its great potential to reduce radiation have been demonstrated.

Application of an RGBD augmented C-arm for minimally invasive scoliosis surgery assistance.

Minimally invasive surgeries (MISs) are gaining popularity as alternatives to conventional open surgeries. In thoracoscopic scoliosis MIS, fluoroscopy is used to guide pedicle screw placement and to visualise the effect of the intervention on the spine curvature. However, cosmetic external appearance is the most important concern for patients, while correction of the spine and achieving coronal and sagittal trunk balance are the top priorities for surgeons. The authors present the feasibility study of the first intra-operative assistive system for scoliosis surgery composed of a single RGBD camera affixed on a C-arm which allows visualising in real time the surgery effects on the patient trunk surface in the transverse plane. They perform three feasibility experiments from simulated data based on scoliotic patients to live acquisition from non-scoliotic mannequin and person, all showing that the proposed system accuracy is comparable with scoliotic surface reconstruction state of art.

Inverse visualization concept for RGB-D augmented C-arms.

X-ray is still the essential imaging for many minimally-invasive interventions. Overlaying X-ray images with an optical view of the surgery scene has been demonstrated to be an efficient way to reduce radiation exposure and surgery time. However, clinicians are recommended to place the X-ray source under the patient table while the optical view of the real scene must be captured from the top in order to see the patient, surgical tools, and the surgical site. With the help of a RGB-D (red-green-blue-depth) camera, which can measure depth in addition to color, the 3D model of the real scene is registered to the X-ray image. However, fusing two opposing viewpoints and visualizing them in the context of medical applications has never been attempted. In this paper, we propose first experiences of a novel inverse visualization technique for RGB-D augmented C-arms. A user study consisting of 16 participants demonstrated that our method shows a meaningful visualization with potential in providing clinicians multi-modal fused data in real-time during surgery.

An augmented reality C-arm for intraoperative assessment of the mechanical axis: a preclinical study.

BACKGROUND: Determination of lower limb alignment is a prerequisite for successful orthopedic surgical treatment. Traditional methods include the electrocautery cord, alignment rod, or axis board which rely solely on C-arm fluoroscopy navigation and are radiation intensive. STUDY OBJECTIVES: To assess a new augmented reality technology in determining lower limb alignment. METHODS: A camera-augmented mobile C-arm (CamC) technology was used to create a panorama image consisting of hip, knee, and ankle X-rays. Twenty-five human cadaver legs were used for validation with random varus or valgus deformations. Five clinicians performed experiments that consisted in achieving acceptable mechanical axis deviation. The applicability of the CamC technology was assessed with direct comparison to ground-truth CT. A t test, Pearson's correlation, and ANOVA were used to determine statistical significance. RESULTS: The value of Pearson's correlation coefficient R was 0.979 which demonstrates a strong positive correlation between the CamC and ground-truth CT data. The analysis of variance produced a p value equal to 0.911 signifying that clinician expertise differences were not significant with regard to the type of system used to assess mechanical axis deviation. CONCLUSION: All described measurements demonstrated valid measurement of lower limb alignment. With minimal effort, clinicians required only 3 X-ray image acquisitions using the augmented reality technology to achieve reliable mechanical axis deviation.

Device- and system-independent personal touchless user interface for operating rooms : One personal UI to control all displays in an operating room.

INTRODUCTION: In the modern day operating room, the surgeon performs surgeries with the support of different medical systems that showcase patient information, physiological data, and medical images. It is generally accepted that numerous interactions must be performed by the surgical team to control the corresponding medical system to retrieve the desired information. Joysticks and physical keys are still present in the operating room due to the disadvantages of mouses, and surgeons often communicate instructions to the surgical team when requiring information from a specific medical system. In this paper, a novel user interface is developed that allows the surgeon to personally perform touchless interaction with the various medical systems, switch effortlessly among them, all of this without modifying the systems' software and hardware. METHODS: To achieve this, a wearable RGB-D sensor is mounted on the surgeon's head for inside-out tracking of his/her finger with any of the medical systems' displays. Android devices with a special application are connected to the computers on which the medical systems are running, simulating a normal USB mouse and keyboard. When the surgeon performs interaction using pointing gestures, the desired cursor position in the targeted medical system display, and gestures, are transformed into general events and then sent to the corresponding Android device. Finally, the application running on the Android devices generates the corresponding mouse or keyboard events according to the targeted medical system. RESULTS AND CONCLUSION: To simulate an operating room setting, our unique user interface was tested by seven medical participants who performed several interactions with the visualization of CT, MRI, and fluoroscopy images at varying distances from them. Results from the system usability scale and NASA-TLX workload index indicated a strong acceptance of our proposed user interface.

Precise 3D/2D calibration between a RGB-D sensor and a C-arm fluoroscope.

PURPOSE: Calibration and registration are the first steps for augmented reality and mixed reality applications. In the medical field, the calibration between an RGB-D camera and a C-arm fluoroscope is a new topic which introduces challenges. METHOD: A convenient and efficient calibration phantom is designed by combining the traditional calibration object of X-ray images with a checkerboard plane. After the localization of the 2D marker points in the X-ray images and the corresponding 3D points from the RGB-D images, we calculate the projection matrix from the RGB-D sensor coordinates to the X-ray, instead of estimating the extrinsic and intrinsic parameters simultaneously. VALIDATION: In order to evaluate the effect of every step of our calibration process, we performed five experiments by combining different steps leading to the calibration. We also compared our calibration method to Tsai's method to evaluate the advancement of our solution. At last, we simulated the process of estimating the rotation movement of the RGB-D camera using MATLAB and demonstrate that calculating the projection matrix can reduce the angle error of the rotation. RESULTS: A RMS reprojection error of 0.5 mm is achieved using our calibration method which is promising for surgical applications. Our calibration method is more accurate when compared to Tsai's method. Lastly, the simulation result shows that using a projection matrix has a lower error than using intrinsic and extrinsic parameters in the rotation estimation. CONCLUSIONS: We designed and evaluated a 3D/2D calibration method for the combination of a RGB-D camera and a C-arm fluoroscope.

Automatic 3D reconstruction of electrophysiology catheters from two-view monoplane C-arm image sequences.

PURPOSE: Catheter guidance is a vital task for the success of electrophysiology interventions. It is usually provided through fluoroscopic images that are taken intra-operatively. The cardiologists, who are typically equipped with C-arm systems, scan the patient from multiple views rotating the fluoroscope around one of its axes. The resulting sequences allow the cardiologists to build a mental model of the 3D position of the catheters and interest points from the multiple views. METHOD: We describe and compare different 3D catheter reconstruction strategies and ultimately propose a novel and robust method for the automatic reconstruction of 3D catheters in non-synchronized fluoroscopic sequences. This approach does not purely rely on triangulation but incorporates prior knowledge about the catheters. In conjunction with an automatic detection method, we demonstrate the performance of our method compared to ground truth annotations. RESULTS: In our experiments that include 20 biplane datasets, we achieve an average reprojection error of 0.43 mm and an average reconstruction error of 0.67 mm compared to gold standard annotation. CONCLUSIONS: In clinical practice, catheters suffer from complex motion due to the combined effect of heartbeat and respiratory motion. As a result, any 3D reconstruction algorithm via triangulation is imprecise. We have proposed a new method that is fully automatic and highly accurate to reconstruct catheters in three dimensions.

Personalized augmented reality for anatomy education.

Anatomy education is a challenging but vital element in forming future medical professionals. In this work, a personalized and interactive augmented reality system is developed to facilitate education. This system behaves as a "magic mirror" which allows personalized in-situ visualization of anatomy on the user's body. Real-time volume visualization of a CT dataset creates the illusion that the user can look inside their body. The system comprises a RGB-D sensor as a real-time tracking device to detect the user moving in front of a display. In addition, the magic mirror system shows text information, medical images, and 3D models of organs that the user can interact with. Through the participation of 7 clinicians and 72 students, two user studies were designed to respectively assess the precision and acceptability of the magic mirror system for education. The results of the first study demonstrated that the average precision of the augmented reality overlay on the user body was 0.96 cm, while the results of the second study indicate 86.1% approval for the educational value of the magic mirror, and 91.7% approval for the augmented reality capability of displaying organs in three dimensions. The usefulness of this unique type of personalized augmented reality technology has been demonstrated in this paper.

Intra-operative disruptions, surgeon's mental workload, and technical performance in a full-scale simulated procedure.

BACKGROUND AND AIM: Surgical flow disruptions occur frequently and jeopardize perioperative care and surgical performance. So far, insights into subjective and cognitive implications of intra-operative disruptions for surgeons and inherent consequences for performance are inconsistent. This study aimed to investigate the effect of surgical flow disruption on surgeon's intra-operative workload and technical performance. METHODS: In a full-scale OR simulation, 19 surgeons were randomly allocated to either of the two disruption scenarios (telephone call vs. patient discomfort). Using a mixed virtual reality simulator with a computerized, high-fidelity mannequin, all surgeons were trained in performing a vertebroplasty procedure and subsequently performed such a procedure under experimental conditions. Standardized measures on subjective workload and technical performance (trocar positioning deviation from expert-defined standard, number, and duration of X-ray acquisitions) were collected. RESULTS: Intra-operative workload during simulated disruption scenarios was significantly higher compared to training sessions (p < .01). Surgeons in the telephone call scenario experienced significantly more distraction compared to their colleagues in the patient discomfort scenario (p < .05). However, workload tended to be increased in surgeons who coped with distractions due to patient discomfort. Technical performance was not significantly different between both disruption scenarios. We found a significant association between surgeons' intra-operative workload and technical performance such that surgeons with increased mental workload tended to perform worse (ß = .55, p = .04). CONCLUSIONS: Surgical flow disruptions affect surgeons' intra-operative workload. Increased mental workload was associated with inferior technical performance. Our simulation-based findings emphasize the need to establish smooth surgical flow which is characterized by a low level of process deviations and disruptions.

Intra-operative augmented reality in distal locking.

PURPOSE: To design an augmented reality solution that assists surgeons during the distal locking of intramedullary nailing procedures. METHOD: Traditionally, the procedure is performed under X-ray guidance and requires a significant amount of time and radiation exposure. To absolve these complications, we propose video guidance that allows surgeons to achieve both the down-the-beam position of the intramedullary nail and its subsequent locking. For the down-the-beam position, the IM nail pose in X-ray is calculated using a 2D/3D registration scheme and later related to the patient leg pose which is calculated using video-tracked AR markers. For the distal locking, surgeons use an augmented radiolucent drill in which its tip position is detected and tracked in real-time under video guidance. VALIDATION: To evaluate the feasibility of our solution, we performed a preclinical study on dry bone phantom with the participation of four clinicians. RESULTS: Participants achieved 100 % success rate in the down-the beam positioning and 93 % success rate in distal locking using only two X-ray images in 100 s. CONCLUSIONS: We confirmed that intra-operative navigation using augmented reality provides an alternative way to perform distal locking in a safe and timely manner.

Multi-modal intra-operative navigation during distal locking of intramedullary nails.

The interlocking of intramedullary nails is a technically demanding procedure which involves a considerable amount of X-ray acquisitions; one study lists as many as 48 to successfully complete the procedure and fix screws into 4-6 mm distal holes of the nail. We propose to design an augmented radiolucent drill to assist surgeons in completing the distal locking procedure without any additional X-ray acquisitions. Using an augmented reality fluoroscope that coregisters optical and X-ray images, we exploit solely the optical images to detect the augmented radiolucent drill and estimate its tip position in real-time. Consequently, the surgeons will be able to maintain the down the beam positioning required to drill the screws into the nail holes successfully. To evaluate the accuracy of the proposed augmented drill, we perform a preclinical study involving six surgeons and ask them to perform distal locking on dry bone phantoms. Surgeons completed distal locking 98.3% of the time using only a single X-ray image with an average navigation time of 1.4 ± 0.9 min per hole.

Machine learning-based augmented reality for improved surgical scene understanding.

In orthopedic and trauma surgery, AR technology can support surgeons in the challenging task of understanding the spatial relationships between the anatomy, the implants and their tools. In this context, we propose a novel augmented visualization of the surgical scene that mixes intelligently the different sources of information provided by a mobile C-arm combined with a Kinect RGB-Depth sensor. Therefore, we introduce a learning-based paradigm that aims at (1) identifying the relevant objects or anatomy in both Kinect and X-ray data, and (2) creating an object-specific pixel-wise alpha map that permits relevance-based fusion of the video and the X-ray images within one single view. In 12 simulated surgeries, we show very promising results aiming at providing for surgeons a better surgical scene understanding as well as an improved depth perception.

Fully automatic catheter localization in C-arm images using l1-sparse coding.

We propose a method to perform automatic detection and tracking of electrophysiology (EP) catheters in C-arm fluoroscopy sequences. Our approach does not require any initialization, is completely automatic, and can concurrently track an arbitrary number of overlapping catheters. After a pre-processing step, we employ sparse coding to first detect candidate catheter tips, and subsequently detect and track the catheters. The proposed technique is validated on 2835 C-arm images, which include 39,690 manually selected ground-truth catheter electrodes. Results demonstrated sub-millimeter detection accuracy and real-time tracking performances.