THA-Net: A Deep Learning Solution for Next-Generation Templating and Patient-specific Surgical Execution.

BACKGROUND: This study introduces THA-Net, a deep learning inpainting algorithm for simulating postoperative total hip arthroplasty (THA) radiographs from a single preoperative pelvis radiograph input, while being able to generate predictions either unconditionally (algorithm chooses implants) or conditionally (surgeon chooses implants). METHODS: The THA-Net is a deep learning algorithm which receives an input preoperative radiograph and subsequently replaces the target hip joint with THA implants to generate a synthetic yet realistic postoperative radiograph. We trained THA-Net on 356,305 pairs of radiographs from 14,357 patients from a single institution's total joint registry and evaluated the validity (quality of surgical execution) and realism (ability to differentiate real and synthetic radiographs) of its outputs against both human-based and software-based criteria. RESULTS: The surgical validity of synthetic postoperative radiographs was significantly higher than their real counterparts (mean difference: 0.8 to 1.1 points on 10-point Likert scale, P < .001), but they were not able to be differentiated in terms of realism in blinded expert review. Synthetic images showed excellent validity and realism when analyzed with already validated deep learning models. CONCLUSION: We developed a THA next-generation templating tool that can generate synthetic radiographs graded higher on ultimate surgical execution than real radiographs from training data. Further refinement of this tool may potentiate patient-specific surgical planning and enable technologies such as robotics, navigation, and augmented reality (an online demo of THA-Net is available at: https://demo.osail.ai/tha_net).

THA-AID: Deep Learning Tool for Total Hip Arthroplasty Automatic Implant Detection With Uncertainty and Outlier Quantification.

BACKGROUND: Revision total hip arthroplasty (THA) requires preoperatively identifying in situ implants, a time-consuming and sometimes unachievable task. Although deep learning (DL) tools have been attempted to automate this process, existing approaches are limited by classifying few femoral and zero acetabular components, only classify on anterior-posterior (AP) radiographs, and do not report prediction uncertainty or flag outlier data. METHODS: This study introduces Total Hip Arhtroplasty Automated Implant Detector (THA-AID), a DL tool trained on 241,419 radiographs that identifies common designs of 20 femoral and 8 acetabular components from AP, lateral, or oblique views and reports prediction uncertainty using conformal prediction and outlier detection using a custom framework. We evaluated THA-AID using internal, external, and out-of-domain test sets and compared its performance with human experts. RESULTS: THA-AID achieved internal test set accuracies of 98.9% for both femoral and acetabular components with no significant differences based on radiographic view. The femoral classifier also achieved 97.0% accuracy on the external test set. Adding conformal prediction increased true label prediction by 0.1% for acetabular and 0.7 to 0.9% for femoral components. More than 99% of out-of-domain and >89% of in-domain outlier data were correctly identified by THA-AID. CONCLUSIONS: The THA-AID is an automated tool for implant identification from radiographs with exceptional performance on internal and external test sets and no decrement in performance based on radiographic view. Importantly, this is the first study in orthopedics to our knowledge including uncertainty quantification and outlier detection of a DL model.

Anonymizing Radiographs Using an Object Detection Deep Learning Algorithm.

Radiographic markers contain protected health information that must be removed before public release. This work presents a deep learning algorithm that localizes radiographic markers and selectively removes them to enable de-identified data sharing. The authors annotated 2000 hip and pelvic radiographs to train an object detection computer vision model. Data were split into training, validation, and test sets at the patient level. Extracted markers were then characterized using an image processing algorithm, and potentially useful markers (eg, "L" and "R") without identifying information were retained. The model achieved an area under the precision-recall curve of 0.96 on the internal test set. The de-identification accuracy was 100% (400 of 400), with a de-identification false-positive rate of 1% (eight of 632) and a retention accuracy of 93% (359 of 386) for laterality markers. The algorithm was further validated on an external dataset of chest radiographs, achieving a de-identification accuracy of 96% (221 of 231). After fine-tuning the model on 20 images from the external dataset to investigate the potential for improvement, a 99.6% (230 of 231, P = .04) de-identification accuracy and decreased false-positive rate of 5% (26 of 512) were achieved. These results demonstrate the effectiveness of a two-pass approach in image de-identification. Keywords: Conventional Radiography, Skeletal-Axial, Thorax, Experimental Investigations, Supervised Learning, Transfer Learning, Convolutional Neural Network (CNN) Supplemental material is available for this article. © RSNA, 2023 See also the commentary by Chang and Li in this issue.

A Stepwise Approach to Analyzing Musculoskeletal Imaging Data With Artificial Intelligence.

The digitization of medical records and expanding electronic health records has created an era of "Big Data" with an abundance of available information ranging from clinical notes to imaging studies. In the field of rheumatology, medical imaging is used to guide both diagnosis and treatment of a wide variety of rheumatic conditions. Although there is an abundance of data to analyze, traditional methods of image analysis are human resource intensive. Fortunately, the growth of artificial intelligence (AI) may be a solution to handle large datasets. In particular, computer vision is a field within AI that analyzes images and extracts information. Computer vision has impressive capabilities and can be applied to rheumatologic conditions, necessitating a need to understand how computer vision works. In this article, we provide an overview of AI in rheumatology and conclude with a five step process to plan and conduct research in the field of computer vision. The five steps include (1) project definition, (2) data handling, (3) model development, (4) performance evaluation, and (5) deployment into clinical care.

Few-shot biomedical image segmentation using diffusion models: Beyond image generation.

BACKGROUND: Medical image analysis pipelines often involve segmentation, which requires a large amount of annotated training data, which is time-consuming and costly. To address this issue, we proposed leveraging generative models to achieve few-shot image segmentation. METHODS: We trained a denoising diffusion probabilistic model (DDPM) on 480,407 pelvis radiographs to generate 256 âœ• 256 px synthetic images. The DDPM was conditioned on demographic and radiologic characteristics and was rigorously validated by domain experts and objective image quality metrics (Frechet inception distance [FID] and inception score [IS]). For the next step, three landmarks (greater trochanter [GT], lesser trochanter [LT], and obturator foramen [OF]) were annotated on 45 real-patient radiographs; 25 for training and 20 for testing. To extract features, each image was passed through the pre-trained DDPM at three timesteps and for each pass, features from specific blocks were extracted. The features were concatenated with the real image to form an image with 4225 channels. The feature-set was broken into random patches, which were fed to a U-Net. Dice Similarity Coefficient (DSC) was used to compare the performance with a vanilla U-Net trained on radiographs. RESULTS: Expert accuracy was 57.5 % in determining real versus generated images, while the model reached an FID = 7.2 and IS = 210. The segmentation UNet trained on the 20 feature-sets achieved a DSC of 0.90, 0.84, and 0.61 for OF, GT, and LT segmentation, respectively, which was at least 0.30 points higher than the naively trained model. CONCLUSION: We demonstrated the applicability of DDPMs as feature extractors, facilitating medical image segmentation with few annotated samples.

Demystifying Statistics and Machine Learning in Analysis of Structured Tabular Data.

Electronic health records have facilitated the extraction and analysis of a vast amount of data with many variables for clinical care and research. Conventional regression-based statistical methods may not capture all the complexities in high-dimensional data analysis. Therefore, researchers are increasingly using machine learning (ML)-based methods to better handle these more challenging datasets for the discovery of hidden patterns in patients' data and for classification and predictive purposes. This article describes commonly used ML methods in structured data analysis with examples in orthopedic surgery. We present practical considerations in starting an ML project and appraising published studies in this field.

An Overview of Machine Learning in Orthopedic Surgery: An Educational Paper.

The growth of artificial intelligence combined with the collection and storage of large amounts of data in the electronic medical record collection has created an opportunity for orthopedic research and translation into the clinical environment. Machine learning (ML) is a type of artificial intelligence tool well suited for processing the large amount of available data. Specific areas of ML frequently used by orthopedic surgeons performing total joint arthroplasty include tabular data analysis (spreadsheets), medical imaging processing, and natural language processing (extracting concepts from text). Previous studies have discussed models able to identify fractures in radiographs, identify implant type in radiographs, and determine the stage of osteoarthritis based on walking analysis. Despite the growing popularity of ML, there are limitations including its reliance on "good" data, potential for overfitting, long life cycle for creation, and ability to only perform one narrow task. This educational article will further discuss a general overview of ML, discussing these challenges and including examples of successfully published models.

Application of Natural Language Processing in Total Joint Arthroplasty: Opportunities and Challenges.

Total joint arthroplasty is becoming one of the most common surgeries within the United States, creating an abundance of analyzable data to improve patient experience and outcomes. Unfortunately, a large majority of this data is concealed in electronic health records only accessible by manual extraction, which takes extensive time and resources. Natural language processing (NLP), a field within artificial intelligence, may offer a viable alternative to manual extraction. Using NLP, a researcher can analyze written and spoken data and extract data in an organized manner suitable for future research and clinical use. This article will first discuss common subtasks involved in an NLP pipeline, including data preparation, modeling, analysis, and external validation, followed by examples of NLP projects. Challenges and limitations of NLP will be discussed, closing with future directions of NLP projects, including large language models.

Educational Overview of the Concept and Application of Computer Vision in Arthroplasty.

Image data has grown exponentially as systems have increased their ability to collect and store it. Unfortunately, there are limits to human resources both in time and knowledge to fully interpret and manage that data. Computer Vision (CV) has grown in popularity as a discipline for better understanding visual data. Computer Vision has become a powerful tool for imaging analytics in orthopedic surgery, allowing computers to evaluate large volumes of image data with greater nuance than previously possible. Nevertheless, even with the growing number of uses in medicine, literature on the fundamentals of CV and its implementation is mainly oriented toward computer scientists rather than clinicians, rendering CV unapproachable for most orthopedic surgeons as a tool for clinical practice and research. The purpose of this article is to summarize and review the fundamental concepts of CV application for the orthopedic surgeon and musculoskeletal researcher.

Defining Clinically Meaningful Subgroups for Risk Stratification in Patients Undergoing Revision Total Hip Arthroplasty: A Combined Unsupervised and Supervised Machine Learning Approach.

BACKGROUND: Studies developing predictive models from large datasets to risk-stratify patients under going revision total hip arthroplasties (rTHAs) are limited. We used machine learning (ML) to stratify patients undergoing rTHA into risk-based subgroups. METHODS: We retrospectively identified 7,425 patients who underwent rTHA from a national database. An unsupervised random forest algorithm was used to partition patients into high-risk and low-risk strata based on similarities in rates of mortality, reoperation, and 25 other postoperative complications. A risk calculator was produced using a supervised ML algorithm to identify high-risk patients based on preoperative parameters. RESULTS: There were 3,135 and 4,290 patients identified in the high-risk and low-risk subgroups, respectively. Each group significantly differed by rate of 30-day mortalities, unplanned reoperations/readmissions, routine discharges, and hospital lengths of stay (P < .05). An Extreme Gradient Boosting algorithm identified preoperative platelets < 200, hematocrit > 35 or < 20, increasing age, albumin < 3, international normalized ratio > 2, body mass index > 35, American Society of Anesthesia class ≥ 3, blood urea nitrogen > 50 or < 30, creatinine > 1.5, diagnosis of hypertension or coagulopathy, and revision for periprosthetic fracture and infection as predictors of high risk. CONCLUSION: Clinically meaningful risk strata in patients undergoing rTHA were identified using an ML clustering approach. Preoperative labs, demographics, and surgical indications have the greatest impact on differentiating high versus low risk. LEVEL OF EVIDENCE: III.

A Deep Learning Tool for Automated Landmark Annotation on Hip and Pelvis Radiographs.

BACKGROUND: Automatic methods for labeling and segmenting pelvis structures can improve the efficiency of clinical and research workflows and reduce the variability introduced with manual labeling. The purpose of this study was to develop a single deep learning model to annotate certain anatomical structures and landmarks on antero-posterior (AP) pelvis radiographs. METHODS: A total of 1,100 AP pelvis radiographs were manually annotated by 3 reviewers. These images included a mix of preoperative and postoperative images as well as a mix of AP pelvis and hip images. A convolutional neural network was trained to segment 22 different structures (7 points, 6 lines, and 9 shapes). Dice score, which measures overlap between model output and ground truth, was calculated for the shapes and lines structures. Euclidean distance error was calculated for point structures. RESULTS: Dice score averaged across all images in the test set was 0.88 and 0.80 for the shape and line structures, respectively. For the 7-point structures, average distance between real and automated annotations ranged from 1.9 mm to 5.6 mm, with all averages falling below 3.1 mm except for the structure labeling the center of the sacrococcygeal junction, where performance was low for both human and machine-produced labels. Blinded qualitative evaluation of human and machine produced segmentations did not reveal any drastic decrease in performance of the automatic method. CONCLUSION: We present a deep learning model for automated annotation of pelvis radiographs that flexibly handles a variety of views, contrasts, and operative statuses for 22 structures and landmarks.

Fracture Gap Closure and Reduction Are Affected by the Orientation of the Headless Compression Screw.

BACKGROUND: We evaluated the impact of a variable-pitch headless screw's angle of insertion relative to the fracture plane on fracture gap closure and reduction. METHODS: Variable-pitch, fully threaded headless screws were inserted into polyurethane blocks of "normal" bone model density using a custom jig. Separate trials were completed with a 28-mm screw placed perpendicular and oblique/longitudinal to varying fracture planes (0°, 15°, 30°, 45°, and 60°). Fluoroscopic images were taken after each turn during screw insertion and analyzed. Initial screw push-off, residual fracture gap at optimal fracture gap reduction, and malreduction were determined in each trial. Statistical analysis was performed via a 1-way analysis of variance followed by Student t tests. RESULTS: Malreduction was found to be significantly different between the perpendicular (1.88 mm ± 1.38) and the oblique/longitudinal (0.58 mm ± 0.23) screws. The malreduction increased for the perpendicular screw as the fracture angle increased (60° > 45°=30° > 15° > 0°). Residual fracture gap at optimal fracture gap reduction was also found to be significantly different between the perpendicular (0.97 ± 0.42) and oblique/longitudinal (1.43 ± 1.14) screws. The residual fracture gap increased for the oblique/longitudinal screw as the fracture angle increased, although the oblique/longitudinal screw with a 60° fracture angle was the only configuration significantly larger than all the other configurations. Screw push-off was not found to be significantly different between the oblique/longitudinal screw and perpendicular screw trials. CONCLUSIONS: The perpendicular screw had a larger malreduction that increased with fracture angle, whereas the oblique/longitudinal screw had a larger residual fracture gap that increased with fracture angle.

Creating High Fidelity Synthetic Pelvis Radiographs Using Generative Adversarial Networks: Unlocking the Potential of Deep Learning Models Without Patient Privacy Concerns.

BACKGROUND: In this work, we applied and validated an artificial intelligence technique known as generative adversarial networks (GANs) to create large volumes of high-fidelity synthetic anteroposterior (AP) pelvis radiographs that can enable deep learning (DL)-based image analyses, while ensuring patient privacy. METHODS: AP pelvis radiographs with native hips were gathered from an institutional registry between 1998 and 2018. The data was used to train a model to create 512 × 512 pixel synthetic AP pelvis images. The network was trained on 25 million images produced through augmentation. A set of 100 random images (50/50 real/synthetic) was evaluated by 3 orthopaedic surgeons and 2 radiologists to discern real versus synthetic images. Two models (joint localization and segmentation) were trained using synthetic images and tested on real images. RESULTS: The final model was trained on 37,640 real radiographs (16,782 patients). In a computer assessment of image fidelity, the final model achieved an "excellent" rating. In a blinded review of paired images (1 real, 1 synthetic), orthopaedic surgeon reviewers were unable to correctly identify which image was synthetic (accuracy = 55%, Kappa = 0.11), highlighting synthetic image fidelity. The synthetic and real images showed equivalent performance when they were assessed by established DL models. CONCLUSION: This work shows the ability to use a DL technique to generate a large volume of high-fidelity synthetic pelvis images not discernible from real imaging by computers or experts. These images can be used for cross-institutional sharing and model pretraining, further advancing the performance of DL models without risk to patient data safety. LEVEL OF EVIDENCE: Level III.

The Effect of Derotational Kirschner Wires on Fracture Gap Reduction With Variable-Pitch Headless Screws.

PURPOSE: We evaluated the impact of angled derotational Kirschner wires (K-wires) on fracture gap reduction with variable-pitch headless screws. METHODS: Fully threaded variable-pitch headless screws (20 and 28 mm) were inserted into "normal" bone models of polyurethane blocks. In separate trials, derotational K-wires were inserted at predetermined angles of 0°, 15°, 30°, and 40° and compared with each other, with no K-wire as a control. Fluoroscopic images taken after each screw turn were analyzed. The optimal fracture gap closure, initial screw push-off, and screw back-out gap creation were determined and compared at various derotational K-wire angles. RESULTS: Initial screw push-off due to screw insertion and screw back-out gap creation were not significantly affected by the angle of the derotational K-wire. With a 20-mm screw, only a 40° derotational K-wire led to significantly less gap closure compared with control and with 0°, 15°, and 30° derotational K-wires. It led to an approximately 60% decrease in gap closure compared with no K-wire. With the 28-mm screw, compared with no K-wire, 15° and 30° derotational K-wires led to statistically significant decreases in gap closure (approximately 25%), whereas a 40° derotational K-wire led to an approximately 60% decrease. With the 28-mm screw, the 40° derotational K-wire also led to a statistically significant smaller gap closure when compared with 0°, 15°, and 30° derotational K-wires. CONCLUSIONS: A derotational K-wire placed in parallel to the planned trajectory of a headless compression screw does not affect fracture gap closure. With greater angulation of the derotational K-wire, the fracture gap is still closed, but less tightly. CLINICAL RELEVANCE: Derotational K-wires can help prevent fracture fragment rotation during headless compression screw insertion. At small deviations from parallel (≤30°), fracture gap closure achieved by the screw is minimally affected. At greater angles (ie, 40°), fracture gap closure may be substantially reduced, preventing fracture compression.