Integrating Radiomics and Neural Networks for Knee Osteoarthritis Incidence Prediction.

OBJECTIVE: Accurately predicting knee osteoarthritis (KOA) is essential for early detection and personalized treatment. We aimed to develop and test a magnetic resonance imaging (MRI)-based joint space (JS) radiomic model (RM) to predict radiographic KOA incidence through neural networks by integrating meniscus and femorotibial cartilage radiomic features. METHODS: In the Osteoarthritis Initiative cohort, participants with knees without radiographic KOA at baseline but at high risk for radiographic KOA were included. Patients' knees developed radiographic KOA, whereas control knees did not over four years. We randomly split the participants into development and test cohorts (8:2) and extracted features from baseline three-dimensional double-echo steady-state sequence MRI. Model performance was evaluated using an area under the receiver operating characteristic curve (AUC), sensitivity, and specificity in both cohorts. Nine resident surgeons performed the reader experiment without/with the JS-RM aid. RESULTS: Our study included 549 knees in the development cohort (275 knees of patients with KOA vs 274 knees of controls) and 137 knees in the test cohort (68 knees of patients with KOA vs 69 knees of controls). In the test cohort, JS-RM had a favorable accuracy for predicting the radiographic KOA incidence with an AUC of 0.931 (95% confidence interval [CI] 0.876-0.963), a sensitivity of 84.4% (95% CI 83.9%-84.9%), and a specificity of 85.6% (95% CI 85.2%-86.0%). The mean specificity and sensitivity of resident surgeons through MRI reading in predicting radiographic KOA incidence were increased from 0.474 (95% CI 0.333-0.614) and 0.586 (95% CI 0.429-0.743) without the assistance of JS-RM to 0.874 (95% CI 0.847-0.901) and 0.812 (95% CI 0.742-0.881) with JS-RM assistance, respectively (P < 0.001). CONCLUSION: JS-RM integrating the features of the meniscus and cartilage showed improved predictive values in radiographic KOA incidence.

A Multi-Task Deep Learning Method for Detection of Meniscal Tears in MRI Data from the Osteoarthritis Initiative Database.

We present a novel and computationally efficient method for the detection of meniscal tears in Magnetic Resonance Imaging (MRI) data. Our method is based on a Convolutional Neural Network (CNN) that operates on complete 3D MRI scans. Our approach detects the presence of meniscal tears in three anatomical sub-regions (anterior horn, body, posterior horn) for both the Medial Meniscus (MM) and the Lateral Meniscus (LM) individually. For optimal performance of our method, we investigate how to preprocess the MRI data and how to train the CNN such that only relevant information within a Region of Interest (RoI) of the data volume is taken into account for meniscal tear detection. We propose meniscal tear detection combined with a bounding box regressor in a multi-task deep learning framework to let the CNN implicitly consider the corresponding RoIs of the menisci. We evaluate the accuracy of our CNN-based meniscal tear detection approach on 2,399 Double Echo Steady-State (DESS) MRI scans from the Osteoarthritis Initiative database. In addition, to show that our method is capable of generalizing to other MRI sequences, we also adapt our model to Intermediate-Weighted Turbo Spin-Echo (IW TSE) MRI scans. To judge the quality of our approaches, Receiver Operating Characteristic (ROC) curves and Area Under the Curve (AUC) values are evaluated for both MRI sequences. For the detection of tears in DESS MRI, our method reaches AUC values of 0.94, 0.93, 0.93 (anterior horn, body, posterior horn) in MM and 0.96, 0.94, 0.91 in LM. For the detection of tears in IW TSE MRI data, our method yields AUC values of 0.84, 0.88, 0.86 in MM and 0.95, 0.91, 0.90 in LM. In conclusion, the presented method achieves high accuracy for detecting meniscal tears in both DESS and IW TSE MRI data. Furthermore, our method can be easily trained and applied to other MRI sequences.

Towards novel osteoarthritis biomarkers: Multi-criteria evaluation of 46,996 segmented knee MRI data from the Osteoarthritis Initiative.

Convolutional neural networks (CNNs) are the state-of-the-art for automated assessment of knee osteoarthritis (KOA) from medical image data. However, these methods lack interpretability, mainly focus on image texture, and cannot completely grasp the analyzed anatomies' shapes. In this study we assess the informative value of quantitative features derived from segmentations in order to assess their potential as an alternative or extension to CNN-based approaches regarding multiple aspects of KOA. Six anatomical structures around the knee (femoral and tibial bones, femoral and tibial cartilages, and both menisci) are segmented in 46,996 MRI scans. Based on these segmentations, quantitative features are computed, i.e., measurements such as cartilage volume, meniscal extrusion and tibial coverage, as well as geometric features based on a statistical shape encoding of the anatomies. The feature quality is assessed by investigating their association to the Kellgren-Lawrence grade (KLG), joint space narrowing (JSN), incident KOA, and total knee replacement (TKR). Using gold standard labels from the Osteoarthritis Initiative database the balanced accuracy (BA), the area under the Receiver Operating Characteristic curve (AUC), and weighted kappa statistics are evaluated. Features based on shape encodings of femur, tibia, and menisci plus the performed measurements showed most potential as KOA biomarkers. Differentiation between non-arthritic and severely arthritic knees yielded BAs of up to 99%, 84% were achieved for diagnosis of early KOA. Weighted kappa values of 0.73, 0.72, and 0.78 were achieved for classification of the grade of medial JSN, lateral JSN, and KLG, respectively. The AUC was 0.61 and 0.76 for prediction of incident KOA and TKR within one year, respectively. Quantitative features from automated segmentations provide novel biomarkers for KLG and JSN classification and show potential for incident KOA and TKR prediction. The validity of these features should be further evaluated, especially as extensions of CNN-based approaches. To foster such developments we make all segmentations publicly available together with this publication.

VerSe: A Vertebrae labelling and segmentation benchmark for multi-detector CT images.

Vertebral labelling and segmentation are two fundamental tasks in an automated spine processing pipeline. Reliable and accurate processing of spine images is expected to benefit clinical decision support systems for diagnosis, surgery planning, and population-based analysis of spine and bone health. However, designing automated algorithms for spine processing is challenging predominantly due to considerable variations in anatomy and acquisition protocols and due to a severe shortage of publicly available data. Addressing these limitations, the Large Scale Vertebrae Segmentation Challenge (VerSe) was organised in conjunction with the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) in 2019 and 2020, with a call for algorithms tackling the labelling and segmentation of vertebrae. Two datasets containing a total of 374 multi-detector CT scans from 355 patients were prepared and 4505 vertebrae have individually been annotated at voxel level by a human-machine hybrid algorithm (https://osf.io/nqjyw/, https://osf.io/t98fz/). A total of 25 algorithms were benchmarked on these datasets. In this work, we present the results of this evaluation and further investigate the performance variation at the vertebra level, scan level, and different fields of view. We also evaluate the generalisability of the approaches to an implicit domain shift in data by evaluating the top-performing algorithms of one challenge iteration on data from the other iteration. The principal takeaway from VerSe: the performance of an algorithm in labelling and segmenting a spine scan hinges on its ability to correctly identify vertebrae in cases of rare anatomical variations. The VerSe content and code can be accessed at: https://github.com/anjany/verse.

Fully automated Assessment of Knee Alignment from Full-Leg X-Rays employing a "YOLOv4 And Resnet Landmark regression Algorithm" (YARLA): Data from the Osteoarthritis Initiative.

BACKGROUND AND OBJECTIVE: We present a fully automated method for the quantification of knee alignment from full-leg radiographs. METHODS: A state-of-the-art object detector, YOLOv4, was trained to locate regions of interests in full-leg radiographs for the hip joint, knee, and ankle. Residual neural networks were trained to regress landmark coordinates for each region of interest. Based on the detected landmarks the knee alignment, i.e., the hip-knee-ankle (HKA) angle was computed. The accuracy of landmark detection was evaluated by a comparison to manually placed ones for 180 radiographs. The accuracy of HKA angle computations was assessed on the basis of 2,943 radiographs by a comparison to results of two independent image reading studies (Cooke; Duryea) both publicly accessible via the Osteoarthritis Initiative. The agreement was evaluated using Spearman's Rho, weighted kappa, and regarding the correspondence of the class assignment. RESULTS: The average deviation of landmarks manually placed by experts and automatically detected ones by our proposed "YOLOv4 And Resnet Landmark regression Algorithm" (YARLA) was less than 2.0 ± 1.5 mm for all structures. The average mismatch between HKA angle determinations of Cooke and Duryea was 0.09 ± 0.63°; YARLA resulted in a mismatch of 0.09 ± 0.73° compared to Cooke and of 0.18 ± 0.67° compared to Duryea. Cooke and Duryea agreed almost perfectly with respect to a weighted kappa value of 0.86, and showed an excellent reliability as measured by a Spearman's Rho value of 0.98. Similar values were achieved by YARLA, i.e., a weighted kappa value of 0.83 and 0.87 and a Spearman's Rho value of 0.98 and 0.98 compared to Cooke and Duryea, respectively. Cooke and Duryea agreed in 91% of all class assignments and YARLA did so in 90% against Cooke and 92% against Duryea. CONCLUSIONS: YARLA yields HKA angles similar to those of human experts and provides a basis for an automated assessment of knee alignment in full-leg radiographs.

Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: Data from the Osteoarthritis Initiative.

We present a method for the automated segmentation of knee bones and cartilage from magnetic resonance imaging (MRI) that combines a priori knowledge of anatomical shape with Convolutional Neural Networks (CNNs). The proposed approach incorporates 3D Statistical Shape Models (SSMs) as well as 2D and 3D CNNs to achieve a robust and accurate segmentation of even highly pathological knee structures. The shape models and neural networks employed are trained using data from the Osteoarthritis Initiative (OAI) and the MICCAI grand challenge "Segmentation of Knee Images 2010" (SKI10), respectively. We evaluate our method on 40 validation and 50 submission datasets from the SKI10 challenge. For the first time, an accuracy equivalent to the inter-observer variability of human readers is achieved in this challenge. Moreover, the quality of the proposed method is thoroughly assessed using various measures for data from the OAI, i.e. 507 manual segmentations of bone and cartilage, and 88 additional manual segmentations of cartilage. Our method yields sub-voxel accuracy for both OAI datasets. We make the 507 manual segmentations as well as our experimental setup publicly available to further aid research in the field of medical image segmentation. In conclusion, combining localized classification via CNNs with statistical anatomical knowledge via SSMs results in a state-of-the-art segmentation method for knee bones and cartilage from MRI data.

Validation of three-dimensional models of the distal femur created from surgical navigation point cloud data for intraoperative and postoperative analysis of total knee arthroplasty.

PURPOSE: Despite the success of total knee arthroplasty, there continues to be a significant proportion of patients who are dissatisfied. One explanation may be a shape mismatch between pre- and postoperative distal femurs. The purpose of this study was to investigate methods suitable for matching a statistical shape model (SSM) to intraoperatively acquired point cloud data from a surgical navigation system and to validate these against the preoperative magnetic resonance imaging (MRI) data from the same patients. METHODS: A total of 10 patients who underwent navigated total knee arthroplasty also had an MRI scan <2 months preoperatively. The standard surgical protocol was followed which included partial digitization of the distal femur. Two different methods were employed to fit the SSM to the digitized point cloud data, based on (1) iterative closest points and (2) Gaussian mixture models. The available MRI data were manually segmented and the reconstructed three-dimensional surfaces used as ground truth against which the SSM fit was compared. RESULTS: For both approaches, the difference between the SSM-generated femur and the surface generated from MRI segmentation averaged less than 1.7 mm, with maximum errors occurring in less clinically important areas. CONCLUSION: The results demonstrated good correspondence with the distal femoral morphology even in cases of sparse datasets. Application of this technique will allow for measurement of mismatch between pre- and postoperative femurs retrospectively on any case done using the surgical navigation system and could be integrated into the surgical navigation unit to provide real-time feedback.

Shape-aware surface reconstruction from sparse 3D point-clouds.

The reconstruction of an object's shape or surface from a set of 3D points plays an important role in medical image analysis, e.g. in anatomy reconstruction from tomographic measurements or in the process of aligning intra-operative navigation and preoperative planning data. In such scenarios, one usually has to deal with sparse data, which significantly aggravates the problem of reconstruction. However, medical applications often provide contextual information about the 3D point data that allow to incorporate prior knowledge about the shape that is to be reconstructed. To this end, we propose the use of a statistical shape model (SSM) as a prior for surface reconstruction. The SSM is represented by a point distribution model (PDM), which is associated with a surface mesh. Using the shape distribution that is modelled by the PDM, we formulate the problem of surface reconstruction from a probabilistic perspective based on a Gaussian Mixture Model (GMM). In order to do so, the given points are interpreted as samples of the GMM. By using mixture components with anisotropic covariances that are "oriented" according to the surface normals at the PDM points, a surface-based fitting is accomplished. Estimating the parameters of the GMM in a maximum a posteriori manner yields the reconstruction of the surface from the given data points. We compare our method to the extensively used Iterative Closest Points method on several different anatomical datasets/SSMs (brain, femur, tibia, hip, liver) and demonstrate superior accuracy and robustness on sparse data.