OMERACT validation of a deep learning algorithm for automated absolute quantification of knee joint effusion versus manual semi-quantitative assessment.

OBJECTIVE: To begin evaluating deep learning (DL)-automated quantification of knee joint effusion-synovitis via the OMERACT filter. METHODS: A DL algorithm previously trained on Osteoarthritis Initiative (OAI) knee MRI automatically quantified effusion volume in MRI of 53 OAI subjects, which were also scored semi-quantitatively via KIMRISS and MOAKS by 2-6 readers. RESULTS: DL-measured knee effusion correlated significantly with experts' assessments (Kendall's tau 0.34-0.43) CONCLUSION: The close correlation of automated DL knee joint effusion quantification to KIMRISS manual semi-quantitative scoring demonstrated its criterion validity. Further assessments of discrimination and truth vs. clinical outcomes are still needed to fully satisfy OMERACT filter requirements.

Self-supervised-RCNN for medical image segmentation with limited data annotation.

Many successful methods developed for medical image analysis based on machine learning use supervised learning approaches, which often require large datasets annotated by experts to achieve high accuracy. However, medical data annotation is time-consuming and expensive, especially for segmentation tasks. To overcome the problem of learning with limited labeled medical image data, an alternative deep learning training strategy based on self-supervised pretraining on unlabeled imaging data is proposed in this work. For the pretraining, different distortions are arbitrarily applied to random areas of unlabeled images. Next, a Mask-RCNN architecture is trained to localize the distortion location and recover the original image pixels. This pretrained model is assumed to gain knowledge of the relevant texture in the images from the self-supervised pretraining on unlabeled imaging data. This provides a good basis for fine-tuning the model to segment the structure of interest using a limited amount of labeled training data. The effectiveness of the proposed method in different pretraining and fine-tuning scenarios was evaluated based on the Osteoarthritis Initiative dataset with the aim of segmenting effusions in MRI datasets of the knee. Applying the proposed self-supervised pretraining method improved the Dice score by up to 18% compared to training the models using only the limited annotated data. The proposed self-supervised learning approach can be applied to many other medical image analysis tasks including anomaly detection, segmentation, and classification.

Comparison of Dynamic Knee Contact Mechanics with T2 Imaging in Different Ages of Healthy Participants.

Aging is a known risk factor for Osteoarthritis (OA), however, relations between cartilage composition and aging remain largely unknown in understanding human OA. T2 imaging provides an approach to assess cartilage composition. Whether these T2 relaxation times in the joint contact region change with time during gait remain unexplored. The study purpose was to demonstrate a methodology for linking dynamic joint contact mechanics to cartilage composition as measured by T2 relaxometry. T2 relaxation times for unloaded cartilage were measured in a 3T General Electric magnetic resonance (MR) scanner in this preliminary study. High-speed biplanar video-radiography (HSBV) was captured for five 20-30-year-old and five 50-60-year-old participants with asymptomatic knees. By mapping the T2 cartilages to the dynamic contact regions, T2 values were averaged over the contact area at each measurement within the gait cycle. T2 values demonstrated a functional relationship across the gait cycle. There were no statistically significant differences between 20- and 30-year-old and 50-60-year-old participant T2 values at first force peak of the gait cycle in the medial femur (p = 1.00, U = 12) or in the medial tibia (p = 0.31, U = 7). In the medial and lateral femur in swing phase, the joint moved from a region of high T2 values at 75% of gait to a minimum at 85-95% of swing. The lateral femur and tibia demonstrated similar patterns to the medial compartments but were less pronounced. This research advances understanding of the linkage between cartilage contact and cartilage composition. The change from a high T2 value at ~ 75% of gait to a lower value near the initiation of terminal swing (90% gait) indicates that there are changes to T2 averages corresponding to changes in the contact region across the gait cycle. No differences were found between age groups for healthy participants. These preliminary findings provide interesting insights into the cartilage composition corresponding to dynamic cyclic motion and inform mechanisms of osteoarthritis.

Improved-Mask R-CNN: Towards an accurate generic MSK MRI instance segmentation platform (data from the Osteoarthritis Initiative).

INTRODUCTION: Objective assessment of osteoarthritis (OA) Magnetic Resonance Imaging (MRI) scans can address the limitations of the current OA assessment approaches. Detecting and extracting bone, cartilage, and joint fluid is a necessary component for the objective assessment of OA, which helps to quantify tissue characteristics such as volume and thickness. Many algorithms, based on Artificial Intelligence (AI), have been proposed over recent years for segmenting bone and soft tissues. Most of these segmentation methods suffer from the class imbalance problem, can't differentiate between the same anatomic structure, or do not support segmenting different rang of tissue sizes. Mask R-CNN is an instance segmentation framework, meaning it segments and distinct each object of interest like different anatomical structures (e.g. bone and cartilage) using a single model. In this study, the Mask R-CNN architecture was deployed to address the need for a segmentation method that is applicable to use for different tissue scales, pathologies, and MRI sequences associated with OA, without having a problem with imbalanced classes. In addition, we modified the Mask R-CNN to improve segmentation accuracy around instance edges. METHODS: A total of 500 adult knee MRI scans from the publicly available Osteoarthritis Initiative (OAI), and 97 hip MRI scans from adults with symptomatic hip OA, evaluated by two readers, were used for training and validating the network. Three specific modifications to Mask R-CNN yielded the improved-Mask R-CNN (iMaskRCNN): an additional ROIAligned block, an extra decoder block in the segmentation header, and connecting them using a skip connection. The results were evaluated using Hausdorff distance, dice score for bone and cartilage segmentation, and differences in detected volume, dice score, and coefficients of variation (CoV) for effusion segmentation. RESULTS: The iMaskRCNN led to improved bone and cartilage segmentation compared to Mask RCNN as indicated with the increase in dice score from 95% to 98% for the femur, 95-97% for the tibia, 71-80% for the femoral cartilage, and 81-82% for the tibial cartilage. For the effusion detection, the dice score improved with iMaskRCNN 72% versus Mask R-CNN 71%. The CoV values for effusion detection between Reader1 and Mask R-CNN (0.33), Reader1 and iMaskRCNN (0.34), Reader2 and Mask R-CNN (0.22), Reader2 and iMaskRCNN (0.29) are close to CoV between two readers (0.21), indicating a high agreement between the human readers and both Mask R-CNN and iMaskRCNN. CONCLUSION: Mask R-CNN and iMaskRCNN can reliably and simultaneously extract different scale articular tissues involved in OA, forming the foundation for automated assessment of OA. The iMaskRCNN results show that the modification improved the network performance around the edges.

Automatic Assessment Of Hip Effusion From MRI.

Joint effusion is a hallmark of osteoarthritis (OA) associated with stiffness, and may relate to pain, disability, and long-term outcomes. However, it is difficult to quantify accurately. We propose a new Deep Learning (DL) approach for automatic effusion assessment from Magnetic Resonance Imaging (MRI) using volumetric quantification measures (VQM). We developed a new multiplane ensemble convolutional neural network (CNN) approach for 1) localizing bony anatomy and 2) detecting effusion regions. CNNs were trained on femoral head and effusion regions manually segmented from 3856 images (63 patients). Upon validation on a non-overlapping set of 2040 images (34 patients) DL showed high agreement with ground-truth in terms of Dice score (0.85), sensitivity (0.86) and precision (0.83). Agreement of VQM per-patient was high for DL vs experts in term of Intraclass correlation coefficient (ICC)= 0.88[0.80,0.93]. We expect this technique to reduce inter-observer variability in effusion assessment, reducing expert time and potentially improving the quality of OA care.Clinical Relevance- Our technique for automatic assessment of hip MRI can be used for volumetric measurement of effusion. We expect this to reduce variability in OA biomarker assessment and provide more reliable indicators for disease progression.

MRI Knee Domain Translation for Unsupervised Segmentation By CycleGAN (data from Osteoarthritis initiative (OAI)).

Accurate quantification of bone and cartilage features is the key to efficient management of knee osteoarthritis (OA). Bone and cartilage tissues can be accurately segmented from magnetic resonance imaging (MRI) data using supervised Deep Learning (DL) methods. DL training is commonly conducted using large datasets with expert-labeled annotations. DL models perform better if distributions of testing data (target domains) are close to those of training data (source domains). However, in practice, data distributions of images from different MRI scanners and sequences are different and DL models need to re-trained on each dataset separately. We propose a domain adaptation (DA) framework using the CycleGAN model for MRI translation that would aid in unsupervised MRI data segmentation. We have validated our pipeline on five scans from the Osteoarthritis Initiative (OAI) dataset. Using this pipeline, we translated TSE Fat Suppressed MRI sequences to pseudo-DESS images. An improved MaskRCNN (IMaskRCNN) instance segmentation network trained on DESS was used to segment cartilage and femoral head regions in TSE Fat Suppressed sequences. Segmentations of the I-MaskRCNN correlated well with approximated manual segmentation obtained from nearest DESS slices (DICE = 0.76) without the need for retraining. We anticipate this technique will aid in automatic unsupervised assessment of knee MRI using commonly acquired MRI sequences and save experts' time that would otherwise be required for manual segmentation.Clinical relevance- This technique paves the way to automatically convert one MRI sequence to its equivalent as if acquired by a different protocol or different magnet, facilitating robust, hardware-independent automated analysis. For example, routine clinically acquired knee MRI could be converted to high-resolution high-contrast images suitable for automated detection of cartilage defects.

fMRI-Informed EEG for brain mapping of imagined lower limb movement: Feasibility of a brain computer interface.

BACKGROUND: EEG and fMRI have contributed greatly to our understanding of brain activity and its link to behaviors by helping to identify both when and where the activity occurs. This is particularly important in the development of brain-computer interfaces (BCIs), where feed forward systems gather data from imagined brain activity and then send that information to an effector. The purpose of this study was to develop and evaluate a computational approach that enables an accurate mapping of spatial brain activity (fMRI) in relation to the temporal receptors (EEG electrodes) associated with imagined lower limb movement. NEW METHOD: EEG and fMRI data from 16 healthy, male participants while imagining lower limb movement were used for this purpose. A combined analysis of fMRI data and EEG electrode locations was developed to identify EEG electrodes with a high likelihood of capturing imagined lower limb movement originating from various clusters of brain activity. This novel feature selection tool was used to develop an artificial neural network model to classify right and left lower limb movement. RESULTS: Results showed that left versus right lower limb imagined movement could be classified with 66.5% accuracy using this approach. Comparison with existing methods: Adopting a purely data-driven approach for feature selection to use in the right/left classification task resulted in the same accuracy (66.6%) but with reduced interpretability. CONCLUSIONS: The developed fMRI-informed EEG approach could pave the way towards improved brain computer interfaces for lower limb movement while also being applicable to other systems where fMRI could be helpful to inform EEG acquisition and processing.

Volumetric quantitative measurement of hip effusions by manual versus automated artificial intelligence techniques: An OMERACT preliminary validation study.

OBJECTIVE: Preliminary assessment, via OMERACT filter, of manual and automated MRI hip effusion Volumetric Quantitative Measurement (VQM). METHODS: For 358 hips (93 osteoarthritis subjects, bilateral, 2 time points), 2 radiologists performed manual VQM using custom Matlab software. A Mask R-CNN artificial-intelligence (AI) tool was trained to automatically compute joint fluid volumes. RESULTS: Manual VQM had excellent inter-observer reliability (ICC 0.96). AI predicted hip fluid volumes with ICC 0.86 (status), 0.58 (change) vs. 2 human readers. CONCLUSION: Hip joint fluid volumes are reliably assessed by VQM. It is feasible to automate this approach using AI, with promising initial reliability.