Deep learning applications in osteoarthritis imaging.

Deep learning (DL) is one of the most exciting new areas in medical imaging. This article will provide a review of current applications of DL in osteoarthritis (OA) imaging, including methods used for cartilage lesion detection, OA diagnosis, cartilage segmentation, and OA risk assessment. DL techniques have been shown to have similar diagnostic performance as human readers for detecting and grading cartilage lesions within the knee on MRI. A variety of DL methods have been developed for detecting and grading the severity of knee OA and various features of knee OA on X-rays using standardized classification systems with diagnostic performance similar to human readers. Multiple DL approaches have been described for fully automated segmentation of cartilage and other knee tissues and have achieved higher segmentation accuracy than currently used methods with substantial reductions in segmentation times. Various DL models analyzing baseline X-rays and MRI have been developed for OA risk assessment. These models have shown high diagnostic performance for predicting a wide variety of OA outcomes, including the incidence and progression of radiographic knee OA, the presence and progression of knee pain, and future total knee replacement. The preliminary results of DL applications in OA imaging have been encouraging. However, many DL techniques require further technical refinement to maximize diagnostic performance. Furthermore, the generalizability of DL approaches needs to be further investigated in prospective studies using large image datasets acquired at different institutions with different imaging hardware before they can be implemented in clinical practice and research studies.

Prediction of Total Knee Replacement and Diagnosis of Osteoarthritis by Using Deep Learning on Knee Radiographs: Data from the Osteoarthritis Initiative.

Background The methods for assessing knee osteoarthritis (OA) do not provide enough comprehensive information to make robust and accurate outcome predictions. Purpose To develop a deep learning (DL) prediction model for risk of OA progression by using knee radiographs in patients who underwent total knee replacement (TKR) and matched control patients who did not undergo TKR. Materials and Methods In this retrospective analysis that used data from the OA Initiative, a DL model on knee radiographs was developed to predict both the likelihood of a patient undergoing TKR within 9 years and Kellgren-Lawrence (KL) grade. Study participants included a case-control matched subcohort between 45 and 79 years. Patients were matched to control patients according to age, sex, ethnicity, and body mass index. The proposed model used a transfer learning approach based on the ResNet34 architecture with sevenfold nested cross-validation. Receiver operating characteristic curve analysis and conditional logistic regression assessed model performance for predicting probability and risk of TKR compared with clinical observations and two binary outcome prediction models on the basis of radiographic readings: KL grade and OA Research Society International (OARSI) grade. Results Evaluated were 728 participants including 324 patients (mean age, 64 years ± 8 [standard deviation]; 222 women) and 324 control patients (mean age, 64 years ± 8; 222 women). The prediction model based on DL achieved an area under the receiver operating characteristic curve (AUC) of 0.87 (95% confidence interval [CI]: 0.85, 0.90), outperforming a baseline prediction model by using KL grade with an AUC of 0.74 (95% CI: 0.71, 0.77; P < .001). The risk for TKR increased with probability that a person will undergo TKR from the DL model (odds ratio [OR], 7.7; 95% CI: 2.3, 25; P < .001), KL grade (OR, 1.92; 95% CI: 1.17, 3.13; P = .009), and OARSI grade (OR, 1.20; 95% CI: 0.41, 3.50; P = .73). Conclusion The proposed deep learning model better predicted risk of total knee replacement in osteoarthritis than did binary outcome models by using standard grading systems. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Richardson in this issue.

Specific absorption rate implications of within-scan patient head motion for ultra-high field MRI.

PURPOSE: This study investigates the implications of all degrees of freedom of within-scan patient head motion on patient safety. METHODS: Electromagnetic simulations were performed by displacing and/or rotating a virtual body model inside an 8-channel transmit array to simulate 6 degrees of freedom of motion. Rotations of up to 20° and displacements of up to 20 mm including off-axis axial/coronal translations were investigated, yielding 104 head positions. Quadrature excitation, RF shimming, and multi-spoke parallel-transmit excitation pulses were designed for axial slice-selection at 7T, for seven slices across the head. Variation of whole-head specific absorption rate (SAR) and 10-g averaged local SAR of the designed pulses, as well as the change in the maximum eigenvalue (worst-case pulse) were investigated by comparing off-center positions to the central position. RESULTS: In their respective worst-cases, patient motion increased the eigenvalue-based local SAR by 42%, whole-head SAR by 60%, and the 10-g averaged local SAR by 210%. Local SAR was observed to be more sensitive to displacements along right-left and anterior-posterior directions than displacement in the superior-inferior direction and rotation. CONCLUSION: This is the first study to investigate the effect of all 6 degrees of freedom of motion on safety of practical pulses. Although the results agree with the literature for overlapping cases, the results demonstrate higher increases (up to 3.1-fold) in local SAR for off-axis displacement in the axial plane, which had received less attention in the literature. This increase in local SAR could potentially affect the local SAR compliance of subjects, unless realistic within-scan patient motion is taken into account during pulse design.

Synthesized tissue-equivalent dielectric phantoms using salt and polyvinylpyrrolidone solutions.

PURPOSE: To explore the use of polyvinylpyrrolidone (PVP) for simulated materials with tissue-equivalent dielectric properties. METHODS: PVP and salt were used to control, respectively, relative permittivity and electrical conductivity in a collection of 63 samples with a range of solute concentrations. Their dielectric properties were measured with a commercial probe and fitted to a 3D polynomial in order to establish an empirical recipe. The material's thermal properties and MR spectra were measured. RESULTS: The empirical polynomial recipe (available at https://www.amri.ninds.nih.gov/cgi-bin/phantomrecipe) provides the PVP and salt concentrations required for dielectric materials with permittivity and electrical conductivity values between approximately 45 and 78, and 0.1 to 2 siemens per meter, respectively, from 50 MHz to 4.5 GHz. The second- (solute concentrations) and seventh- (frequency) order polynomial recipe provided less than 2.5% relative error between the measured and target properties. PVP side peaks in the spectra were minor and unaffected by temperature changes. CONCLUSION: PVP-based phantoms are easy to prepare and nontoxic, and their semitransparency makes air bubbles easy to identify. The polymer can be used to create simulated material with a range of dielectric properties, negligible spectral side peaks, and long T2 relaxation time, which are favorable in many MR applications. Magn Reson Med 80:413-419, 2018. © 2017 International Society for Magnetic Resonance in Medicine.

Improved detection of fMRI activation in the cerebellum at 7T with dielectric pads extending the imaging region of a commercial head coil.

BACKGROUND: There is growing interest in detecting cerebro-cerebellar circuits, which requires adequate blood oxygenation level dependent contrast and signal-to-noise ratio (SNR) throughout the brain. Although 7T scanners offer increased SNR, coverage of commercial head coils is currently limited to the cerebrum. PURPOSE: To improve cerebellar functional MRI (fMRI) at 7T with high permittivity material (HPM) pads extending the sensitivity of a commercial coil. STUDY TYPE: Simulations were used to determine HPM pad configuration and assess radiofrequency (RF) safety. In vivo experiments were performed to evaluate RF field distributions and SNR and assess improvements of cerebellar fMRI. SUBJECTS: Eight healthy volunteers enrolled in a prospective motor fMRI study with and without HPM. FIELD STRENGTH/SEQUENCE: Gradient echo (GRE) echo planar imaging for fMRI, turbo FLASH for flip angle mapping, GRE sequence for SNR maps, and T1 -weighted MPRAGE were acquired with and without HPM pads at 7T. ASSESSMENT: Field maps, SNR maps, and anatomical images were evaluated for coverage. Simulation results were used to assess SAR levels of the experiment. Activation data from fMRI experiments were compared with and without HPM pads. STATISTICAL TESTS: fMRI data were analyzed using FEAT FSL for each subject followed by group level analysis using paired t-test of acquisitions with and without HPM. RESULTS: Simulations showed 52% improvement in transmit efficiency in cerebellum with HPM and SAR levels well below recommended limits. Experiments showed 27% improvement in SNR in cerebellum and improvement in coverage on T1 -weighted images. fMRI showed greater cerebellar activation in individual subjects with the HPM pad present (Z > = 4), especially in inferior slices of cerebellum, with 59% average increase in number of activated voxels in the cerebellum. Group-level analysis showed improved functional activation (Z > = 2.3) in cerebellar regions with HPM pads without loss of measured activation elsewhere. DATA CONCLUSION: HPM pads can improve cerebellar fMRI at 7T with a commonly-used head coil without compromising RF safety. LEVEL OF EVIDENCE: 2 Technical Efficacy: Stage 1 J. MAGN. RESON. IMAGING 2018;48:431-440.

Manipulating transmit and receive sensitivities of radiofrequency surface coils using shielded and unshielded high-permittivity materials.

OBJECTIVE: To use high-permittivity materials (HPM) positioned near radiofrequency (RF) surface coils to manipulate transmit/receive field patterns. MATERIALS AND METHODS: A large HPM pad was placed below the RF coil to extend the field of view (FOV). The resulting signal-to-noise ratio (SNR) was compared with that of other coil configurations covering the same FOV in simulations and experiments at 7 T. Transmit/receive efficiency was evaluated when HPM discs with or without a partial shield were positioned at a distance from the coil. Finally, we evaluated the increase in transmit homogeneity for a four-channel array with HPM discs interposed between adjacent coil elements. RESULTS: Various configurations of HPM increased SNR, transmit/receive efficiency, excitation/reception sensitivity overlap, and FOV when positioned near a surface coil. For a four-channel array driven in quadrature, shielded HPM discs enhanced the field below the discs as well as at the center of the sample as compared with other configurations with or without unshielded HPM discs. CONCLUSION: Strategically positioning HPM at a distance from a surface coil or array can increase the overlap between excitation/reception sensitivities, and extend the FOV of a single coil for reduction of the number of channels in an array while minimally affecting the SNR.

Parallel transmission RF pulse design with strict temperature constraints.

RF safety in parallel transmission (pTx) is generally ensured by imposing specific absorption rate (SAR) limits during pTx RF pulse design. There is increasing interest in using temperature to ensure safety in MRI. In this work, we present a local temperature correlation matrix formalism and apply it to impose strict constraints on maximum absolute temperature in pTx RF pulse design for head and hip regions. Electromagnetic field simulations were performed on the head and hip of virtual body models. Temperature correlation matrices were calculated for four different exposure durations ranging between 6 and 24 min using simulated fields and body-specific constants. Parallel transmission RF pulses were designed using either SAR or temperature constraints, and compared with each other and unconstrained RF pulse design in terms of excitation fidelity and safety. The use of temperature correlation matrices resulted in better excitation fidelity compared with the use of SAR in parallel transmission RF pulse design (for the 6 min exposure period, 8.8% versus 21.0% for the head and 28.0% versus 32.2% for the hip region). As RF exposure duration increases (from 6 min to 24 min), the benefit of using temperature correlation matrices on RF pulse design diminishes. However, the safety of the subject is always guaranteed (the maximum temperature was equal to 39°C). This trend was observed in both head and hip regions, where the perfusion rates are very different.

Subject- and resource-specific monitoring and proactive management of parallel radiofrequency transmission.

PURPOSE: Develop a practical comprehensive package for proactive management of parallel radiofrequency (RF) transmission. METHODS: With a constrained optimization framework and predictive models from a prescan based multichannel calibration, we presented a method supporting design and optimization of parallel RF excitation pulses that accurately obey the forward/reflected peak and average power limits of the RF power amplifiers in parallel transmit imaging experiments and Bloch simulations. Moreover, local SAR limits were incorporated into the parallel RF excitation pulses using electromagnetic field simulations. Virtual transmit coils concept for minimization of reflected power (effecting subject-specific matching) was additionally demonstrated by leveraging experimentally calibrated power models. RESULTS: Incorporation of experimentally calibrated power prediction models resulted in accurate compliance with prescribed hardware and global specific absorption rate (SAR) limits. Incorporation of spatial average 10 g SAR models, facilitated by simplifying numerical approximations, provided assurance of patient safety. RF pulses designed with various constraints demonstrated excellent excitation fidelity-the normalized root-mean-square error of the simulated excitation profiles was 2.6% for the fully constrained pulses, comparable to that of the unconstrained pulses. An RF shimming example showed a reduction of the reflected-to-forward power ratio to 1.7% from a conventional approach's 8.1%. CONCLUSION: Using the presented RF pulse design method, effective proactive management of the multifaceted power and SAR limits was demonstrated in experimental and simulation studies. Magn Reson Med 76:20-31, 2016. © 2015 Wiley Periodicals, Inc.

Radiofrequency energy deposition and radiofrequency power requirements in parallel transmission with increasing distance from the coil to the sample.

PURPOSE: We investigated global specific absorption rate (SAR) and radiofrequency (RF) power requirements in parallel transmission as the distance between the transmit coils and the sample was increased. METHODS: We calculated ultimate intrinsic SAR (UISAR), which depends on object geometry and electrical properties but not on coil design, and we used it as the reference to compare the performance of various transmit arrays. We investigated the case of fixing coil size and increasing the number of coils while moving the array away from the sample, as well as the case of fixing coil number and scaling coil dimensions. We also investigated RF power requirements as a function of lift-off, and tracked local SAR distributions associated with global SAR optima. RESULTS: In all cases, the target excitation profile was achieved and global SAR (as well as associated maximum local SAR) decreased with lift-off, approaching UISAR, which was constant for all lift-offs. We observed a lift-off value that optimizes the balance between global SAR and power losses in coil conductors. We showed that, using parallel transmission, global SAR can decrease at ultra high fields for finite arrays with a sufficient number of transmit elements. CONCLUSION: For parallel transmission, the distance between coils and object can be optimized to reduce SAR and minimize RF power requirements associated with homogeneous excitation.

Heat equation inversion framework for average SAR calculation from magnetic resonance thermal imaging.

Deposition of radiofrequency (RF) energy can be quantified via electric field or temperature change measurements. Magnetic resonance imaging has been used as a tool to measure three dimensional small temperature changes associated with RF radiation exposure. When duration of RF exposure is long, conversion from temperature change to specific absorption rate (SAR) is nontrivial due to prominent heat-diffusion and conduction effects. In this work, we demonstrated a method for calculation of SAR via an inversion of the heat equation including heat-diffusion and conduction effects. This method utilizes high-resolution three dimensional magnetic resonance temperature images and measured thermal properties of the phantom to achieve accurate calculation of SAR. Accuracy of the proposed method was analyzed with respect to operating frequency of a dipole antenna and parameters used in heat equation inversion. Bioelectromagnetics. 37:493-503, 2016. © 2016 Wiley Periodicals, Inc.

A method for safety testing of radiofrequency/microwave-emitting devices using MRI.

PURPOSE: Strict regulations are imposed on the amount of radiofrequency (RF) energy that devices can emit to prevent excessive deposition of RF energy into the body. In this study, we investigated the application of MR temperature mapping and 10-g average specific absorption rate (SAR) computation for safety evaluation of RF-emitting devices. METHODS: Quantification of the RF power deposition was shown for an MRI-compatible dipole antenna and a non-MRI-compatible mobile phone via phantom temperature change measurements. Validation of the MR temperature mapping method was demonstrated by comparison with physical temperature measurements and electromagnetic field simulations. MR temperature measurements alongside physical property measurements were used to reconstruct 10-g average SAR. RESULTS: The maximum temperature change for a dipole antenna and the maximum 10-g average SAR were 1.83°C and 12.4 W/kg, respectively, for simulations and 1.73°C and 11.9 W/kg, respectively, for experiments. The difference between MR and probe thermometry was <0.15°C. The maximum temperature change and the maximum 10-g average SAR for a cell phone radiating at maximum output for 15 min was 1.7°C and 0.54 W/kg, respectively. CONCLUSION: Information acquired using MR temperature mapping and thermal property measurements can assess RF/microwave safety with high resolution and fidelity.

Finite element analysis applied to 3-T MR imaging of proximal femur microarchitecture: lower bone strength in patients with fragility fractures compared with control subjects.

PURPOSE: To determine the feasibility of using finite element analysis applied to 3-T magnetic resonance (MR) images of proximal femur microarchitecture for detection of lower bone strength in subjects with fragility fractures compared with control subjects without fractures. MATERIALS AND METHODS: This prospective study was institutional review board approved and HIPAA compliant. Written informed consent was obtained. Postmenopausal women with (n = 22) and without (n = 22) fragility fractures were matched for age and body mass index. All subjects underwent standard dual-energy x-ray absorptiometry. Images of proximal femur microarchitecture were obtained by using a high-spatial-resolution three-dimensional fast low-angle shot sequence at 3 T. Finite element analysis was applied to compute elastic modulus as a measure of strength in the femoral head and neck, Ward triangle, greater trochanter, and intertrochanteric region. The Mann-Whitney test was used to compare bone mineral density T scores and elastic moduli between the groups. The relationship (R(2)) between elastic moduli and bone mineral density T scores was assessed. RESULTS: Patients with fractures showed lower elastic modulus than did control subjects in all proximal femur regions (femoral head, 8.51-8.73 GPa vs 9.32-9.67 GPa; P = .04; femoral neck, 3.11-3.72 GPa vs 4.39-4.82 GPa; P = .04; Ward triangle, 1.85-2.21 GPa vs 3.98-4.13 GPa; P = .04; intertrochanteric region, 1.62-2.18 GPa vs 3.86-4.47 GPa; P = .006-.007; greater trochanter, 0.65-1.21 GPa vs 1.96-2.62 GPa; P = .01-.02), but no differences in bone mineral density T scores. There were weak relationships between elastic moduli and bone mineral density T scores in patients with fractures (R(2) = 0.25-0.31, P = .02-.04), but not in control subjects. CONCLUSION Finite element analysis applied to high-spatial-resolution 3-T MR images of proximal femur microarchitecture can allow detection of lower elastic modulus, a marker of bone strength, in subjects with fragility fractures compared with control subjects. MR assessment of proximal femur strength may provide information about bone quality that is not provided by dual-energy x-ray absorptiometry.

Feasibility of three-dimensional MRI of proximal femur microarchitecture at 3 tesla using 26 receive elements without and with parallel imaging.

PURPOSE: High-resolution imaging of deeper anatomy such as the hip is challenging due to low signal-to-noise ratio (SNR), necessitating long scan times. Multi-element coils can increase SNR and reduce scan time through parallel imaging (PI). We assessed the feasibility of using a 26-element receive coil setup to perform 3 Tesla (T) MRI of proximal femur microarchitecture without and with PI. MATERIALS AND METHODS: This study had institutional review board approval. We scanned 13 subjects on a 3T scanner using 26 receive-elements and a three-dimensional fast low-angle shot (FLASH) sequence without and with PI (acceleration factors [AF] 2, 3, 4). We assessed SNR, depiction of individual trabeculae, PI performance (1/g-factor), and image quality with PI (1 = nonvisualization to 5 = excellent). RESULTS: SNR maps demonstrate higher SNR for the 26-element setup compared with a 12-element setup for hip MRI. Without PI, individual proximal femur trabeculae were well-depicted, including microarchitectural deterioration in osteoporotic subjects. With PI, 1/g values for the 26-element/12-element receive-setup were 0.71/0.45, 0.56/0.25, and 0.44/0.08 at AF2, AF3, and AF4, respectively. Image quality was: AF1, excellent (4.8 ± 0.4); AF2, good (4.2 ± 1.0); AF3, average (3.3 ± 1.0); AF4, nonvisualization (1.4 ± 0.9). CONCLUSION: A 26-element receive-setup permits 3T MRI of proximal femur microarchitecture with good image quality up to PI AF2.

MRI of the hip at 7T: feasibility of bone microarchitecture, high-resolution cartilage, and clinical imaging.

PURPOSE: To demonstrate the feasibility of performing bone microarchitecture, high-resolution cartilage, and clinical imaging of the hip at 7T. MATERIALS AND METHODS: This study had Institutional Review Board approval. Using an 8-channel coil constructed in-house, we imaged the hips of 15 subjects on a 7T magnetic resonance imaging (MRI) scanner. We applied: 1) a T1-weighted 3D fast low angle shot (3D FLASH) sequence (0.23 × 0.23 × 1-1.5 mm(3) ) for bone microarchitecture imaging; 2) T1-weighted 3D FLASH (water excitation) and volumetric interpolated breath-hold examination (VIBE) sequences (0.23 × 0.23 × 1.5 mm(3) ) with saturation or inversion recovery-based fat suppression for cartilage imaging; 3) 2D intermediate-weighted fast spin-echo (FSE) sequences without and with fat saturation (0.27 × 0.27 × 2 mm) for clinical imaging. RESULTS: Bone microarchitecture images allowed visualization of individual trabeculae within the proximal femur. Cartilage was well visualized and fat was well suppressed on FLASH and VIBE sequences. FSE sequences allowed visualization of cartilage, the labrum (including cartilage and labral pathology), joint capsule, and tendons. CONCLUSION: This is the first study to demonstrate the feasibility of performing a clinically comprehensive hip MRI protocol at 7T, including high-resolution imaging of bone microarchitecture and cartilage, as well as clinical imaging.

The impact of data augmentation and transfer learning on the performance of deep learning models for the segmentation of the hip on 3D magnetic resonance images.

Different pathologies of the hip are characterized by the abnormal shape of the bony structures of the joint, namely the femur and the acetabulum. Three-dimensional (3D) models of the hip can be used for diagnosis, biomechanical simulation, and planning of surgical treatments. These models can be generated by building 3D surfaces of the joint's structures segmented on magnetic resonance (MR) images. Deep learning can avoid time-consuming manual segmentations, but its performance depends on the amount and quality of the available training data. Data augmentation and transfer learning are two approaches used when there is only a limited number of datasets. In particular, data augmentation can be used to artificially increase the size and diversity of the training datasets, whereas transfer learning can be used to build the desired model on top of a model previously trained with similar data. This study investigates the effect of data augmentation and transfer learning on the performance of deep learning for the automatic segmentation of the femur and acetabulum on 3D MR images of patients diagnosed with femoroacetabular impingement. Transfer learning was applied starting from a model trained for the segmentation of the bony structures of the shoulder joint, which bears some resemblance to the hip joint. Our results suggest that data augmentation is more effective than transfer learning, yielding a Dice similarity coefficient compared to ground-truth manual segmentations of 0.84 and 0.89 for the acetabulum and femur, respectively, whereas the Dice coefficient was 0.78 and 0.88 for the model based on transfer learning. The Accuracy for the two anatomical regions was 0.95 and 0.97 when using data augmentation, and 0.87 and 0.96 when using transfer learning. Data augmentation can improve the performance of deep learning models by increasing the diversity of the training dataset and making the models more robust to noise and variations in image quality. The proposed segmentation model could be combined with radiomic analysis for the automatic evaluation of hip pathologies.

Artificial intelligence in knee osteoarthritis: A comprehensive review for 2022.

Objective: The aim of this literature review is to yield a comprehensive and exhaustive overview of the existing evidence and up-to-date applications of artificial intelligence for knee osteoarthritis. Methods: A literature review was performed by using PubMed, Google Scholar, and IEEE databases for articles published in peer-reviewed journals in 2022. The articles focusing on the use of artificial intelligence in diagnosis and prognosis of knee osteoarthritis and accelerating the image acquisition were selected. For each selected study, the code availability, considered number of patients and knees, imaging type, covariates, grading type of osteoarthritis, models, validation approaches, objectives, and results were reviewed. Results: 395 articles were screened, and 35 of them were reviewed. Eight articles were based on diagnosis, six on prognosis prediction, three on classification, three on accelerated image acquisition, and 15 on segmentation of knee osteoarthritis. 57% of the articles used MRI, 26% radiography, 6% MRI together with radiography, 6% ultrasonography, and 6% only clinical data. 23% of the articles made the computer codes available for their study, and 26% used clinical data. External validation and nested cross-validation were used in 17% and 14% of articles, respectively. Conclusions: The use of artificial intelligence provided a promising potential to enhance the detection and management of knee osteoarthritis. Translating the developed models into clinics is still in the early stages of development. The translation of artificial intelligence models is expected to be further examined in prospective studies to support clinicians in improving routine healthcare practice.

Radiology Reports Improve Visual Representations Learned from Radiographs.

Although human's ability to visually understand the structure of the World plays a crucial role in perceiving the World and making appropriate decisions, human perception does not solely rely on vision but amalgamates the information from acoustic, verbal, and visual stimuli. An active area of research has been revolving around designing an efficient framework that adapts to multiple modalities and ideally improves the performance of existing tasks. While numerous frameworks have proved effective on natural datasets like ImageNet, a limited number of studies have been carried out in the biomedical domain. In this work, we extend the available frameworks for natural data to biomedical data by leveraging the abundant, unstructured multi-modal data available as radiology images and reports. We attempt to answer the question, "For multi-modal learning, self-supervised learning and joint learning using both learning strategies, which one improves the visual representation for downstream chest radiographs classification tasks the most?". Our experiments indicated that in limited labeled data settings with 1% and 10% labeled data, the joint learning with multi-modal and self-supervised models outperforms self-supervised learning and is at par with multi-modal learning. Additionally, we found that multi-modal learning is generally more robust on out-of-distribution datasets. The code is publicly available online.

Prediction of total knee replacement using deep learning analysis of knee MRI.

Current methods for assessing knee osteoarthritis (OA) do not provide comprehensive information to make robust and accurate outcome predictions. Deep learning (DL) risk assessment models were developed to predict the progression of knee OA to total knee replacement (TKR) over a 108-month follow-up period using baseline knee MRI. Participants of our retrospective study consisted of 353 case-control pairs of subjects from the Osteoarthritis Initiative with and without TKR over a 108-month follow-up period matched according to age, sex, ethnicity, and body mass index. A traditional risk assessment model was created to predict TKR using baseline clinical risk factors. DL models were created to predict TKR using baseline knee radiographs and MRI. All DL models had significantly higher (p < 0.001) AUCs than the traditional model. The MRI and radiograph ensemble model and MRI ensemble model (where TKR risk predicted by several contrast-specific DL models were averaged to get the ensemble TKR risk prediction) had the highest AUCs of 0.90 (80% sensitivity and 85% specificity) and 0.89 (79% sensitivity and 86% specificity), respectively, which were significantly higher (p < 0.05) than the AUCs of the radiograph and multiple MRI models (where the DL models were trained to predict TKR risk using single contrast or 2 contrasts together as input). DL models using baseline MRI had a higher diagnostic performance for predicting TKR than a traditional model using baseline clinical risk factors and a DL model using baseline knee radiographs.

Towards Automatic Cartilage Quantification in Clinical Trials - Continuing from the 2019 IWOAI Knee Segmentation Challenge.

Objective: To evaluate whether the deep learning (DL) segmentation methods from the six teams that participated in the IWOAI 2019 Knee Cartilage Segmentation Challenge are appropriate for quantifying cartilage loss in longitudinal clinical trials. Design: We included 556 subjects from the Osteoarthritis Initiative study with manually read cartilage volume scores for the baseline and 1-year visits. The teams used their methods originally trained for the IWOAI 2019 challenge to segment the 1130 knee MRIs. These scans were anonymized and the teams were blinded to any subject or visit identifiers. Two teams also submitted updated methods. The resulting 9,040 segmentations are available online.The segmentations included tibial, femoral, and patellar compartments. In post-processing, we extracted medial and lateral tibial compartments and geometrically defined central medial and lateral femoral sub-compartments. The primary study outcome was the sensitivity to measure cartilage loss as defined by the standardized response mean (SRM). Results: For the tibial compartments, several of the DL segmentation methods had SRMs similar to the gold standard manual method. The highest DL SRM was for the lateral tibial compartment at 0.38 (the gold standard had 0.34). For the femoral compartments, the gold standard had higher SRMs than the automatic methods at 0.31/0.30 for medial/lateral compartments. Conclusion: The lower SRMs for the DL methods in the femoral compartments at 0.2 were possibly due to the simple sub-compartment extraction done during post-processing. The study demonstrated that state-of-the-art DL segmentation methods may be used in standardized longitudinal single-scanner clinical trials for well-defined cartilage compartments.