Semisupervised transfer learning for evaluation of model classification performance.

In many modern machine learning applications, changes in covariate distributions and difficulty in acquiring outcome information have posed challenges to robust model training and evaluation. Numerous transfer learning methods have been developed to robustly adapt the model itself to some unlabeled target populations using existing labeled data in a source population. However, there is a paucity of literature on transferring performance metrics, especially receiver operating characteristic (ROC) parameters, of a trained model. In this paper, we aim to evaluate the performance of a trained binary classifier on unlabeled target population based on ROC analysis. We proposed Semisupervised Transfer lEarning of Accuracy Measures (STEAM), an efficient three-step estimation procedure that employs (1) double-index modeling to construct calibrated density ratio weights and (2) robust imputation to leverage the large amount of unlabeled data to improve estimation efficiency. We establish the consistency and asymptotic normality of the proposed estimator under the correct specification of either the density ratio model or the outcome model. We also correct for potential overfitting bias in the estimators in finite samples with cross-validation. We compare our proposed estimators to existing methods and show reductions in bias and gains in efficiency through simulations. We illustrate the practical utility of the proposed method on evaluating prediction performance of a phenotyping model for rheumatoid arthritis (RA) on a temporally evolving EHR cohort.

The contributions of cartilage endplate composition and vertebral bone marrow fat to intervertebral disc degeneration in patients with chronic low back pain.

PURPOSE: The composition of the subchondral bone marrow and cartilage endplate (CEP) could affect intervertebral disc health by influencing vertebral perfusion and nutrient diffusion. However, the relative contributions of these factors to disc degeneration in patients with chronic low back pain (cLBP) have not been quantified. The goal of this study was to use compositional biomarkers derived from quantitative MRI to establish how CEP composition (surrogate for permeability) and vertebral bone marrow fat fraction (BMFF, surrogate for perfusion) relate to disc degeneration. METHODS: MRI data from 60 patients with cLBP were included in this prospective observational study (28 female, 32 male; age = 40.0 ± 11.9 years, 19-65 [mean ± SD, min-max]). Ultra-short echo-time MRI was used to calculate CEP T2* relaxation times (reflecting biochemical composition), water-fat MRI was used to calculate vertebral BMFF, and T1ρ MRI was used to calculate T1ρ relaxation times in the nucleus pulposus (NP T1ρ, reflecting proteoglycan content and degenerative grade). Univariate linear regression was used to assess the independent effects of CEP T2* and vertebral BMFF on NP T1ρ. Mixed effects multivariable linear regression accounting for age, sex, and BMI was used to assess the combined relationship between variables. RESULTS: CEP T2* and vertebral BMFF were independently associated with NP T1ρ (p = 0.003 and 0.0001, respectively). After adjusting for age, sex, and BMI, NP T1ρ remained significantly associated with CEP T2* (p = 0.0001) but not vertebral BMFF (p = 0.43). CONCLUSION: Poor CEP composition plays a significant role in disc degeneration severity and can affect disc health both with and without deficits in vertebral perfusion.

Opal: an implementation science tool for machine learning clinical decision support in anesthesia.

Opal is the first published example of a full-stack platform infrastructure for an implementation science designed for ML in anesthesia that solves the problem of leveraging ML for clinical decision support. Users interact with a secure online Opal web application to select a desired operating room (OR) case cohort for data extraction, visualize datasets with built-in graphing techniques, and run in-client ML or extract data for external use. Opal was used to obtain data from 29,004 unique OR cases from a single academic institution for pre-operative prediction of post-operative acute kidney injury (AKI) based on creatinine KDIGO criteria using predictors which included pre-operative demographic, past medical history, medications, and flowsheet information. To demonstrate utility with unsupervised learning, Opal was also used to extract intra-operative flowsheet data from 2995 unique OR cases and patients were clustered using PCA analysis and k-means clustering. A gradient boosting machine model was developed using an 80/20 train to test ratio and yielded an area under the receiver operating curve (ROC-AUC) of 0.85 with 95% CI [0.80-0.90]. At the default probability decision threshold of 0.5, the model sensitivity was 0.9 and the specificity was 0.8. K-means clustering was performed to partition the cases into two clusters and for hypothesis generation of potential groups of outcomes related to intraoperative vitals. Opal's design has created streamlined ML functionality for researchers and clinicians in the perioperative setting and opens the door for many future clinical applications, including data mining, clinical simulation, high-frequency prediction, and quality improvement.

A cryo-tomography-based volumetric model of the actin core of mouse vestibular hair cell stereocilia lacking plastin 1.

Electron cryo-tomography allows for high-resolution imaging of stereocilia in their native state. Because their actin filaments have a higher degree of order, we imaged stereocilia from mice lacking the actin crosslinker plastin 1 (PLS1). We found that while stereocilia actin filaments run 13 nm apart in parallel for long distances, there were gaps of significant size that were stochastically distributed throughout the actin core. Actin crosslinkers were distributed through the stereocilium, but did not occupy all possible binding sites. At stereocilia tips, protein density extended beyond actin filaments, especially on the side of the tip where a tip link is expected to anchor. Along the shaft, repeating density was observed that corresponds to actin-to-membrane connectors. In the taper region, most actin filaments terminated near the plasma membrane. The remaining filaments twisted together to make a tighter bundle than was present in the shaft region; the spacing between them decreased from 13 nm to 9 nm, and the apparent filament diameter decreased from 6.4 to 4.8 nm. Our models illustrate detailed features of distinct structural domains that are present within the stereocilium.

Deep-learning-based biomarker of spinal cartilage endplate health using ultra-short echo time magnetic resonance imaging.

Background: T2* relaxation times in the spinal cartilage endplate (CEP) measured using ultra-short echo time magnetic resonance imaging (UTE MRI) reflect aspects of biochemical composition that influence the CEP's permeability to nutrients. Deficits in CEP composition measured using T2* biomarkers from UTE MRI are associated with more severe intervertebral disc degeneration in patients with chronic low back pain (cLBP). The goal of this study was to develop an objective, accurate, and efficient deep-learning-based method for calculating biomarkers of CEP health using UTE images. Methods: Multi-echo UTE MRI of the lumbar spine was acquired from a prospectively enrolled cross-sectional and consecutive cohort of 83 subjects spanning a wide range of ages and cLBP-related conditions. CEPs from the L4-S1 levels were manually segmented on 6,972 UTE images and used to train neural networks utilizing the u-net architecture. CEP segmentations and mean CEP T2* values derived from manually- and model-generated segmentations were compared using Dice scores, sensitivity, specificity, Bland-Altman, and receiver-operator characteristic (ROC) analysis. Signal-to-noise (SNR) and contrast-to-noise (CNR) ratios were calculated and related to model performance. Results: Compared with manual CEP segmentations, model-generated segmentations achieved sensitives of 0.80-0.91, specificities of 0.99, Dice scores of 0.77-0.85, area under the receiver-operating characteristic curve values of 0.99, and precision-recall (PR) AUC values of 0.56-0.77, depending on spinal level and sagittal image position. Mean CEP T2* values and principal CEP angles derived from the model-predicted segmentations had low bias in an unseen test dataset (T2* bias =0.33±2.37 ms, angle bias =0.36±2.65°). To simulate a hypothetical clinical scenario, the predicted segmentations were used to stratify CEPs into high, medium, and low T2* groups. Group predictions had diagnostic sensitivities of 0.77-0.86 and specificities of 0.86-0.95. Model performance was positively associated with image SNR and CNR. Conclusions: The trained deep learning models enable accurate, automated CEP segmentations and T2* biomarker computations that are statistically similar to those from manual segmentations. These models address limitations with inefficiency and subjectivity associated with manual methods. Such techniques could be used to elucidate the role of CEP composition in disc degeneration etiology and guide emerging therapies for cLBP.

Knowledge-Driven Online Multimodal Automated Phenotyping System.

Though electronic health record (EHR) systems are a rich repository of clinical information with large potential, the use of EHR-based phenotyping algorithms is often hindered by inaccurate diagnostic records, the presence of many irrelevant features, and the requirement for a human-labeled training set. In this paper, we describe a knowledge-driven online multimodal automated phenotyping (KOMAP) system that i) generates a list of informative features by an online narrative and codified feature search engine (ONCE) and ii) enables the training of a multimodal phenotyping algorithm based on summary data. Powered by composite knowledge from multiple EHR sources, online article corpora, and a large language model, features selected by ONCE show high concordance with the state-of-the-art AI models (GPT4 and ChatGPT) and encourage large-scale phenotyping by providing a smaller but highly relevant feature set. Validation of the KOMAP system across four healthcare centers suggests that it can generate efficient phenotyping algorithms with robust performance. Compared to other methods requiring patient-level inputs and gold-standard labels, the fully online KOMAP provides a significant opportunity to enable multi-center collaboration.

Evaluation of human cartilage endplate composition using MRI: Spatial variation, association with adjacent disc degeneration, and in vivo repeatability.

Cartilage endplate (CEP) biochemical composition may influence disc degeneration and regeneration. However, evaluating CEP composition in patients remains a challenge. We used T2* mapping from ultrashort echo-time (UTE) magnetic resonance imaging (MRI), which is sensitive to CEP hydration, to investigate spatial variations in CEP T2* values and to determine how CEP T2* values correlate with adjacent disc degeneration. Thirteen human cadavers (56.4 ± 12.7 years) and seven volunteers (36.9 ± 10.9 years) underwent 3T MRI, including UTE and T1ρ mapping sequences. Spatial mappings of T2* values in L4-S1 CEPs were generated from UTE images and compared between subregions. In the abutting discs, mean T1ρ values in the nucleus pulposus were compared between CEPs with high vs low T2* values. To assess in vivo repeatability, precision errors in mean T2* values, and intraclass correlation coefficients (ICC) were measured from repeat scans. Results showed that CEP T2* values were highest centrally and lowest posteriorly. In the youngest individuals (<50 years), who had mild-to-moderately degenerated Pfirrmann grade II-III discs, low CEP T2* values associated with severer disc degeneration: T1ρ values were 26.7% lower in subjects with low CEP T2* values (P = .025). In older individuals, CEP T2* values did not associate with disc degeneration (P = .39-.62). Precision errors in T2* ranged from 1.7 to 2.6 ms, and reliability was good-to-excellent (ICC = 0.89-0.94). These findings suggest that deficits in CEP composition, as indicated by low T2* values, associate with severer disc degeneration during the mild-to-moderate stages. Measuring CEP T2* values with UTE MRI may clarify the role of CEP composition in patients with mild-to-moderate disc degeneration.