Merlin: A Vision Language Foundation Model for 3D Computed Tomography.

Over 85 million computed tomography (CT) scans are performed annually in the US, of which approximately one quarter focus on the abdomen. Given the current shortage of both general and specialized radiologists, there is a large impetus to use artificial intelligence to alleviate the burden of interpreting these complex imaging studies while simultaneously using the images to extract novel physiological insights. Prior state-of-the-art approaches for automated medical image interpretation leverage vision language models (VLMs) that utilize both the image and the corresponding textual radiology reports. However, current medical VLMs are generally limited to 2D images and short reports. To overcome these shortcomings for abdominal CT interpretation, we introduce Merlin - a 3D VLM that leverages both structured electronic health records (EHR) and unstructured radiology reports for pretraining without requiring additional manual annotations. We train Merlin using a high-quality clinical dataset of paired CT scans (6+ million images from 15,331 CTs), EHR diagnosis codes (1.8+ million codes), and radiology reports (6+ million tokens) for training. We comprehensively evaluate Merlin on 6 task types and 752 individual tasks. The non-adapted (off-the-shelf) tasks include zero-shot findings classification (31 findings), phenotype classification (692 phenotypes), and zero-shot cross-modal retrieval (image to findings and image to impressions), while model adapted tasks include 5-year chronic disease prediction (6 diseases), radiology report generation, and 3D semantic segmentation (20 organs). We perform internal validation on a test set of 5,137 CTs, and external validation on 7,000 clinical CTs and on two public CT datasets (VerSe, TotalSegmentator). Beyond these clinically-relevant evaluations, we assess the efficacy of various network architectures and training strategies to depict that Merlin has favorable performance to existing task-specific baselines. We derive data scaling laws to empirically assess training data needs for requisite downstream task performance. Furthermore, unlike conventional VLMs that require hundreds of GPUs for training, we perform all training on a single GPU. This computationally efficient design can help democratize foundation model training, especially for health systems with compute constraints. We plan to release our trained models, code, and dataset, pending manual removal of all protected health information.

Reproducibility of Quantitative Double-Echo Steady-State T2 Mapping of Knee Cartilage.

BACKGROUND: Cartilage T2 can detect joints at risk of developing osteoarthritis. The quantitative double-echo steady state (qDESS) sequence is attractive for knee cartilage T2 mapping because of its acquisition time of under 5 minutes. Understanding the reproducibility errors associated with qDESS T2 is essential to profiling the technical performance of this biomarker. PURPOSE: To examine the combined acquisition and segmentation reproducibility of knee cartilage qDESS T2 using two different regional analysis schemes: 1) manual segmentation of subregions loaded during common activities and 2) automatic subregional segmentation. STUDY TYPE: Prospective. SUBJECTS: 11 uninjured participants (age: 28 ± 3 years; 8 (73%) female). FIELD STRENGTH/SEQUENCE: 3-T, qDESS. ASSESSMENT: Test-retest T2 maps were acquired twice on the same day and with a 1-week interval between scans. For each acquisition, average cartilage T2 was calculated in four manually segmented regions encompassing tibiofemoral contact areas during common activities and 12 automatically segmented regions from the deep-learning open-source framework for musculoskeletal MRI analysis (DOSMA) encompassing medial and lateral anterior, central, and posterior tibiofemoral regions. Test-retest T2 values from matching regions were used to evaluate reproducibility. STATISTICAL TESTS: Coefficients of variation (%CV), root-mean-square-average-CV (%RMSA-CV), and intraclass correlation coefficients (ICCs) assessed test-retest T2 reproducibility. The median of test-retest standard deviations was used for T2 precision. Bland-Altman (BA) analyses examined test-retest biases. The smallest detectable difference (SDD) was defined as the BA limit of agreement of largest magnitude. Significance was accepted for P < 0.05. RESULTS: All cartilage regions across both segmentation schemes demonstrated intraday and interday qDESS T2 CVs and RMSA-CVs of ≤5%. T2 ICC values >0.75 were observed in the majority of regions but were more variable in interday tibial comparisons. Test-retest T2 precision was <1.3 msec. The T2 SDD was 3.8 msec. DATA CONCLUSION: Excellent CV and RMSA-CV reproducibility may suggest that qDESS T2 increases or decreases >5% (3.8 msec) could represent changes to cartilage composition. TECHNICAL EFFICACY: Stage 2.

ShapeMed-Knee: A Dataset and Neural Shape Model Benchmark for Modeling 3D Femurs.

Analyzing anatomic shapes of tissues and organs is pivotal for accurate disease diagnostics and clinical decision-making. One prominent disease that depends on anatomic shape analysis is osteoarthritis, which affects 30 million Americans. To advance osteoarthritis diagnostics and prognostics, we introduce ShapeMed-Knee, a 3D shape dataset with 9,376 high-resolution, medical-imaging-based 3D shapes of both femur bone and cartilage. Besides data, ShapeMed-Knee includes two benchmarks for assessing reconstruction accuracy and five clinical prediction tasks that assess the utility of learned shape representations. Leveraging ShapeMed-Knee, we develop and evaluate a novel hybrid explicit-implicit neural shape model which achieves up to 40% better reconstruction accuracy than a statistical shape model and implicit neural shape model. Our hybrid models achieve state-of-the-art performance for preserving cartilage biomarkers; they're also the first models to successfully predict localized structural features of osteoarthritis, outperforming shape models and convolutional neural networks applied to raw magnetic resonance images and segmentations. The ShapeMed-Knee dataset provides medical evaluations to reconstruct multiple anatomic surfaces and embed meaningful disease-specific information. ShapeMed-Knee reduces barriers to applying 3D modeling in medicine, and our benchmarks highlight that advancements in 3D modeling can enhance the diagnosis and risk stratification for complex diseases. The dataset, code, and benchmarks will be made freely accessible.

Applications of Artificial Intelligence for Pediatric Cancer Imaging.

Artificial intelligence (AI) is transforming medical imaging of adult patients. However, its utilization in pediatric oncology imaging remains constrained, in part due to the inherent data scarcity associated with childhood cancers. Pediatric cancers are rare, and imaging technologies are evolving rapidly, leading to insufficient data of a particular type to effectively train these algorithms. The small market size of pediatrics compared to adults could also contribute to this challenge, as market size is a driver of commercialization. This article provides an overview of the current state of AI applications for pediatric cancer imaging, including applications for medical image acquisition, processing, reconstruction, segmentation, diagnosis, staging, and treatment response monitoring. While current developments are promising, impediments due to diverse anatomies of growing children and nonstandardized imaging protocols have led to limited clinical translation thus far. Opportunities include leveraging reconstruction algorithms to achieve accelerated low-dose imaging and automating the generation of metric-based staging and treatment monitoring scores. Transfer-learning of adult-based AI models to pediatric cancers, multi-institutional data sharing, and ethical data privacy practices for pediatric patients with rare cancers will be keys to unlocking AI's full potential for clinical translation and improved outcomes for these young patients.

Missing Wedge Completion via Unsupervised Learning with Coordinate Networks.

Cryogenic electron tomography (cryoET) is a powerful tool in structural biology, enabling detailed 3D imaging of biological specimens at a resolution of nanometers. Despite its potential, cryoET faces challenges such as the missing wedge problem, which limits reconstruction quality due to incomplete data collection angles. Recently, supervised deep learning methods leveraging convolutional neural networks (CNNs) have considerably addressed this issue; however, their pretraining requirements render them susceptible to inaccuracies and artifacts, particularly when representative training data is scarce. To overcome these limitations, we introduce a proof-of-concept unsupervised learning approach using coordinate networks (CNs) that optimizes network weights directly against input projections. This eliminates the need for pretraining, reducing reconstruction runtime by 3 - 20× compared to supervised methods. Our in silico results show improved shape completion and reduction of missing wedge artifacts, assessed through several voxel-based image quality metrics in real space and a novel directional Fourier Shell Correlation (FSC) metric. Our study illuminates benefits and considerations of both supervised and unsupervised approaches, guiding the development of improved reconstruction strategies.

Missing Wedge Completion via Unsupervised Learning with Coordinate Networks.

Cryogenic electron tomography (cryoET) is a powerful tool in structural biology, enabling detailed 3D imaging of biological specimens at a resolution of nanometers. Despite its potential, cryoET faces challenges such as the missing wedge problem, which limits reconstruction quality due to incomplete data collection angles. Recently, supervised deep learning methods leveraging convolutional neural networks (CNNs) have considerably addressed this issue; however, their pretraining requirements render them susceptible to inaccuracies and artifacts, particularly when representative training data is scarce. To overcome these limitations, we introduce a proof-of-concept unsupervised learning approach using coordinate networks (CNs) that optimizes network weights directly against input projections. This eliminates the need for pretraining, reducing reconstruction runtime by 3-20× compared to supervised methods. Our in silico results show improved shape completion and reduction of missing wedge artifacts, assessed through several voxel-based image quality metrics in real space and a novel directional Fourier Shell Correlation (FSC) metric. Our study illuminates benefits and considerations of both supervised and unsupervised approaches, guiding the development of improved reconstruction strategies.

Automated abdominal CT contrast phase detection using an interpretable and open-source artificial intelligence algorithm.

OBJECTIVES: To develop and validate an open-source artificial intelligence (AI) algorithm to accurately detect contrast phases in abdominal CT scans. MATERIALS AND METHODS: Retrospective study aimed to develop an AI algorithm trained on 739 abdominal CT exams from 2016 to 2021, from 200 unique patients, covering 1545 axial series. We performed segmentation of five key anatomic structures-aorta, portal vein, inferior vena cava, renal parenchyma, and renal pelvis-using TotalSegmentator, a deep learning-based tool for multi-organ segmentation, and a rule-based approach to extract the renal pelvis. Radiomics features were extracted from the anatomical structures for use in a gradient-boosting classifier to identify four contrast phases: non-contrast, arterial, venous, and delayed. Internal and external validation was performed using the F1 score and other classification metrics, on the external dataset "VinDr-Multiphase CT". RESULTS: The training dataset consisted of 172 patients (mean age, 70 years ± 8, 22% women), and the internal test set included 28 patients (mean age, 68 years ± 8, 14% women). In internal validation, the classifier achieved an accuracy of 92.3%, with an average F1 score of 90.7%. During external validation, the algorithm maintained an accuracy of 90.1%, with an average F1 score of 82.6%. Shapley feature attribution analysis indicated that renal and vascular radiodensity values were the most important for phase classification. CONCLUSION: An open-source and interpretable AI algorithm accurately detects contrast phases in abdominal CT scans, with high accuracy and F1 scores in internal and external validation, confirming its generalization capability. CLINICAL RELEVANCE STATEMENT: Contrast phase detection in abdominal CT scans is a critical step for downstream AI applications, deploying algorithms in the clinical setting, and for quantifying imaging biomarkers, ultimately allowing for better diagnostics and increased access to diagnostic imaging. KEY POINTS: Digital Imaging and Communications in Medicine labels are inaccurate for determining the abdominal CT scan phase. AI provides great help in accurately discriminating the contrast phase. Accurate contrast phase determination aids downstream AI applications and biomarker quantification.

Abdominal CT metrics in 17,646 patients reveal associations between myopenia, myosteatosis, and medical phenotypes: a phenome-wide association study.

BACKGROUND: Deep learning facilitates large-scale automated imaging evaluation of body composition. However, associations of body composition biomarkers with medical phenotypes have been underexplored. Phenome-wide association study (PheWAS) techniques search for medical phenotypes associated with biomarkers. A PheWAS integrating large-scale analysis of imaging biomarkers and electronic health record (EHR) data could discover previously unreported associations and validate expected associations. Here we use PheWAS methodology to determine the association of abdominal CT-based skeletal muscle metrics with medical phenotypes in a large North American cohort. METHODS: An automated deep learning pipeline was used to measure skeletal muscle index (SMI; biomarker of myopenia) and skeletal muscle density (SMD; biomarker of myosteatosis) from abdominal CT scans of adults between 2012 and 2018. A PheWAS was performed with logistic regression using patient sex and age as covariates to assess for associations between CT-derived muscle metrics and 611 common EHR-derived medical phenotypes. PheWAS P values were considered significant at a Bonferroni corrected threshold (α = 0.05/1222). FINDINGS: 17,646 adults (mean age, 56 years ± 19 [SD]; 57.5% women) were included. CT-derived SMI was significantly associated with 268 medical phenotypes; SMD with 340 medical phenotypes. Previously unreported associations with the highest magnitude of significance included higher SMI with decreased cardiac dysrhythmias (OR [95% CI], 0.59 [0.55-0.64]; P < 0.0001), decreased epilepsy (OR, 0.59 [0.50-0.70]; P < 0.0001), and increased elevated prostate-specific antigen (OR, 1.84 [1.47-2.31]; P < 0.0001), and higher SMD with decreased decubitus ulcers (OR, 0.36 [0.31-0.42]; P < 0.0001), sleep disorders (OR, 0.39 [0.32-0.47]; P < 0.0001), and osteomyelitis (OR, 0.43 [0.36-0.52]; P < 0.0001). INTERPRETATION: PheWAS methodology reveals previously unreported associations between CT-derived biomarkers of myopenia and myosteatosis and EHR medical phenotypes. The high-throughput PheWAS technique applied on a population scale can generate research hypotheses related to myopenia and myosteatosis and can be adapted to research possible associations of other imaging biomarkers with hundreds of EHR medical phenotypes. FUNDING: National Institutes of Health, Stanford AIMI-HAI pilot grant, Stanford Precision Health and Integrated Diagnostics, Stanford Cardiovascular Institute, Stanford Center for Digital Health, and Stanford Knight-Hennessy Scholars.

Adapted large language models can outperform medical experts in clinical text summarization.

Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP) tasks, their effectiveness on a diverse range of clinical summarization tasks remains unproven. Here we applied adaptation methods to eight LLMs, spanning four distinct clinical summarization tasks: radiology reports, patient questions, progress notes and doctor-patient dialogue. Quantitative assessments with syntactic, semantic and conceptual NLP metrics reveal trade-offs between models and adaptation methods. A clinical reader study with 10 physicians evaluated summary completeness, correctness and conciseness; in most cases, summaries from our best-adapted LLMs were deemed either equivalent (45%) or superior (36%) compared with summaries from medical experts. The ensuing safety analysis highlights challenges faced by both LLMs and medical experts, as we connect errors to potential medical harm and categorize types of fabricated information. Our research provides evidence of LLMs outperforming medical experts in clinical text summarization across multiple tasks. This suggests that integrating LLMs into clinical workflows could alleviate documentation burden, allowing clinicians to focus more on patient care.

Self-Supervised Learning Improves Accuracy and Data Efficiency for IMU-Based Ground Reaction Force Estimation.

OBJECTIVE: Recent deep learning techniques hold promise to enable IMU-driven kinetic assessment; however, they require large extents of ground reaction force (GRF) data to serve as labels for supervised model training. We thus propose using existing self-supervised learning (SSL) techniques to leverage large IMU datasets to pre-train deep learning models, which can improve the accuracy and data efficiency of IMU-based GRF estimation. METHODS: We performed SSL by masking a random portion of the input IMU data and training a transformer model to reconstruct the masked portion. We systematically compared a series of masking ratios across three pre-training datasets that included real IMU data, synthetic IMU data, or a combination of the two. Finally, we built models that used pre-training and labeled data to estimate GRF during three prediction tasks: overground walking, treadmill walking, and drop landing. RESULTS: When using the same amount of labeled data, SSL pre-training significantly improved the accuracy of 3-axis GRF estimation during walking compared to baseline models trained by conventional supervised learning. Fine-tuning SSL model with 1-10% of walking data yielded comparable accuracy to training baseline model with 100% of walking data. The optimal masking ratio for SSL is 6.25-12.5%. CONCLUSION: SSL leveraged large real and synthetic IMU datasets to increase the accuracy and data efficiency of deep-learning-based GRF estimation, reducing the need for labeled data. SIGNIFICANCE: This work, with its open-source code and models, may unlock broader use cases of IMU-driven kinetic assessment by mitigating the scarcity of GRF measurements in practical applications.

Self-Supervised Learning Improves Accuracy and Data Efficiency for IMU-Based Ground Reaction Force Estimation.

Objective: Recent deep learning techniques hold promise to enable IMU-driven kinetic assessment; however, they require large extents of ground reaction force (GRF) data to serve as labels for supervised model training. We thus propose using existing self-supervised learning (SSL) techniques to leverage large IMU datasets to pre-train deep learning models, which can improve the accuracy and data efficiency of IMU-based GRF estimation. Methods: We performed SSL by masking a random portion of the input IMU data and training a transformer model to reconstruct the masked portion. We systematically compared a series of masking ratios across three pre-training datasets that included real IMU data, synthetic IMU data, or a combination of the two. Finally, we built models that used pre-training and labeled data to estimate GRF during three prediction tasks: overground walking, treadmill walking, and drop landing. Results: When using the same amount of labeled data, SSL pre-training significantly improved the accuracy of 3-axis GRF estimation during walking compared to baseline models trained by conventional supervised learning. Fine-tuning SSL model with 1-10% of walking data yielded comparable accuracy to training baseline model with 100% of walking data. The optimal masking ratio for SSL is 6.25-12.5%. Conclusion: SSL leveraged large real and synthetic IMU datasets to increase the accuracy and data efficiency of deep-learning-based GRF estimation, reducing the need for labeled data. Significance: This work, with its open-source code and models, may unlock broader use cases of IMU-driven kinetic assessment by mitigating the scarcity of GRF measurements in practical applications.

Biomarkers of Body Composition.

The importance and impact of imaging biomarkers has been increasing over the past few decades. We review the relevant clinical and imaging terminology needed to understand the clinical and research applications of body composition. Imaging biomarkers of bone, muscle, and fat tissues obtained with dual-energy X-ray absorptiometry, computed tomography, magnetic resonance imaging, and ultrasonography are described.

Differences in Anatomic Adaptation and Injury Patterns Related to Valgus Extension Overload in Overhead Throwing Athletes.

The purpose of our study was to determine differences in adaptative and injury patterns in the elbow related to valgus extension overload (VEO) in overhead throwing athletes by age. A total of 86 overhead throwing athletes and 23 controls underwent MRI or MR arthrography (MRA) of the elbow. Throwing athletes were divided by age into three groups: ≤16 years (26 subjects), 17-19 years (25 subjects), and ≥20 years (35 subjects). Consensus interpretation of each MRI was performed, with measurements of ulnar collateral ligament (UCL) thickness and subchondral sclerosis at the radial head, humeral trochlea, and olecranon process. A higher frequency of apophyseal and stress injuries was seen in adolescent athletes and increased incidence of soft tissue injuries was observed in older athletes. Early adaptive and degenerative changes were observed with high frequency independent of age. Significant differences were observed between athletes and controls for UCL thickness (p < 0.001) and subchondral sclerosis at the radial head (p < 0.001), humeral trochlea (p < 0.001), and olecranon process (p < 0.001). Significant differences based on athlete age were observed for UCL thickness (p < 0.001) and subchondral sclerosis at the olecranon process (p = 0.002). Our study highlights differences in anatomic adaptations related to VEO at the elbow between overhead throwing athletes and control subjects, as well as across age in throwing athletes.

Skeletal Muscle Area on CT: Determination of an Optimal Height Scaling Power and Testing for Mortality Risk Prediction.

BACKGROUND. Sarcopenia is commonly assessed on CT by use of the skeletal muscle index (SMI), which is calculated as the skeletal muscle area (SMA) at L3 divided by patient height squared (i.e., a height scaling power of 2). OBJECTIVE. The purpose of this study was to determine the optimal height scaling power for SMA measurements on CT and to test the influence of the derived optimal scaling power on the utility of SMI in predicting all-cause mortality. METHODS. This retrospective study included 16,575 patients (6985 men, 9590 women; mean age, 56.4 years) who underwent abdominal CT from December 2012 through October 2018. The SMA at L3 was determined using automated software. The sample was stratified into two groups: 5459 patients without major medical conditions (based on ICD-9 and ICD-10 codes) who were included in the analysis for determining the optimal height scaling power and 11,116 patients with major medical conditions who were included for the purpose of testing this power. The optimal scaling power was determined by allometric analysis (whereby regression coefficients were fitted to log-linear sex-specific models relating height to SMA) and by analysis of statistical independence of SMI from height across scaling powers. Cox proportional hazards models were used to test the influence of the derived optimal scaling power on the utility of SMI in predicting all-cause mortality. RESULTS. In allometric analysis, the regression coefficient of log(height) in patients 40 years old and younger was 1.02 in men and 1.08 in women, and in patients older than 40 years old, it was 1.07 in men and 1.10 in women (all p < .05 vs regression coefficient of 2). In analyses for statistical independence of SMI from height, the optimal height scaling power (i.e., those yielding correlations closest to 0) was, in patients 40 years old and younger, 0.97 in men and 1.08 in women, whereas in patients older than 40 years old, it was 1.03 in men and 1.09 in women. In the Cox model used for testing, SMI predicted all-cause mortality with a higher concordance index using of a height scaling power of 1 rather than 2 in men (0.675 vs 0.663, p < .001) and in women (0.664 vs 0.653, p < .001). CONCLUSION. The findings support a height scaling power of 1, rather than a conventional power of 2, for SMI computation. CLINICAL IMPACT. A revised height scaling power for SMI could impact the utility of CT-based sarcopenia diagnoses in risk assessment.

Accelerated Musculoskeletal Magnetic Resonance Imaging.

With a substantial growth in the use of musculoskeletal MRI, there has been a growing need to improve MRI workflow, and faster imaging has been suggested as one of the solutions for a more efficient examination process. Consequently, there have been considerable advances in accelerated MRI scanning methods. This article aims to review the basic principles and applications of accelerated musculoskeletal MRI techniques including widely used conventional acceleration methods, more advanced deep learning-based techniques, and new approaches to reduce scan time. Specifically, conventional accelerated MRI techniques, including parallel imaging, compressed sensing, and simultaneous multislice imaging, and deep learning-based accelerated MRI techniques, including undersampled MR image reconstruction, super-resolution imaging, artifact correction, and generation of unacquired contrast images, are discussed. Finally, new approaches to reduce scan time, including synthetic MRI, novel sequences, and new coil setups and designs, are also reviewed. We believe that a deep understanding of these fast MRI techniques and proper use of combined acceleration methods will synergistically improve scan time and MRI workflow in daily practice. EVIDENCE LEVEL: 3 TECHNICAL EFFICACY: Stage 1.

Clinical Text Summarization: Adapting Large Language Models Can Outperform Human Experts.

Sifting through vast textual data and summarizing key information from electronic health records (EHR) imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown immense promise in natural language processing (NLP) tasks, their efficacy on a diverse range of clinical summarization tasks has not yet been rigorously demonstrated. In this work, we apply domain adaptation methods to eight LLMs, spanning six datasets and four distinct clinical summarization tasks: radiology reports, patient questions, progress notes, and doctor-patient dialogue. Our thorough quantitative assessment reveals trade-offs between models and adaptation methods in addition to instances where recent advances in LLMs may not improve results. Further, in a clinical reader study with ten physicians, we show that summaries from our best-adapted LLMs are preferable to human summaries in terms of completeness and correctness. Our ensuing qualitative analysis highlights challenges faced by both LLMs and human experts. Lastly, we correlate traditional quantitative NLP metrics with reader study scores to enhance our understanding of how these metrics align with physician preferences. Our research marks the first evidence of LLMs outperforming human experts in clinical text summarization across multiple tasks. This implies that integrating LLMs into clinical workflows could alleviate documentation burden, empowering clinicians to focus more on personalized patient care and the inherently human aspects of medicine.

Opportunistic assessment of ischemic heart disease risk using abdominopelvic computed tomography and medical record data: a multimodal explainable artificial intelligence approach.

Current risk scores using clinical risk factors for predicting ischemic heart disease (IHD) events-the leading cause of global mortality-have known limitations and may be improved by imaging biomarkers. While body composition (BC) imaging biomarkers derived from abdominopelvic computed tomography (CT) correlate with IHD risk, they are impractical to measure manually. Here, in a retrospective cohort of 8139 contrast-enhanced abdominopelvic CT examinations undergoing up to 5 years of follow-up, we developed multimodal opportunistic risk assessment models for IHD by automatically extracting BC features from abdominal CT images and integrating these with features from each patient's electronic medical record (EMR). Our predictive methods match and, in some cases, outperform clinical risk scores currently used in IHD risk assessment. We provide clinical interpretability of our model using a new method of determining tissue-level contributions from CT along with weightings of EMR features contributing to IHD risk. We conclude that such a multimodal approach, which automatically integrates BC biomarkers and EMR data, can enhance IHD risk assessment and aid primary prevention efforts for IHD. To further promote research, we release the Opportunistic L3 Ischemic heart disease (OL3I) dataset, the first public multimodal dataset for opportunistic CT prediction of IHD.

OpenCap: Human movement dynamics from smartphone videos.

Measures of human movement dynamics can predict outcomes like injury risk or musculoskeletal disease progression. However, these measures are rarely quantified in large-scale research studies or clinical practice due to the prohibitive cost, time, and expertise required. Here we present and validate OpenCap, an open-source platform for computing both the kinematics (i.e., motion) and dynamics (i.e., forces) of human movement using videos captured from two or more smartphones. OpenCap leverages pose estimation algorithms to identify body landmarks from videos; deep learning and biomechanical models to estimate three-dimensional kinematics; and physics-based simulations to estimate muscle activations and musculoskeletal dynamics. OpenCap's web application enables users to collect synchronous videos and visualize movement data that is automatically processed in the cloud, thereby eliminating the need for specialized hardware, software, and expertise. We show that OpenCap accurately predicts dynamic measures, like muscle activations, joint loads, and joint moments, which can be used to screen for disease risk, evaluate intervention efficacy, assess between-group movement differences, and inform rehabilitation decisions. Additionally, we demonstrate OpenCap's practical utility through a 100-subject field study, where a clinician using OpenCap estimated musculoskeletal dynamics 25 times faster than a laboratory-based approach at less than 1% of the cost. By democratizing access to human movement analysis, OpenCap can accelerate the incorporation of biomechanical metrics into large-scale research studies, clinical trials, and clinical practice.