Multi-animal pose estimation, identification and tracking with DeepLabCut.

Estimating the pose of multiple animals is a challenging computer vision problem: frequent interactions cause occlusions and complicate the association of detected keypoints to the correct individuals, as well as having highly similar looking animals that interact more closely than in typical multi-human scenarios. To take up this challenge, we build on DeepLabCut, an open-source pose estimation toolbox, and provide high-performance animal assembly and tracking-features required for multi-animal scenarios. Furthermore, we integrate the ability to predict an animal's identity to assist tracking (in case of occlusions). We illustrate the power of this framework with four datasets varying in complexity, which we release to serve as a benchmark for future algorithm development.

Resistin predicts disease severity and survival in patients with pulmonary arterial hypertension.

BACKGROUND: Abnormal remodeling of distal pulmonary arteries in patients with pulmonary arterial hypertension (PAH) leads to progressively increased pulmonary vascular resistance, followed by right ventricular hypertrophy and failure. Despite considerable advancements in PAH treatment prognosis remains poor. We aim to evaluate the potential for using the cytokine resistin as a genetic and biological marker for disease severity and survival in a large cohort of patients with PAH. METHODS: Biospecimens, clinical, and genetic data for 1121 adults with PAH, including 808 with idiopathic PAH (IPAH) and 313 with scleroderma-associated PAH (SSc-PAH), were obtained from a national repository. Serum resistin levels were measured by ELISA, and associations between resistin levels, clinical variables, and single nucleotide polymorphism genotypes were examined with multivariable regression models. Machine-learning (ML) algorithms were applied to develop and compare risk models for mortality prediction. RESULTS: Resistin levels were significantly higher in all PAH samples and PAH subtype (IPAH and SSc-PAH) samples than in controls (P < .0001) and had significant discriminative abilities (AUCs of 0.84, 0.82, and 0.91, respectively; P < .001). High resistin levels (above 4.54 ng/mL) in PAH patients were associated with older age (P = .001), shorter 6-min walk distance (P = .001), and reduced cardiac performance (cardiac index, P = .016). Interestingly, mutant carriers of either rs3219175 or rs3745367 had higher resistin levels (adjusted P = .0001). High resistin levels in PAH patients were also associated with increased risk of death (hazard ratio: 2.6; 95% CI: 1.27-5.33; P < .0087). Comparisons of ML-derived survival models confirmed satisfactory prognostic value of the random forest model (AUC = 0.70, 95% CI: 0.62-0.79) for PAH. CONCLUSIONS: This work establishes the importance of resistin in the pathobiology of human PAH. In line with its function in rodent models, serum resistin represents a novel biomarker for PAH prognostication and may indicate a new therapeutic avenue. ML-derived survival models highlighted the importance of including resistin levels to improve performance. Future studies are needed to develop multi-marker assays that improve noninvasive risk stratification.

A machine learning based approach towards high-dimensional mediation analysis.

Mediation analysis is used to investigate the role of intermediate variables (mediators) that lie in the path between an exposure and an outcome variable. While significant research has focused on developing methods for assessing the influence of mediators on the exposure-outcome relationship, current approaches do not easily extend to settings where the mediator is high-dimensional. These situations are becoming increasingly common with the rapid increase of new applications measuring massive numbers of variables, including brain imaging, genomics, and metabolomics. In this work, we introduce a novel machine learning based method for identifying high dimensional mediators. The proposed algorithm iterates between using a machine learning model to map the high-dimensional mediators onto a lower-dimensional space, and using the predicted values as input in a standard three-variable mediation model. Hence, the machine learning model is trained to maximize the likelihood of the mediation model. Importantly, the proposed algorithm is agnostic to the machine learning model that is used, providing significant flexibility in the types of situations where it can be used. We illustrate the proposed methodology using data from two functional Magnetic Resonance Imaging (fMRI) studies. First, using data from a task-based fMRI study of thermal pain, we combine the proposed algorithm with a deep learning model to detect distributed, network-level brain patterns mediating the relationship between stimulus intensity (temperature) and reported pain at the single trial level. Second, using resting-state fMRI data from the Human Connectome Project, we combine the proposed algorithm with a connectome-based predictive modeling approach to determine brain functional connectivity measures that mediate the relationship between fluid intelligence and working memory accuracy. In both cases, our multivariate mediation model links exposure variables (thermal pain or fluid intelligence), high dimensional brain measures (single-trial brain activation maps or resting-state brain connectivity) and behavioral outcomes (pain report or working memory accuracy) into a single unified model. Using the proposed approach, we are able to identify brain-based measures that simultaneously encode the exposure variable and correlate with the behavioral outcome.

A simple computer vision pipeline reveals the effects of isolation on social interaction dynamics in Drosophila.

Isolation profoundly influences social behavior in all animals. In humans, isolation has serious effects on health. Drosophila melanogaster is a powerful model to study small-scale, temporally-transient social behavior. However, longer-term analysis of large groups of flies is hampered by the lack of effective and reliable tools. We built a new imaging arena and improved the existing tracking algorithm to reliably follow a large number of flies simultaneously. Next, based on the automatic classification of touch and graph-based social network analysis, we designed an algorithm to quantify changes in the social network in response to prior social isolation. We observed that isolation significantly and swiftly enhanced individual and local social network parameters depicting near-neighbor relationships. We explored the genome-wide molecular correlates of these behavioral changes and found that whereas behavior changed throughout the six days of isolation, gene expression alterations occurred largely on day one. These changes occurred mostly in metabolic genes, and we verified the metabolic changes by showing an increase of lipid content in isolated flies. In summary, we describe a highly reliable tracking and analysis pipeline for large groups of flies that we use to unravel the behavioral, molecular and physiological impact of isolation on social network dynamics in Drosophila.

Brains of verbal memory specialists show anatomical differences in language, memory and visual systems.

We studied a group of verbal memory specialists to determine whether intensive oral text memory is associated with structural features of hippocampal and lateral-temporal regions implicated in language processing. Professional Vedic Sanskrit Pandits in India train from childhood for around 10years in an ancient, formalized tradition of oral Sanskrit text memorization and recitation, mastering the exact pronunciation and invariant content of multiple 40,000-100,000 word oral texts. We conducted structural analysis of gray matter density, cortical thickness, local gyrification, and white matter structure, relative to matched controls. We found massive gray matter density and cortical thickness increases in Pandit brains in language, memory and visual systems, including i) bilateral lateral temporal cortices and ii) the anterior cingulate cortex and the hippocampus, regions associated with long and short-term memory. Differences in hippocampal morphometry matched those previously documented for expert spatial navigators and individuals with good verbal working memory. The findings provide unique insight into the brain organization implementing formalized oral knowledge systems.

Early predictive values of clinical assessments for ARDS mortality: a machine-learning approach.

Acute respiratory distress syndrome (ARDS) is a devastating critical care syndrome with significant morbidity and mortality. The objective of this study was to evaluate the predictive values of dynamic clinical indices by developing machine-learning (ML) models for early and accurate clinical assessment of the disease prognosis of ARDS. We conducted a retrospective observational study by applying dynamic clinical data collected in the ARDSNet FACTT Trial (n = 1000) to ML-based algorithms for predicting mortality. In order to compare the significance of clinical features dynamically, we further applied the random forest (RF) model to nine selected clinical parameters acquired at baseline and day 3 independently. An RF model trained using clinical data collected at day 3 showed improved performance and prognostication efficacy (area under the curve [AUC]: 0.84, 95% CI: 0.78-0.89) compared to baseline with an AUC value of 0.72 (95% CI: 0.65-0.78). Mean airway pressure (MAP), bicarbonate, age, platelet count, albumin, heart rate, and glucose were the most significant clinical indicators associated with mortality at day 3. Thus, clinical features collected early (day 3) improved performance of integrative ML models with better prognostication for mortality. Among these, MAP represented the most important feature for ARDS patients' early risk stratification.

Artificial intelligence and body composition.

AIMS: Although obesity is associated with chronic disease, a large section of the population with high BMI does not have an increased risk of metabolic disease. Increased visceral adiposity and sarcopenia are also risk factors for metabolic disease in people with normal BMI. Artificial Intelligence (AI) techniques can help assess and analyze body composition parameters for predicting cardiometabolic health. The purpose of the study was to systematically explore literature involving AI techniques for body composition assessment and observe general trends. METHODS: We searched the following databases: Embase, Web of Science, and PubMed. There was a total of 354 search results. After removing duplicates, irrelevant studies, and reviews(a total of 303), 51 studies were included in the systematic review. RESULTS: AI techniques have been studied for body composition analysis in the context of diabetes mellitus, hypertension, cancer and many specialized diseases. Imaging techniques employed for AI methods include CT (Computerized Tomography), MRI (Magnetic Resonance Imaging), ultrasonography, plethysmography, and EKG(Electrocardiogram). Automatic segmentation of body composition by deep learning with convolutional networks has helped determine and quantify muscle mass. Limitations include heterogeneity of study populations, inherent bias in sampling, and lack of generalizability. Different bias mitigation strategies should be evaluated to address these problems and improve the applicability of AI to body composition analysis. CONCLUSIONS: AI assisted measurement of body composition might assist in improved cardiovascular risk stratification when applied in the appropriate clinical context.

Relationship Between TSH Levels and Cognition in the Young Adult: An Analysis of the Human Connectome Project Data.

CONTEXT: The nature of the relationship between serum thyrotropin (TSH) levels and higher cognitive abilities is unclear, especially within the normal reference range and in the younger population. OBJECTIVE: To assess the relationship between serum TSH levels and mental health and sleep quality parameters (fluid intelligence [Gf], MMSE (Mini-Mental State Examination), depression scores, and, finally, Pittsburgh Sleep Quality Index (PSQI) scores (working memory, processing speed, and executive function) in young adults. METHODS: This was a retrospective analysis of the data from the Human Connectome Project (HCP). The HCP consortium is seeking to map human brain circuits systematically and identify their relationship to behavior in healthy adults. Included were 391 female and 412 male healthy participants aged 22-35 years at the time of the screening interview. We excluded persons with serum TSH levels outside the reference range (0.4-4.5 mU/L). TSH was transformed logarithmically (log TSH). All the key variables were normalized and then linear regression analysis was performed to assess the relationship between log TSH as a cofactor and Gf as the dependent variable. Finally, a machine learning method, random forest regression, predicted Gf from the dependent variables (including alcohol and tobacco use). The main outcome was normalized Gf (nGf) and Gf scores. RESULTS: Log TSH was a significant co-predictor of nGF in females (ß = 0.31(±0.1), P < .01) but not in males. Random forest analysis showed that the model(s) had a better predictive value for females (r = 0.39, mean absolute error [MAE] = 0.81) than males (r = 0.24, MAE = 0.77). CONCLUSION: Higher serum TSH levels might be associated with higher Gf scores in young women.

Role of Artificial Intelligence in Radiogenomics for Cancers in the Era of Precision Medicine.

Radiogenomics, a combination of "Radiomics" and "Genomics," using Artificial Intelligence (AI) has recently emerged as the state-of-the-art science in precision medicine, especially in oncology care. Radiogenomics syndicates large-scale quantifiable data extracted from radiological medical images enveloped with personalized genomic phenotypes. It fabricates a prediction model through various AI methods to stratify the risk of patients, monitor therapeutic approaches, and assess clinical outcomes. It has recently shown tremendous achievements in prognosis, treatment planning, survival prediction, heterogeneity analysis, reoccurrence, and progression-free survival for human cancer study. Although AI has shown immense performance in oncology care in various clinical aspects, it has several challenges and limitations. The proposed review provides an overview of radiogenomics with the viewpoints on the role of AI in terms of its promises for computational as well as oncological aspects and offers achievements and opportunities in the era of precision medicine. The review also presents various recommendations to diminish these obstacles.

Body fat predicts exercise capacity in persons with Type 2 Diabetes Mellitus: A machine learning approach.

Diabetes mellitus is associated with increased cardiovascular disease (CVD) related morbidity, mortality and death. Exercise capacity in persons with type 2 diabetes has been shown to be predictive of cardiovascular events. In this study, we used the data from the prospective randomized LOOK AHEAD study and used machine learning algorithms to help predict exercise capacity (measured in Mets) from the baseline data that included cardiovascular history, medications, blood pressure, demographic information, anthropometric and Dual-energy X-Ray Absorptiometry (DXA) measured body composition metrics. We excluded variables with high collinearity and included DXA obtained Subtotal (total minus head) fat percentage and Subtotal lean mass (gms). Thereafter, we used different machine learning methods to predict maximum exercise capacity. The different machine learning models showed a strong predictive performance for both females and males. Our study shows that using baseline data from a large prospective cohort, we can predict maximum exercise capacity in persons with diabetes mellitus. We show that subtotal fat percentage is the most important feature for predicting the exercise capacity for males and females after accounting for other important variables. Until now, BMI and waist circumference were commonly used surrogates for adiposity and there was a relative under-appreciation of body composition metrics for understanding the pathophysiology of CVD. The recognition of body fat percentage as an important marker in determining CVD risk has prognostic implications with respect to cardiovascular morbidity and mortality.

Machine learning approach to predicting albuminuria in persons with type 2 diabetes: An analysis of the LOOK AHEAD Cohort.

Albuminuria and estimated glomerular filtration rate (e-GFR) are early markers of renal disease and cardiovascular outcomes in persons with diabetes. Although body composition has been shown to predict systolic blood pressure, its application in predicting albuminuria is unknown. In this study, we have used machine learning methods to assess the risk of albuminuria in persons with diabetes using body composition and other determinants of metabolic health. This study is a comparative analysis of the different methods to predict albuminuria in persons with diabetes mellitus who are older than 40 years of age, using the LOOK AHEAD study cohort-baseline characteristics. Age, different metrics of body composition, duration of diabetes, hemoglobin A1c, serum creatinine, serum triglycerides, serum cholesterol, serum HDL, serum LDL, maximum exercise capacity, systolic blood pressure, diastolic blood pressure, and the ankle-brachial index are used as predictors of albuminuria. We used Area under the curve (AUC) as a metric to compare the classification results of different algorithms, and we show that AUC for the different models are as follows: Random forest classifier-0.65, gradient boost classifier-0.61, logistic regression-0.66, support vector classifier -0.61, multilayer perceptron -0.67, and stacking classifier-0.62. We used the Random forest model to show that the duration of diabetes, A1C, serum triglycerides, SBP, Maximum exercise Capacity, serum creatinine, subtotal lean mass, DBP, and subtotal fat mass are important features for the classification of albuminuria. In summary, when applied to metabolic imaging (using DXA), machine learning techniques offer unique insights into the risk factors that determine the development of albuminuria in diabetes.

Real-time quantitation of thyroidal radioiodine uptake in thyroid disease with monitoring by a collar detection device.

Radioactive iodine (RAI) is safe and effective in most patients with hyperthyroidism but not all individuals are cured by the first dose, and most develop post-RAI hypothyroidism. Postoperative RAI therapy for remnant ablation is successful in 80-90% of thyroid cancer patients and sometimes induces remission of nonresectable cervical and/or distant metastatic disease but the effective tumor dose is usually not precisely known and must be moderated to avoid short- and long-term adverse effects on other tissues. The Collar Therapy Indicator (COTI) is a radiation detection device embedded in a cloth collar secured around the patient's neck and connected to a recording and data transmission box. In previously published experience, the data can be collected at multiple time points, reflecting local cervical RAI exposure and correlating well with conventional methods. We evaluated the real-time uptake of RAI in patients with hyperthyroid Graves' disease and thyroid cancer. We performed a pilot feasibility prospective study. Data were analyzed using R© (version 4.0.3, The R Foundation for Statistical Computing, 2020), and Python (version 3.6, Matplotlib version 3.0.3). The COTI was able to provide a quantitative temporal pattern of uptake within the thyroid in persons with Graves' disease and lateralized the remnant tissue in persons with thyroid cancer. The study has demonstrated that the portable collar radiation detection device outside of a healthcare facility is accurate and feasible for use after administration of RAI for diagnostic studies and therapy to provide a complete collection of fractional target radioactivity data compared to that traditionally acquired with clinic-based measurements at one or two time-points.Clinical Trials Registration NCT03517579, DOR 5/7/2018.

Artificial intelligence may offer insight into factors determining individual TSH level.

The factors that determine Serum Thyrotropin (TSH) levels have been examined through different methods, using different covariates. However, the use of machine learning methods has so far not been studied in population databases like NHANES (National Health and Nutritional Examination Survey) to predict TSH. In this study, we performed a comparative analysis of different machine learning methods like Linear regression, Random forest, Support vector machine, multilayer perceptron and stacking regression to predict TSH and classify individuals with normal, low and high TSH levels. We considered Free T4, Anti-TPO antibodies, T3, Body Mass Index (BMI), Age and Ethnicity as the predictor variables. A total of 9818 subjects were included in this comparative analysis. We used coefficient of determination (r2) value to compare the results for predicting the TSH and show that the Random Forest, Gradient Boosting and Stacking Regression perform equally well in predicting TSH and achieve the highest r2 value = 0.13, with mean absolute error of 0.78. Moreover, we found that Anti-TPO is the most important feature in predicting TSH followed by Age, BMI, T3 and Free-T4 for the regression analysis. While classifying TSH into normal, high or low levels, our comparative analysis also shows that Random forest performs the best in the classification study, performed with individuals with normal, high and low levels of TSH. We found the following Areas Under Curve (AUC); for low TSH, AUC = 0.61, normal TSH, AUC = 0.61 and elevated TSH AUC = 0.69. Additionally, we found that Anti-TPO was the most important feature in classifying TSH. In this study, we suggest that artificial intelligence and machine learning methods might offer an insight into the complex hypothalamic-pituitary -thyroid axis and may be an invaluable tool that guides us in making appropriate therapeutic decisions (thyroid hormone dosing) for the individual patient.

Using DeepLabCut for 3D markerless pose estimation across species and behaviors.

Noninvasive behavioral tracking of animals during experiments is critical to many scientific pursuits. Extracting the poses of animals without using markers is often essential to measuring behavioral effects in biomechanics, genetics, ethology, and neuroscience. However, extracting detailed poses without markers in dynamically changing backgrounds has been challenging. We recently introduced an open-source toolbox called DeepLabCut that builds on a state-of-the-art human pose-estimation algorithm to allow a user to train a deep neural network with limited training data to precisely track user-defined features that match human labeling accuracy. Here, we provide an updated toolbox, developed as a Python package, that includes new features such as graphical user interfaces (GUIs), performance improvements, and active-learning-based network refinement. We provide a step-by-step procedure for using DeepLabCut that guides the user in creating a tailored, reusable analysis pipeline with a graphical processing unit (GPU) in 1-12 h (depending on frame size). Additionally, we provide Docker environments and Jupyter Notebooks that can be run on cloud resources such as Google Colaboratory.

Network-specific sex differentiation of intrinsic brain function in males with autism.

Background: The male predominance in the prevalence of autism spectrum disorder (ASD) has motivated research on sex differentiation in ASD. Multiple sources of evidence have suggested a neurophenotypic convergence of ASD-related characteristics and typical sex differences. Two existing, albeit competing, models provide predictions on such neurophenotypic convergence. These two models are testable with neuroimaging. Specifically, the Extreme Male Brain (EMB) model predicts that ASD is associated with enhanced brain maleness in both males and females with ASD (i.e., a shift-towards-maleness). In contrast, the Gender Incoherence (GI) model predicts a shift-towards-maleness in females, yet a shift-towards-femaleness in males with ASD. Methods: To clarify whether either model applies to the intrinsic functional properties of the brain in males with ASD, we measured the statistical overlap between typical sex differences and ASD-related atypicalities in resting-state fMRI (R-fMRI) datasets largely available in males. Main analyses focused on two large-scale R-fMRI samples: 357 neurotypical (NT) males and 471 NT females from the 1000 Functional Connectome Project and 360 males with ASD and 403 NT males from the Autism Brain Imaging Data Exchange. Results: Across all R-fMRI metrics, results revealed coexisting, but network-specific, shift-towards-maleness and shift-towards-femaleness in males with ASD. A shift-towards-maleness mostly involved the default network, while a shift-towards-femaleness mostly occurred in the somatomotor network. Explorations of the associated cognitive processes using available cognitive ontology maps indicated that higher-order social cognitive functions corresponded to the shift-towards-maleness, while lower-order sensory motor processes corresponded to the shift-towards-femaleness. Conclusions: The present findings suggest that atypical intrinsic brain properties in males with ASD partly reflect mechanisms involved in sexual differentiation. A model based on network-dependent atypical sex mosaicism can synthesize prior competing theories on factors involved in sex differentiation in ASD.

Association of White Matter Structure With Autism Spectrum Disorder and Attention-Deficit/Hyperactivity Disorder.

Importance: Clinical overlap between autism spectrum disorder (ASD) and attention-deficit/hyperactivity disorder (ADHD) is increasingly appreciated, but the underlying brain mechanisms remain unknown to date. Objective: To examine associations between white matter organization and 2 commonly co-occurring neurodevelopmental conditions, ASD and ADHD, through both categorical and dimensional approaches. Design, Setting, and Participants: This investigation was a cross-sectional diffusion tensor imaging (DTI) study at an outpatient academic clinical and research center, the Department of Child and Adolescent Psychiatry at New York University Langone Medical Center. Participants were children with ASD, children with ADHD, or typically developing children. Data collection was ongoing from December 2008 to October 2015. Main Outcomes and Measures: The primary measure was voxelwise fractional anisotropy (FA) analyzed via tract-based spatial statistics. Additional voxelwise DTI metrics included radial diffusivity (RD), mean diffusivity (MD), axial diffusivity (AD), and mode of anisotropy (MA). Results: This cross-sectional DTI study analyzed data from 174 children (age range, 6.0-12.9 years), selected from a larger sample after quality assurance to be group matched on age and sex. After quality control, the study analyzed data from 69 children with ASD (mean [SD] age, 8.9 [1.7] years; 62 male), 55 children with ADHD (mean [SD] age, 9.5 [1.5] years; 41 male), and 50 typically developing children (mean [SD] age, 9.4 [1.5] years; 38 male). Categorical analyses revealed a significant influence of ASD diagnosis on several DTI metrics (FA, MD, RD, and AD), primarily in the corpus callosum. For example, FA analyses identified a cluster of 4179 voxels (TFCE FEW corrected P < .05) in posterior portions of the corpus callosum. Dimensional analyses revealed associations between ASD severity and FA, RD, and MD in more extended portions of the corpus callosum and beyond (eg, corona radiata and inferior longitudinal fasciculus) across all individuals, regardless of diagnosis. For example, FA analyses revealed clusters overall encompassing 12121 voxels (TFCE FWE corrected P < .05) with a significant association with parent ratings in the social responsiveness scale. Similar results were evident using an independent measure of ASD traits (ie, children communication checklist, second edition). Total severity of ADHD-traits was not significantly related to DTI metrics but inattention scores were related to AD in corpus callosum in a cluster sized 716 voxels. All these findings were robust to algorithmic correction of motion artifacts with the DTIPrep software. Conclusions and Relevance: Dimensional analyses provided a more complete picture of associations between ASD traits and inattention and indexes of white matter organization, particularly in the corpus callosum. This transdiagnostic approach can reveal dimensional relationships linking white matter structure to neurodevelopmental symptoms.