Machine learning approaches to evaluate infants' general movements in the writhing stage-a pilot study.

The goals of this study are to describe machine learning techniques employing computer-vision movement algorithms to automatically evaluate infants' general movements (GMs) in the writhing stage. This is a retrospective study of infants admitted 07/2019 to 11/2021 to a level IV neonatal intensive care unit (NICU). Infant GMs, classified by certified expert, were analyzed in two-steps (1) determination of anatomic key point location using a NICU-trained pose estimation model [accuracy determined using object key point similarity (OKS)]; (2) development of a preliminary movement model to distinguish normal versus cramped-synchronized (CS) GMs using cosine similarity and autocorrelation of major joints. GMs were analyzed using 85 videos from 74 infants; gestational age at birth 28.9 ± 4.1 weeks and postmenstrual age (PMA) at time of video 35.9 ± 4.6 weeks The NICU-trained pose estimation model was more accurate (0.91 ± 0.008 OKS) than a generic model (0.83 ± 0.032 OKS, p < 0.001). Autocorrelation values in the lower limbs were significantly different between normal (5 videos) and CS GMs (5 videos, p < 0.05). These data indicate that automated pose estimation of anatomical key points is feasible in NICU patients and that a NICU-trained model can distinguish between normal and CS GMs. These preliminary data indicate that machine learning techniques may represent a promising tool for earlier CP risk assessment in the writhing stage and prior to hospital discharge.

Human-airway surface mesh smoothing based on graph convolutional neural networks.

BACKGROUND AND OBJECTIVE: A detailed representation of the airway geometry in the respiratory system is critical for predicting precise airflow and pressure behaviors in computed tomography (CT)-image-based computational fluid dynamics (CFD). The CT-image-based geometry often contains artifacts, noise, and discontinuities due to the so-called stair step effect. Hence, an advanced surface smoothing is necessary. The existing smoothing methods based on the Laplacian operator drastically shrink airway geometries, resulting in the loss of information related to smaller branches. This study aims to introduce an unsupervised airway-mesh-smoothing learning (AMSL) method that preserves the original geometry of the three-dimensional (3D) airway for accurate CT-image-based CFD simulations. METHOD: The AMSL method jointly trains two graph convolutional neural networks (GCNNs) defined on airway meshes to filter vertex positions and face normal vectors. In addition, it regularizes a combination of loss functions such as reproducibility, smoothness and consistency of vertex positions, and normal vectors. The AMSL adopts the concept of a deep mesh prior model, and it determines the self-similarity for mesh restoration without using a large dataset for training. Images of the airways of 20 subjects were smoothed by the AMSL method, and among them, the data of two subjects were used for the CFD simulations to assess the effect of airway smoothing on flow properties. RESULTS: In 18 of 20 benchmark problems, the proposed smoothing method delivered better results compared with the conventional or state-of-the-art deep learning methods. Unlike the traditional smoothing, the AMSL successfully constructed 20 smoothed airways with airway diameters that were consistent with the original CT images. Besides, CFD simulations with the airways obtained by the AMSL method showed much smaller pressure drop and wall shear stress than the results obtained by the traditional method. CONCLUSIONS: The airway model constructed by the AMSL method reproduces branch diameters accurately without any shrinkage, especially in the case of smaller airways. The accurate estimation of airway geometry using a smoothing method is critical for estimating flow properties in CFD simulations.

A Deep Learning Approach for Neuronal Cell Body Segmentation in Neurons Expressing GCaMP Using a Swin Transformer.

Neuronal cell body analysis is crucial for quantifying changes in neuronal sizes under different physiological and pathologic conditions. Neuronal cell body detection and segmentation mainly rely on manual or pseudo-manual annotations. Manual annotation of neuronal boundaries is time-consuming, requires human expertise, and has intra/interobserver variances. Also, determining where the neuron's cell body ends and where the axons and dendrites begin is taxing. We developed a deep-learning-based approach that uses a state-of-the-art shifted windows (Swin) transformer for automated, reproducible, fast, and unbiased 2D detection and segmentation of neuronal somas imaged in mouse acute brain slices by multiphoton microscopy. We tested our Swin algorithm during different experimental conditions of low and high signal fluorescence. Our algorithm achieved a mean Dice score of 0.91, a precision of 0.83, and a recall of 0.86. Compared with two different convolutional neural networks, the Swin transformer outperformed them in detecting the cell boundaries of GCamP6s expressing neurons. Thus, our Swin transform algorithm can assist in the fast and accurate segmentation of fluorescently labeled neuronal cell bodies in thick acute brain slices. Using our flexible algorithm, researchers can better study the fluctuations in neuronal soma size during physiological and pathologic conditions.

PARC: Physics-aware recurrent convolutional neural networks to assimilate meso scale reactive mechanics of energetic materials.

The thermo-mechanical response of shock-initiated energetic materials (EMs) is highly influenced by their microstructures, presenting an opportunity to engineer EM microstructures in a "materials-by-design" framework. However, the current design practice is limited, as a large ensemble of simulations is required to construct the complex EM structure-property-performance linkages. We present the physics-aware recurrent convolutional (PARC) neural network, a deep learning algorithm capable of learning the mesoscale thermo-mechanics of EM from a modest number of high-resolution direct numerical simulations (DNS). Validation results demonstrated that PARC could predict the themo-mechanical response of shocked EMs with comparable accuracy to DNS but with notably less computation time. The physics-awareness of PARC enhances its modeling capabilities and generalizability, especially when challenged in unseen prediction scenarios. We also demonstrate that visualizing the artificial neurons at PARC can shed light on important aspects of EM thermos-mechanics and provide an additional lens for conceptualizing EM.

Predictive power of deep-learning segmentation based prognostication model in non-small cell lung cancer.

Purpose: The study aims to create a model to predict survival outcomes for non-small cell lung cancer (NSCLC) after treatment with stereotactic body radiotherapy (SBRT) using deep-learning segmentation based prognostication (DESEP). Methods: The DESEP model was trained using imaging from 108 patients with NSCLC with various clinical stages and treatment histories. The model generated predictions based on unsupervised features learned by a deep-segmentation network from computed tomography imaging to categorize patients into high and low risk groups for overall survival (DESEP-predicted-OS), disease specific survival (DESEP-predicted-DSS), and local progression free survival (DESEP-predicted-LPFS). Serial assessments were also performed using auto-segmentation based volumetric RECISTv1.1 and computer-based unidimensional RECISTv1.1 patients was performed. Results: There was a concordance between the DESEP-predicted-LPFS risk category and manually calculated RECISTv1.1 (φ=0.544, p=0.001). Neither the auto-segmentation based volumetric RECISTv1.1 nor the computer-based unidimensional RECISTv1.1 correlated with manual RECISTv1.1 (p=0.081 and p=0.144, respectively). While manual RECISTv1.1 correlated with LPFS (HR=6.97,3.51-13.85, c=0.70, p<0.001), it could not provide insight regarding DSS (p=0.942) or OS (p=0.662). In contrast, the DESEP-predicted methods were predictive of LPFS (HR=3.58, 1.66-7.18, c=0.60, p<0.001), OS (HR=6.31, 3.65-10.93, c=0.71, p<0.001) and DSS (HR=9.25, 4.50-19.02, c=0.69, p<0.001). The promising results of the DESEP model were reproduced for the independent, external datasets of Stanford University, classifying survival and 'dead' group in their Kaplan-Meyer curves (p = 0.019). Conclusion: Deep-learning segmentation based prognostication can predict LPFS as well as OS, and DSS after SBRT for NSCLC. It can be used in conjunction with current standard of care, manual RECISTv1.1 to provide additional insights regarding DSS and OS in NSCLC patients receiving SBRT. Summary: While current standard of care, manual RECISTv1.1 correlated with local progression free survival (LPFS) (HR=6.97,3.51-13.85, c=0.70, p<0.001), it could not provide insight regarding disease specific survival (DSS) (p=0.942) or overall survival (OS) (p=0.662). In contrast, the deep-learning segmentation based prognostication (DESEP)-predicted methods were predictive of LPFS (HR=3.58, 1.66-7.18, c=0.60, p<0.001), OS (HR=6.31, 3.65-10.93, c=0.71, p<0.001) and DSS (HR=9.25, 4.50-19.02, c=0.69, p<0.001). DESEP can be used in conjunction with current standard of care, manual RECISTv1.1 to provide additional insights regarding DSS and OS in NSCLC patients.

Acute coronary event (ACE) prediction following breast radiotherapy by features extracted from 3D CT, dose, and cardiac structures.

PURPOSE: Heart toxicity, such as major acute coronary events (ACE), following breast radiation therapy (RT) is of utmost concern. Thus, many studies have been investigating the effect of mean heart dose (MHD) and dose received in heart sub-structures on toxicity. Most studies focused on the dose thresholds in the heart and its sub-structures, while few studies adopted such computational methods as deep neural networks (DNN) and radiomics. This work aims to construct a feature-driven predictive model for ACE after breast RT. METHODS: A recently proposed two-step predictive model that extracts a number of features from a deep auto-segmentation network and processes the selected features for prediction was adopted. This work refined the auto-segmenting network and feature processing algorithms to enhance performance in cardiac toxicity prediction. In the predictive model, the deep convolutional neural network (CNN) extracted features from 3D computed tomography (CT) images and dose distributions in three automatically segmented heart sub-structures, including the left anterior descending artery (LAD), right coronary artery (RCA), and left ventricle (LV). The optimal feature processing workflow for the extracted features was explored to enhance the prediction accuracy. The regions associated with toxicity were visualized using a class activation map (CAM)-based technique. Our proposed model was validated against a conventional DNN (convolutional and fully connected layers) and radiomics with a patient cohort of 84 cases, including 29 and 55 patient cases with and without ACE. Of the entire 84 cases, 12 randomly chosen cases (5 toxicity and 7 non-toxicity cases) were set aside for independent test, and the remaining 72 cases were applied to 4-fold stratified cross-validation. RESULTS: Our predictive model outperformed the conventional DNN by 38% and 10% and radiomics-based predictive models by 9% and 10% in AUC for 4-fold cross-validations and independent test, respectively. The degree of enhancement was greater when incorporating dose information and heart sub-structures into feature extraction. The model whose inputs were CT, dose, and three sub-structures (LV, LAD, and RCA) reached 96% prediction accuracy on average and 0.94 area under the curve (AUC) on average in the cross-validation, and also achieved prediction accuracy of 83% and AUC of 0.83 in the independent test. On 10 correctly predicted cases out of 12 for the independent test, the activation maps implied that for cases of ACE toxicity, the higher intensity was more likely to be observed inside the LV. CONCLUSIONS: The proposed model characterized by modifications in model input with dose distributions and cardiac sub-structures, and serial processing of feature extraction and feature selection techniques can improve the predictive performance in ACE following breast RT.

Role of NKCC1 and KCC2 during hypoxia-induced neuronal swelling in the neonatal neocortex.

Neonatal hypoxia causes cytotoxic neuronal swelling by the entry of ions and water. Multiple water pathways have been implicated in neurons because these cells lack water channels, and their membrane has a low water permeability. NKCC1 and KCC2 are cation-chloride cotransporters (CCCs) involved in water movement in various cell types. However, the role of CCCs in water movement in neonatal neurons during hypoxia is unknown. We studied the effects of modulating CCCs pharmacologically on neuronal swelling in the neocortex (layer IV/V) of neonatal mice (post-natal day 8-13) during prolonged and brief hypoxia. We used acute brain slices from Clomeleon mice which express a ratiometric fluorophore sensitive to Cl- and exposed them to oxygen-glucose deprivation (OGD) while imaging neuronal size and [Cl-]i by multiphoton microscopy. Neurons were identified using a convolutional neural network algorithm, and changes in the somatic area and [Cl-]i were evaluated using a linear mixed model for repeated measures. We found that (1) neuronal swelling and Cl- accumulation began after OGD, worsened during 20 min of OGD, or returned to baseline during reoxygenation if the exposure to OGD was brief (10 min). (2) Neuronal swelling did not occur when the extracellular Cl- concentration was low. (3) Enhancing KCC2 activity did not alter OGD-induced neuronal swelling but prevented Cl- accumulation; (4) blocking KCC2 led to an increase in Cl- accumulation during prolonged OGD and aggravated neuronal swelling during reoxygenation; (5) blocking NKCC1 reduced neuronal swelling during early but not prolonged OGD and aggravated Cl- accumulation during prolonged OGD; and (6) treatment with the "broad" CCC blocker furosemide reduced both swelling and Cl- accumulation during prolonged and brief OGD, whereas simultaneous NKCC1 and KCC2 inhibition using specific pharmacological blockers aggravated neuronal swelling during prolonged OGD. We conclude that CCCs, and other non-CCCs, contribute to water movement in neocortical neurons during OGD in the neonatal period.

Synthesizing controlled microstructures of porous media using generative adversarial networks and reinforcement learning.

For material modeling and discovery, synthetic microstructures play a critical role as digital twins. They provide stochastic samples upon which direct numerical simulations can be conducted to populate material databases. A large ensemble of simulation data on synthetic microstructures may provide supplemental data to inform and refine macroscopic material models, which might not be feasible from physical experiments alone. However, synthesizing realistic microstructures with realistic microstructural attributes is highly challenging. Thus, it is often oversimplified via rough approximations that may yield an inaccurate representation of the physical world. Here, we propose a novel deep learning method that can synthesize realistic three-dimensional microstructures with controlled structural properties using the combination of generative adversarial networks (GAN) and actor-critic (AC) reinforcement learning. The GAN-AC combination enables the generation of microstructures that not only resemble the appearances of real specimens but also yield user-defined physical quantities of interest (QoI). Our validation experiments confirm that the properties of synthetic microstructures generated by the GAN-AC framework are within a 5% error margin with respect to the target values. The scientific contribution of this paper resides in the novel design of the GAN-AC microstructure generator and the mathematical and algorithmic foundations therein. The proposed method will have a broad and substantive impact on the materials community by providing lenses for analyzing structure-property-performance linkages and for implementing the notion of 'materials-by-design'.

Body shape matters: Evidence from machine learning on body shape-income relationship.

The association between physical appearance and income has been of central interest in social science. However, most previous studies often measured physical appearance using classical proxies from subjective opinions based on surveys. In this study, we use novel data, called CAESAR, which contains three-dimensional (3D) whole-body scans to mitigate possible reporting and measurement errors. We demonstrate the existence of significant nonclassical reporting errors in the reported heights and weights by comparing them with measured counterparts, and show that these discrete measurements are too sparse to provide a complete description of the body shape. Instead, we use a graphical autoencoder to obtain intrinsic features, consisting of human body shapes directly from 3D scans and estimate the relationship between body shapes and family income. We also take into account a possible issue of endogenous body shapes using proxy variables and control functions. The estimation results reveal a statistically significant relationship between physical appearance and family income and that these associations differ across genders. This supports the hypothesis on the physical attractiveness premium in labor market outcomes and its heterogeneity across genders.

Machine Learning Prediction of Fall Risk in Older Adults Using Timed Up and Go Test Kinematics.

Falls among the elderly population cause detrimental physical, mental, financial problems and, in the worst case, death. The increasing number of people entering the higher risk age-range has increased clinicians' attention to intervene. Clinical tools, e.g., the Timed Up and Go (TUG) test, have been created for aiding clinicians in fall-risk assessment. Often simple to evaluate, these assessments are subject to a clinician's judgment. Wearable sensor data with machine learning algorithms were introduced as an alternative to precisely quantify ambulatory kinematics and predict prospective falls. However, they require a long-term evaluation of large samples of subjects' locomotion and complex feature engineering of sensor kinematics. Therefore, it is critical to build an objective fall-risk detection model that can efficiently measure biometric risk factors with minimal costs. We built and studied a sensor data-driven convolutional neural network model to predict older adults' fall-risk status with relatively high sensitivity to geriatrician's expert assessment. The sample in this study is representative of older patients with multiple co-morbidity seen in daily medical practice. Three non-intrusive wearable sensors were used to measure participants' gait kinematics during the TUG test. This data collection ensured convenient capture of various gait impairment aspects at different body locations.

ANMAF: an automated neuronal morphology analysis framework using convolutional neural networks.

Measurement of neuronal size is challenging due to their complex histology. Current practice includes manual or pseudo-manual measurement of somatic areas, which is labor-intensive and prone to human biases and intra-/inter-observer variances. We developed a novel high-throughput neuronal morphology analysis framework (ANMAF), using convolutional neural networks (CNN) to automatically contour the somatic area of fluorescent neurons in acute brain slices. Our results demonstrate considerable agreements between human annotators and ANMAF on detection, segmentation, and the area of somatic regions in neurons expressing a genetically encoded fluorophore. However, in contrast to humans, who exhibited significant variability in repeated measurements, ANMAF produced consistent neuronal contours. ANMAF was generalizable across different imaging protocols and trainable even with a small number of humanly labeled neurons. Our framework can facilitate more rigorous and quantitative studies of neuronal morphology by enabling the segmentation of many fluorescent neurons in thick brain slices in a standardized manner.

Deep learning for synthetic microstructure generation in a materials-by-design framework for heterogeneous energetic materials.

The sensitivity of heterogeneous energetic (HE) materials (propellants, explosives, and pyrotechnics) is critically dependent on their microstructure. Initiation of chemical reactions occurs at hot spots due to energy localization at sites of porosities and other defects. Emerging multi-scale predictive models of HE response to loads account for the physics at the meso-scale, i.e. at the scale of statistically representative clusters of particles and other features in the microstructure. Meso-scale physics is infused in machine-learned closure models informed by resolved meso-scale simulations. Since microstructures are stochastic, ensembles of meso-scale simulations are required to quantify hot spot ignition and growth and to develop models for microstructure-dependent energy deposition rates. We propose utilizing generative adversarial networks (GAN) to spawn ensembles of synthetic heterogeneous energetic material microstructures. The method generates qualitatively and quantitatively realistic microstructures by learning from images of HE microstructures. We show that the proposed GAN method also permits the generation of new morphologies, where the porosity distribution can be controlled and spatially manipulated. Such control paves the way for the design of novel microstructures to engineer HE materials for targeted performance in a materials-by-design framework.

Deep segmentation networks predict survival of non-small cell lung cancer.

Non-small-cell lung cancer (NSCLC) represents approximately 80-85% of lung cancer diagnoses and is the leading cause of cancer-related death worldwide. Recent studies indicate that image-based radiomics features from positron emission tomography/computed tomography (PET/CT) images have predictive power for NSCLC outcomes. To this end, easily calculated functional features such as the maximum and the mean of standard uptake value (SUV) and total lesion glycolysis (TLG) are most commonly used for NSCLC prognostication, but their prognostic value remains controversial. Meanwhile, convolutional neural networks (CNN) are rapidly emerging as a new method for cancer image analysis, with significantly enhanced predictive power compared to hand-crafted radiomics features. Here we show that CNNs trained to perform the tumor segmentation task, with no other information than physician contours, identify a rich set of survival-related image features with remarkable prognostic value. In a retrospective study on pre-treatment PET-CT images of 96 NSCLC patients before stereotactic-body radiotherapy (SBRT), we found that the CNN segmentation algorithm (U-Net) trained for tumor segmentation in PET and CT images, contained features having strong correlation with 2- and 5-year overall and disease-specific survivals. The U-Net algorithm has not seen any other clinical information (e.g. survival, age, smoking history, etc.) than the images and the corresponding tumor contours provided by physicians. In addition, we observed the same trend by validating the U-Net features against an extramural data set provided by Stanford Cancer Institute. Furthermore, through visualization of the U-Net, we also found convincing evidence that the regions of metastasis and recurrence appear to match with the regions where the U-Net features identified patterns that predicted higher likelihoods of death. We anticipate our findings will be a starting point for more sophisticated non-intrusive patient specific cancer prognosis determination. For example, the deep learned PET/CT features can not only predict survival but also visualize high-risk regions within or adjacent to the primary tumor and hence potentially impact therapeutic outcomes by optimal selection of therapeutic strategy or first-line therapy adjustment.

Machine learning-aided exploration of relationship between strength and elastic properties in ascending thoracic aneurysm.

Machine learning was applied to classify tension-strain curves harvested from inflation tests on ascending thoracic aneurysm samples. The curves were classified into rupture and nonrupture groups using prerupture response features. Two groups of features were used as the basis for classification. The first was the constitutive parameters fitted from the tension-strain data, and the second was geometric parameters extracted from the tension-strain curve. Based on the importance scores provided by the machine learning, implications of some features were interrogated. It was found that (1) the value of a constitutive parameter is nearly the same for all members in the rupture group and (2) the strength correlates strongly with a tension in the early phase of response as well as with the end stiffness. The study suggests that the strength, which is not available without rupturing the tissue, may be indirectly inferred from prerupture response features.