The training process of many deep networks explores the same low-dimensional manifold.

We develop information-geometric techniques to analyze the trajectories of the predictions of deep networks during training. By examining the underlying high-dimensional probabilistic models, we reveal that the training process explores an effectively low-dimensional manifold. Networks with a wide range of architectures, sizes, trained using different optimization methods, regularization techniques, data augmentation techniques, and weight initializations lie on the same manifold in the prediction space. We study the details of this manifold to find that networks with different architectures follow distinguishable trajectories, but other factors have a minimal influence; larger networks train along a similar manifold as that of smaller networks, just faster; and networks initialized at very different parts of the prediction space converge to the solution along a similar manifold.

Bias in machine learning models can be significantly mitigated by careful training: Evidence from neuroimaging studies.

Despite the great promise that machine learning has offered in many fields of medicine, it has also raised concerns about potential biases and poor generalization across genders, age distributions, races and ethnicities, hospitals, and data acquisition equipment and protocols. In the current study, and in the context of three brain diseases, we provide evidence which suggests that when properly trained, machine learning models can generalize well across diverse conditions and do not necessarily suffer from bias. Specifically, by using multistudy magnetic resonance imaging consortia for diagnosing Alzheimer's disease, schizophrenia, and autism spectrum disorder, we find that well-trained models have a high area-under-the-curve (AUC) on subjects across different subgroups pertaining to attributes such as gender, age, racial groups and different clinical studies and are unbiased under multiple fairness metrics such as demographic parity difference, equalized odds difference, equal opportunity difference, etc. We find that models that incorporate multisource data from demographic, clinical, genetic factors, and cognitive scores are also unbiased. These models have a better predictive AUC across subgroups than those trained only with imaging features, but there are also situations when these additional features do not help.

A Geometric Interpretation of Stochastic Gradient Descent Using Diffusion Metrics.

This paper is a step towards developing a geometric understanding of a popular algorithm for training deep neural networks named stochastic gradient descent (SGD). We built upon a recent result which observed that the noise in SGD while training typical networks is highly non-isotropic. That motivated a deterministic model in which the trajectories of our dynamical systems are described via geodesics of a family of metrics arising from a certain diffusion matrix; namely, the covariance of the stochastic gradients in SGD. Our model is analogous to models in general relativity: the role of the electromagnetic field in the latter is played by the gradient of the loss function of a deep network in the former.

Aberrant expression of vimentin predisposes oral premalignant lesion derived cells towards transformation.

OBJECTIVE: We have previously reported the aberrant expression of vimentin in human oral premalignant lesions and a 4-Nitroquinoline 1-oxide (4NQO) model of rat lingual carcinogenesis. Hence, we wanted to understand whether the expression of vimentin in early stage contributes to the process of transformation. STUDY DESIGN: Vimentin was stably expressed in oral premalignant lesion derived cells (vimentin negative) and various transformation related phenotypic assays were performed. Since vimentin alone failed to transform the cells, an additional carcinogenic stimulus benzo[a]pyrene (BP) was used. Concomitantly, immunohistochemistry (IHC) was performed on oral leukoplakia and tumor tissues for studying the expression of vimentin and E-cadherin. RESULTS: Exogenous expression of vimentin led to the appearance of EMT and stemness-related signatures. Further, upon BP treatment, vimentin expressing clones showed an increase in vitro and in vivo transformation efficiency. Importantly, high vimentin-low E-cadherin expression significantly correlated with the grade of dysplasia, as also with the lymph node metastasis in oral tumors. CONCLUSION: Our study suggests that the expression of vimentin in early stages may be beneficial, although not sufficient to achieve transformation. Further, high vimentin-low E-cadherin expression, if validated in more number of early oral lesions, may prove useful in the identification of high risk human premalignant lesions.

Hemidesmosomal linker proteins regulate cell motility, invasion and tumorigenicity in oral squamous cell carcinoma derived cells.

BPAG1e and Plectin are hemidesmosomal linker proteins which anchor intermediate filament proteins to the cell surface through ß4 integrin. Recent reports indicate that these proteins play a role in various cellular processes apart from their known anchoring function. However, the available literature is inconsistent. Further, the previous study from our laboratory suggested that Keratin8/18 pair promotes cell motility and tumor progression by deregulating ß4 integrin signaling in oral squamous cell carcinoma (OSCC) derived cells. Based on these findings, we hypothesized that linker proteins may have a role in neoplastic progression of OSCC. Downregulation of hemidesmosomal linker proteins in OSCC derived cells resulted in reduced cell migration accompanied by alterations in actin organization. Further, decreased MMP9 activity led to reduced cell invasion in linker proteins knockdown cells. Moreover, loss of these proteins resulted in reduced tumorigenic potential. SWATH analysis demonstrated upregulation of N-Myc downstream regulated gene 1 (NDRG1) in linker proteins downregulated cells as compared to vector control cells. Further, the defects in phenotype upon linker proteins ablation were rescued upon loss of NDRG1 in linker proteins knockdown background. These data together indicate that hemidesmosomal linker proteins regulate cell motility, invasion and tumorigenicity possibly through NDRG1 in OSCC derived cells.

Investigating Causal Genetic Effects on Overall Survival of Glioblastoma Patients using Normalizing Flow and Structural Causal Model.

Glioblastoma (GBM) is the most common and aggressive brain tumor with short overall survival (OS) of about 15 months. Understanding the causal factors affecting the patient survival is crucial for disease prognosis and treatment planning. Although previous efforts on survival prediction using multi-omics data has yielded useful predictive models, the causation of the correlated genetic risk factors has not been addressed. Recent advances in causal deep learning models enable the study of causality from complex dataset. In this paper, we leverage the recently proposed structural causal model (SCM) with normalizing flows parameterized by deep networks to perform the counterfactual query to investigate the causal relationship between gene mutation and OS with the presence of other confounders including sex, age and radiomics features. The query amounts to the question that what the survival days will be if the gene mutation status has been changed, i.e., from mutant to non-mutant and vice versa. The trained causal model will infer the counterfactual outcome given the intervention on specific gene mutation. We apply multivariate Cox-PH model to find the genes associated with survival, and investigate the causal genetic effect by comparing the original and counterfactual survival days in a bi-directional fashion. Particularly, the following two scenarios are considered: (1) intervention on a specific gene with non-mutant status to generate the counterfactual survival days as if the gene is mutant, with which the original survival days of the subjects with that mutant gene will be compared; (2) intervention on the gene with mutant status and perform the comparison with survival days of subjects with that non-mutant gene. Our experimental results show that no causation of two correlated genes (NF1, RB1) was revealed in the cohort (n=181), while their genetic effects on OS in terms of prolonging or shortening are generally in accordance with clinical findings.

Pre and post-operative serum levels of titanium cobalt and aluminium after implant placement.

The preoperative serum levels and postoperative serum levels of titanium, cobalt and aluminium from dental implants in order to assess the release of these ions and to assess any risk of toxicity from these ions after dental implant placement is of interest to dentists. It was observed that there was very slight increase in serum concentration of titanium, cobalt and aluminium after 12 months of placement of implants as compared to before placement of implants. However the increase was non-significant statistically. Our study concluded that the use of dental implants does not pose any risk of toxicity of metal ions like titanium, aluminium and cobalt because of very slight non-significant increase in serum levels of these ions 12 months after implant placement.

NeuroSynth: MRI-Derived Neuroanatomical Generative Models and Associated Dataset of 18,000 Samples.

Availability of large and diverse medical datasets is often challenged by privacy and data sharing restrictions. For successful application of machine learning techniques for disease diagnosis, prognosis, and precision medicine, large amounts of data are necessary for model building and optimization. To help overcome such limitations in the context of brain MRI, we present NeuroSynth: a collection of generative models of normative regional volumetric features derived from structural brain imaging. NeuroSynth models are trained on real brain imaging regional volumetric measures from the iSTAGING consortium, which encompasses over 40,000 MRI scans across 13 studies, incorporating covariates such as age, sex, and race. Leveraging NeuroSynth, we produce and offer 18,000 synthetic samples spanning the adult lifespan (ages 22-90 years), alongside the model's capability to generate unlimited data. Experimental results indicate that samples generated from NeuroSynth agree with the distributions obtained from real data. Most importantly, the generated normative data significantly enhance the accuracy of downstream machine learning models on tasks such as disease classification. Data and models are available at: https://huggingface.co/spaces/rongguangw/neuro-synth.

How Occam's razor guides human decision-making.

Occam's razor is the principle that, all else being equal, simpler explanations should be preferred over more complex ones. This principle is thought to play a role in human perception and decision-making, but the nature of our presumed preference for simplicity is not understood. Here we use preregistered behavioral experiments informed by formal theories of statistical model selection to show that, when faced with uncertain evidence, human subjects exhibit preferences for particular, theoretically grounded forms of simplicity of the alternative explanations. These forms of simplicity can be understood in terms of geometrical features of statistical models treated as manifolds in the space of the probability distributions, in particular their dimensionality, boundaries, volume, and curvature. The simplicity preferences driven by these features, which are also exhibited by artificial neural networks trained to optimize performance on comparable tasks, generally improve decision accuracy, because they minimize over-sensitivity to noisy observations (i.e., overfitting). However, unlike for artificial networks, for human subjects these preferences persist even when they are maladaptive with respect to the task training and instructions. Thus, these preferences are not simply transient optimizations for particular task conditions but rather a more general feature of human decision-making. Taken together, our results imply that principled notions of statistical model complexity have direct, quantitative relevance to human and machine decision-making and establish a new understanding of the computational foundations, and behavioral benefits, of our predilection for inferring simplicity in the latent properties of our complex world.

Time as a supervisor: temporal regularity and auditory object learning.

Sensory systems appear to learn to transform incoming sensory information into perceptual representations, or "objects," that can inform and guide behavior with minimal explicit supervision. Here, we propose that the auditory system can achieve this goal by using time as a supervisor, i.e., by learning features of a stimulus that are temporally regular. We will show that this procedure generates a feature space sufficient to support fundamental computations of auditory perception. In detail, we consider the problem of discriminating between instances of a prototypical class of natural auditory objects, i.e., rhesus macaque vocalizations. We test discrimination in two ethologically relevant tasks: discrimination in a cluttered acoustic background and generalization to discriminate between novel exemplars. We show that an algorithm that learns these temporally regular features affords better or equivalent discrimination and generalization than conventional feature-selection algorithms, i.e., principal component analysis and independent component analysis. Our findings suggest that the slow temporal features of auditory stimuli may be sufficient for parsing auditory scenes and that the auditory brain could utilize these slowly changing temporal features.

Early Adversity and Accelerated Brain Aging: A Mini-Review.

Early adversity is an important risk factor that influences brain aging. Diverse animal models of early adversity, including gestational stress and postnatal paradigms disrupting dam-pup interactions evoke not only persistent neuroendocrine dysfunction and anxio-depressive behaviors, but also perturb the trajectory of healthy brain aging. The process of brain aging is thought to involve hallmark features such as mitochondrial dysfunction and oxidative stress, evoking impairments in neuronal bioenergetics. Furthermore, brain aging is associated with disrupted proteostasis, progressively defective epigenetic and DNA repair mechanisms, the build-up of neuroinflammatory states, thus cumulatively driving cellular senescence, neuronal and cognitive decline. Early adversity is hypothesized to evoke an "allostatic load" via an influence on several of the key physiological processes that define the trajectory of healthy brain aging. In this review we discuss the evidence that animal models of early adversity impinge on fundamental mechanisms of brain aging, setting up a substratum that can accelerate and compromise the time-line and nature of brain aging, and increase risk for aging-associated neuropathologies.

Embracing the disharmony in medical imaging: A Simple and effective framework for domain adaptation.

Domain shift, the mismatch between training and testing data characteristics, causes significant degradation in the predictive performance in multi-source imaging scenarios. In medical imaging, the heterogeneity of population, scanners and acquisition protocols at different sites presents a significant domain shift challenge and has limited the widespread clinical adoption of machine learning models. Harmonization methods, which aim to learn a representation of data invariant to these differences are the prevalent tools to address domain shift, but they typically result in degradation of predictive accuracy. This paper takes a different perspective of the problem: we embrace this disharmony in data and design a simple but effective framework for tackling domain shift. The key idea, based on our theoretical arguments, is to build a pretrained classifier on the source data and adapt this model to new data. The classifier can be fine-tuned for intra-study domain adaptation. We can also tackle situations where we do not have access to ground-truth labels on target data; we show how one can use auxiliary tasks for adaptation; these tasks employ covariates such as age, gender and race which are easy to obtain but nevertheless correlated to the main task. We demonstrate substantial improvements in both intra-study domain adaptation and inter-study domain generalization on large-scale real-world 3D brain MRI datasets for classifying Alzheimer's disease and schizophrenia.

Chronic hM4Di-DREADD-Mediated Chemogenetic Inhibition of Forebrain Excitatory Neurons in Postnatal or Juvenile Life Does Not Alter Adult Mood-Related Behavior.

G-protein-coupled receptors (GPCRs) coupled to Gi signaling, in particular downstream of monoaminergic neurotransmission, are posited to play a key role during developmental epochs (postnatal and juvenile) in shaping the emergence of adult anxiodepressive behaviors and sensorimotor gating. To address the role of Gi signaling in these developmental windows, we used a CaMKIIα-tTA::TRE hM4Di bigenic mouse line to express the hM4Di-DREADD (designer receptor exclusively activated by designer drugs) in forebrain excitatory neurons and enhanced Gi signaling via chronic administration of the DREADD agonist, clozapine-N-oxide (CNO) in the postnatal window (postnatal days 2-14) or the juvenile window (postnatal days 28-40). We confirmed that the expression of the HA-tagged hM4Di-DREADD was restricted to CaMKIIα-positive neurons in the forebrain, and that the administration of CNO in postnatal or juvenile windows evoked inhibition in forebrain circuits of the hippocampus and cortex, as indicated by a decline in expression of the neuronal activity marker c-Fos. hM4Di-DREADD-mediated inhibition of CaMKIIα-positive forebrain excitatory neurons in postnatal or juvenile life did not impact the weight profile of mouse pups, and also did not influence the normal ontogeny of sensory reflexes. Further, postnatal or juvenile hM4Di-DREADD-mediated inhibition of CaMKIIα-positive forebrain excitatory neurons did not alter anxiety- or despair-like behaviors in adulthood and did not impact sensorimotor gating. Collectively, these results indicate that chemogenetic induction of Gi signaling in CaMKIIα-positive forebrain excitatory neurons in postnatal and juvenile temporal windows does not appear to impinge on the programming of anxiodepressive behaviors in adulthood.

Acute immobilization stress evokes sexually dimorphic peripheral and hippocampal neuroimmune responses in adult rats.

Stress perception and response vary across sexes and may contribute to the sex differences in susceptibility to psychopathology. Stress also engages the immune system and baseline immune system markers are known to be sexually dimorphic. Here, we investigated if the neuroimmune consequences following a single episode of acute immobilization stress (AIS) are sexually dimorphic in male and female Sprague-Dawley rats. We analyzed immune parameters in the periphery, and markers of neuroinflammation in the hippocampus, a key target of stress effects in the brain. We observed sexual dimorphism in the pattern of regulation of peripheral cytokines following stress, with males showing a significant increase in the levels of specific cytokines compared to females. Hippocampal cytokine and neuroinflammation-associated gene expression level analysis did not reveal any sexually dimorphic effects of AIS. However, we noted lower baseline expression levels for specific cytokines and many of the genes analyzed in the hippocampus of control females compared to control males. Finally, we assessed the levels of components of the NLRP3 inflammasome in the hippocampus and observed increased NLRP3 protein levels at baseline in females. We further noted that while males showed an increase in NLRP3 levels following AIS, females failed to show a similar change. Together, our results highlight a sexual dimorphism in neuroimmune consequences following AIS, both in the periphery and within the hippocampus, with males displaying robust proinflammatory changes and similar changes not observed in females. Our study underlines the importance of investigating the effect of sex on neuroimmune consequences following acute stress.

Harmonization with Flow-Based Causal Inference.

Heterogeneity in medical data, e.g., from data collected at different sites and with different protocols in a clinical study, is a fundamental hurdle for accurate prediction using machine learning models, as such models often fail to generalize well. This paper leverages a recently proposed normalizing-flow-based method to perform counterfactual inference upon a structural causal model (SCM), in order to achieve harmonization of such data. A causal model is used to model observed effects (brain magnetic resonance imaging data) that result from known confounders (site, gender and age) and exogenous noise variables. Our formulation exploits the bijection induced by flow for the purpose of harmonization. We infer the posterior of exogenous variables, intervene on observations, and draw samples from the resultant SCM to obtain counterfactuals. This approach is evaluated extensively on multiple, large, real-world medical datasets and displayed better cross-domain generalization compared to state-of-the-art algorithms. Further experiments that evaluate the quality of confounder-independent data generated by our model using regression and classification tasks are provided.

Evaluation of Machine Learning Models for Classifying Upper Extremity Exercises Using Inertial Measurement Unit-Based Kinematic Data.

The amount of home-based exercise prescribed by a physical therapist is difficult to monitor. However, the integration of wearable inertial measurement unit (IMU) devices can aid in monitoring home exercise by analyzing exercise biomechanics. The objective of this study is to evaluate machine learning models for classifying nine different upper extremity exercises, based upon kinematic data captured from an IMU-based device. Fifty participants performed one compound and eight isolation exercises with their right arm. Each exercise was performed ten times for a total of 4500 trials. Joint angles were calculated using IMUs that were placed on the hand, forearm, upper arm, and torso. Various machine learning models were developed with different algorithms and train-test splits. Random forest models with flattened kinematic data as a feature had the greatest accuracy (98.6%). Using triaxial joint range of motion as the feature set resulted in decreased accuracy (91.9%) with faster speeds. Accuracy did not decrease below 90% until training size was decreased to 5% from 50%. Accuracy decreased (88.7%) when splitting data by participant. Upper extremity exercises can be classified accurately using kinematic data from a wearable IMU device. A random forest classification model was developed that quickly and accurately classified exercises. Sampling frequency and lower training splits had a modest effect on performance. When the data were split by subject stratification, larger training sizes were required for acceptable algorithm performance. These findings set the basis for more objective and accurate measurements of home-based exercise using emerging healthcare technologies.

Chronic postnatal chemogenetic activation of forebrain excitatory neurons evokes persistent changes in mood behavior.

Early adversity is a risk factor for the development of adult psychopathology. Common across multiple rodent models of early adversity is increased signaling via forebrain Gq-coupled neurotransmitter receptors. We addressed whether enhanced Gq-mediated signaling in forebrain excitatory neurons during postnatal life can evoke persistent mood-related behavioral changes. Excitatory hM3Dq DREADD-mediated chemogenetic activation of forebrain excitatory neurons during postnatal life (P2-14), but not in juvenile or adult windows, increased anxiety-, despair-, and schizophrenia-like behavior in adulthood. This was accompanied by an enhanced metabolic rate of cortical and hippocampal glutamatergic and GABAergic neurons. Furthermore, we observed reduced activity and plasticity-associated marker expression, and perturbed excitatory/inhibitory currents in the hippocampus. These results indicate that Gq-signaling-mediated activation of forebrain excitatory neurons during the critical postnatal window is sufficient to program altered mood-related behavior, as well as functional changes in forebrain glutamate and GABA systems, recapitulating aspects of the consequences of early adversity.

Stress and adversity in early childhood can have long-lasting effects, predisposing people to mental illness and mood disorders in adult life. The weeks immediately before and after birth are critical for establishing key networks of neurons in the brain. Therefore, any disruption to these neural circuits during this time can be detrimental to emotional development. However, it is still unclear which cellular mechanisms cause these lasting changes in behavior. Studies in animals suggest that these long-term effects could result from abnormalities in a few signaling pathways in the brain. For example, it has been proposed that overstimulating the cells that activate circuits in the forebrain ­ also known as excitatory neurons ­ may contribute to the behavioral changes that persist into adulthood. To test this theory, Pati et al. used genetic engineering to modulate a signaling pathway in male mice, which is known to stimulate excitatory neurons in the forebrain. The experiments showed that prolonged activation of excitatory neurons in the first two weeks after birth resulted in anxious and despair-like behaviors as the animals aged. The mice also displayed discrepancies in how they responded to certain external sensory information, which is a hallmark of schizophrenia-like behavior. However, engineering the same changes in adolescent and adult mice had no effect on their mood-related behaviors. This animal study reinforces just how critical the first few weeks of life are for optimal brain development. It provides an insight into a possible mechanism of how disruption during this time could alter emotional behavior. The findings are also relevant to psychiatrists interested in the underlying causes of mental illness after early childhood adversity.

Effective management of focal reactive gingival overgrowths by diode laser: A review and report of two cases.

BACKGROUND: Focal reactive gingival overgrowths (FRGO) are a common observation in a clinical dental practice that may occur in response to external and internal chronic stimuli in form of fibrous connective tissue lesions in the oral mucosa. Gingiva is the most commonly involved site of oral reactive lesions. For the confirmed diagnosis of FRGO not only clinical, but the histopathological presentation of the lesion plays a vital role. Various surgical treatment modalities like a scalpel, cryosurgery, electrosurgery, and lasers have been applied in the management of FRGO. The laser is new treatment modality being employed for treatment of FRGO. CASE REPORT: The purpose of this paper is to attempt short review on FRGO with the management of FRGO using diode laser. Here, we present effective management of peripheral giant cell granuloma and peripheral ossifying fibroma using diode laser. The follow-up of 01year showed no recurrence in both the cases. CONCLUSIONS: Diode soft tissue laser has added advantages like a bloodless surgical field, reduced bacteremia, minimal intra and postoperative discomfort over conventional modalities. Thus it is highly effective in the surgical management of FRGO.