Hebbian dreaming for small datasets.

The dreaming Hopfield model constitutes a generalization of the Hebbian paradigm for neural networks, that is able to perform on-line learning when "awake" and also to account for off-line "sleeping" mechanisms. The latter have been shown to enhance storing in such a way that, in the long sleep-time limit, this model can reach the maximal storage capacity achievable by networks equipped with symmetric pairwise interactions. In this paper, we inspect the minimal amount of information that must be supplied to such a network to guarantee a successful generalization, and we test it both on random synthetic and on standard structured datasets (i.e., MNIST, Fashion-MNIST and Olivetti). By comparing these minimal thresholds of information with those required by the standard (i.e., always "awake") Hopfield model, we prove that the present network can save up to ∼90% of the dataset size, yet preserving the same performance of the standard counterpart. This suggests that sleep may play a pivotal role in explaining the gap between the large volumes of data required to train artificial neural networks and the relatively small volumes needed by their biological counterparts. Further, we prove that the model Cost function (typically used in statistical mechanics) admits a representation in terms of a standard Loss function (typically used in machine learning) and this allows us to analyze its emergent computational skills both theoretically and computationally: a quantitative picture of its capabilities as a function of its control parameters is achieved and consistency between the two approaches is highlighted. The resulting network is an associative memory for pattern recognition tasks that learns from examples on-line, generalizes correctly (in suitable regions of its control parameters) and optimizes its storage capacity by off-line sleeping: such a reduction of the training cost can be inspiring toward sustainable AI and in situations where data are relatively sparse.

From Pavlov Conditioning to Hebb Learning.

Hebb's learning traces its origin in Pavlov's classical conditioning; however, while the former has been extensively modeled in the past decades (e.g., by the Hopfield model and countless variations on theme), as for the latter, modeling has remained largely unaddressed so far. Furthermore, a mathematical bridge connecting these two pillars is totally lacking. The main difficulty toward this goal lies in the intrinsically different scales of the information involved: Pavlov's theory is about correlations between concepts that are (dynamically) stored in the synaptic matrix as exemplified by the celebrated experiment starring a dog and a ringing bell; conversely, Hebb's theory is about correlations between pairs of neurons as summarized by the famous statement that neurons that fire together wire together. In this letter, we rely on stochastic process theory to prove that as long as we keep neurons' and synapses' timescales largely split, Pavlov's mechanism spontaneously takes place and ultimately gives rise to synaptic weights that recover the Hebbian kernel.

Quantifying heterogeneity to drug response in cancer-stroma kinetics.

A crucial challenge in medicine is choosing which drug (or combination) will be the most advantageous for a particular patient. Usually, drug response rates differ substantially, and the reasons for this response unpredictability remain ambiguous. Consequently, it is central to classify features that contribute to the observed drug response variability. Pancreatic cancer is one of the deadliest cancers with limited therapeutic achievements due to the massive presence of stroma that generates an environment that enables tumor growth, metastasis, and drug resistance. To understand the cancer-stroma cross talk within the tumor microenvironment and to develop personalized adjuvant therapies, there is a necessity for effective approaches that offer measurable data to monitor the effect of drugs at the single-cell level. Here, we develop a computational approach, based on cell imaging, that quantifies the cellular cross talk between pancreatic tumor cells (L3.6pl or AsPC1) and pancreatic stellate cells (PSCs), coordinating their kinetics in presence of the chemotherapeutic agent gemcitabine. We report significant heterogeneity in the organization of cellular interactions in response to the drug. For L3.6pl cells, gemcitabine sensibly decreases stroma-stroma interactions but increases stroma-cancer interactions, overall enhancing motility and crowding. In the AsPC1 case, gemcitabine promotes the interactions among tumor cells, but it does not affect stroma-cancer interplay, possibly suggesting a milder effect of the drug on cell dynamics.

Probing Single-Cell Fermentation Fluxes and Exchange Networks via pH-Sensing Hybrid Nanofibers.

The homeostatic control of their environment is an essential task of living cells. It has been hypothesized that, when microenvironmental pH inhomogeneities are induced by high cellular metabolic activity, diffusing protons act as signaling molecules, driving the establishment of exchange networks sustained by the cell-to-cell shuttling of overflow products such as lactate. Despite their fundamental role, the extent and dynamics of such networks is largely unknown due to the lack of methods in single-cell flux analysis. In this study, we provide direct experimental characterization of such exchange networks. We devise a method to quantify single-cell fermentation fluxes over time by integrating high-resolution pH microenvironment sensing via ratiometric nanofibers with constraint-based inverse modeling. We apply our method to cell cultures with mixed populations of cancer cells and fibroblasts. We find that the proton trafficking underlying bulk acidification is strongly heterogeneous, with maximal single-cell fluxes exceeding typical values by up to 3 orders of magnitude. In addition, a crossover in time from a networked phase sustained by densely connected "hubs" (corresponding to cells with high activity) to a sparse phase dominated by isolated dipolar motifs (i.e., by pairwise cell-to-cell exchanges) is uncovered, which parallels the time course of bulk acidification. Our method addresses issues ranging from the homeostatic function of proton exchange to the metabolic coupling of cells with different energetic demands, allowing for real-time noninvasive single-cell metabolic flux analysis.

Outperforming RBM Feature-Extraction Capabilities by "Dreaming" Mechanism.

Inspired by a formal equivalence between the Hopfield model and restricted Boltzmann machines (RBMs), we design a Boltzmann machine, referred to as the dreaming Boltzmann machine (DBM), which achieves better performances than the standard one. The novelty in our model lies in a precise prescription for intralayer connections among hidden neurons whose strengths depend on features correlations. We analyze learning and retrieving capabilities in DBMs, both theoretically and numerically, and compare them to the RBM reference. We find that, in a supervised scenario, the former significantly outperforms the latter. Furthermore, in the unsupervised case, the DBM achieves better performances both in features extraction and representation learning, especially when the network is properly pretrained. Finally, we compare both models in simple classification tasks and find that the DBM again outperforms the RBM reference.

Fully Automated Computational Approach for Precisely Measuring Organelle Acidification with Optical pH Sensors.

pH balance and regulation within organelles are fundamental to cell homeostasis and proliferation. The ability to track pH in cells becomes significantly important to understand these processes in detail. Fluorescent sensors based on micro- and nanoparticles have been applied to measure intracellular pH; however, an accurate methodology to precisely monitor acidification kinetics of organelles in living cells has not been established, limiting the scope of this class of sensors. Here, silica-based fluorescent microparticles were utilized to probe the pH of intracellular organelles in MDA-MB-231 and MCF-7 breast cancer cells. In addition to the robust, ratiometric, trackable, and bioinert pH sensors, we developed a novel dimensionality reduction algorithm to automatically track and screen massive internalization events of pH sensors. We found that the mean acidification time is comparable among the two cell lines (ΔTMCF-7 = 16.3 min; ΔTMDA-MB-231 = 19.5 min); however, MCF-7 cells showed a much broader heterogeneity in comparison to MDA-MB-231 cells. The use of pH sensors and ratiometric imaging of living cells in combination with a novel computational approach allow analysis of thousands of events in a computationally inexpensive and faster way than the standard routes. The reported methodology can potentially be used to monitor pH as well as several other parameters associated with endocytosis.

The emergence of a concept in shallow neural networks.

We consider restricted Boltzmann machine (RBMs) trained over an unstructured dataset made of blurred copies of definite but unavailable "archetypes" and we show that there exists a critical sample size beyond which the RBM can learn archetypes, namely the machine can successfully play as a generative model or as a classifier, according to the operational routine. In general, assessing a critical sample size (possibly in relation to the quality of the dataset) is still an open problem in machine learning. Here, restricting to the random theory, where shallow networks suffice and the "grandmother-cell" scenario is correct, we leverage the formal equivalence between RBMs and Hopfield networks, to obtain a phase diagram for both the neural architectures which highlights regions, in the space of the control parameters (i.e., number of archetypes, number of neurons, size and quality of the training set), where learning can be accomplished. Our investigations are led by analytical methods based on the statistical-mechanics of disordered systems and results are further corroborated by extensive Monte Carlo simulations.

Deregulation of microtubule organization and RNA metabolism in Arx models for lissencephaly and developmental epileptic encephalopathy.

X-linked lissencephaly with abnormal genitalia (XLAG) and developmental epileptic encephalopathy-1 (DEE1) are caused by mutations in the Aristaless-related homeobox (ARX) gene, which encodes a transcription factor responsible for brain development. It has been unknown whether the phenotypically diverse XLAG and DEE1 phenotypes may converge on shared pathways. To address this question, a label-free quantitative proteomic approach was applied to the neonatal brain of Arx knockout (ArxKO/Y) and knock-in polyalanine (Arx(GCG)7/Y) mice that are respectively models for XLAG and DEE1. Gene ontology and protein-protein interaction analysis revealed that cytoskeleton, protein synthesis and splicing control are deregulated in an allelic-dependent manner. Decreased α-tubulin content was observed both in Arx mice and Arx/alr-1(KO) Caenorhabditis elegans ,and a disorganized neurite network in murine primary neurons was consistent with an allelic-dependent secondary tubulinopathy. As distinct features of Arx(GCG)7/Y mice, we detected eIF4A2 overexpression and translational suppression in cortex and primary neurons. Allelic-dependent differences were also established in alternative splicing (AS) regulated by PUF60 and SAM68. Abnormal AS repertoires in Neurexin-1, a gene encoding multiple pre-synaptic organizers implicated in synaptic remodelling, were detected in Arx/alr-1(KO) animals and in Arx(GCG)7/Y epileptogenic brain areas and depolarized cortical neurons. Consistent with a conserved role of ARX in modulating AS, we propose that the allelic-dependent secondary synaptopathy results from an aberrant Neurexin-1 repertoire. Overall, our data reveal alterations mirroring the overlapping and variant effects caused by null and polyalanine expanded mutations in ARX. The identification of these effects can aid in the design of pathway-guided therapy for ARX endophenotypes and NDDs with overlapping comorbidities.

On the effective initialisation for restricted Boltzmann machines via duality with Hopfield model.

Restricted Boltzmann machines (RBMs) with a binary visible layer of size N and a Gaussian hidden layer of size P have been proved to be equivalent to a Hopfield neural network (HNN) made of N binary neurons and storing P patterns ξ, as long as the weights w in the former are identified with the patterns. Here we aim to leverage this equivalence to find effective initialisations for weights in the RBM when what is available is a set of noisy examples of each pattern, aiming to translate statistical mechanics background available for HNN to the study of RBM's learning and retrieval abilities. In particular, given a set of definite, structureless patterns we build a sample of blurred examples and prove that the initialisation where w corresponds to the empirical average ξ¯ over the sample is a fixed point under stochastic gradient descent. Further, as a toy application of the duality between HNN and RBM, we consider the simplest random auto-encoder (a three layer network made of two RBMs coupled by their hidden layer) and evidence that, as long as the parameter setting corresponds to the retrieval region of the dual HNN, reconstruction and denoising can be accomplished trivially, while when the system is in the spin-glass phase inference algorithms are necessary. This questions the need for larger retrieval regions which we obtain by applying a Gram-Schmidt orthogonalisation to the patterns: in fact, this procedure yields to a set of patterns devoid of correlations and for which the largest retrieval region can be accomplished. Finally we consider an application of duality also in a structured case: we test this approach on the MNIST dataset, and obtain that the network performs already ∼67% of successful classifications, suggesting it can be exploited as a computationally-cheap pre-training.

Analysis of temporal correlation in heart rate variability through maximum entropy principle in a minimal pairwise glassy model.

In this work we apply statistical mechanics tools to infer cardiac pathologies over a sample of M patients whose heart rate variability has been recorded via 24 h Holter device and that are divided in different classes according to their clinical status (providing a repository of labelled data). Considering the set of inter-beat interval sequences [Formula: see text], with [Formula: see text], we estimate their probability distribution [Formula: see text] exploiting the maximum entropy principle. By setting constraints on the first and on the second moment we obtain an effective pairwise [Formula: see text] model, whose parameters are shown to depend on the clinical status of the patient. In order to check this framework, we generate synthetic data from our model and we show that their distribution is in excellent agreement with the one obtained from experimental data. Further, our model can be related to a one-dimensional spin-glass with quenched long-range couplings decaying with the spin-spin distance as a power-law. This allows us to speculate that the 1/f noise typical of heart-rate variability may stem from the interplay between the parasympathetic and orthosympathetic systems.

Detecting cardiac pathologies via machine learning on heart-rate variability time series and related markers.

In this paper we develop statistical algorithms to infer possible cardiac pathologies, based on data collected from 24 h Holter recording over a sample of 2829 labelled patients; labels highlight whether a patient is suffering from cardiac pathologies. In the first part of the work we analyze statistically the heart-beat series associated to each patient and we work them out to get a coarse-grained description of heart variability in terms of 49 markers well established in the reference community. These markers are then used as inputs for a multi-layer feed-forward neural network that we train in order to make it able to classify patients. However, before training the network, preliminary operations are in order to check the effective number of markers (via principal component analysis) and to achieve data augmentation (because of the broadness of the input data). With such groundwork, we finally train the network and show that it can classify with high accuracy (at most ~85% successful identifications) patients that are healthy from those displaying atrial fibrillation or congestive heart failure. In the second part of the work, we still start from raw data and we get a classification of pathologies in terms of their related networks: patients are associated to nodes and links are drawn according to a similarity measure between the related heart-beat series. We study the emergent properties of these networks looking for features (e.g., degree, clustering, clique proliferation) able to robustly discriminate between networks built over healthy patients or over patients suffering from cardiac pathologies. We find overall very good agreement among the two paved routes.

Generalized Guerra's interpolation schemes for dense associative neural networks.

In this work we develop analytical techniques to investigate a broad class of associative neural networks set in the high-storage regime. These techniques translate the original statistical-mechanical problem into an analytical-mechanical one which implies solving a set of partial differential equations, rather than tackling the canonical probabilistic route. We test the method on the classical Hopfield model - where the cost function includes only two-body interactions (i.e., quadratic terms) - and on the "relativistic" Hopfield model - where the (expansion of the) cost function includes p-body (i.e., of degree p) contributions. Under the replica symmetric assumption, we paint the phase diagrams of these models by obtaining the explicit expression of their free energy as a function of the model parameters (i.e., noise level and memory storage). Further, since for non-pairwise models ergodicity breaking is non necessarily a critical phenomenon, we develop a fluctuation analysis and find that criticality is preserved in the relativistic model.

A statistical inference approach to reconstruct intercellular interactions in cell migration experiments.

Migration of cells can be characterized by two prototypical types of motion: individual and collective migration. We propose a statistical inference approach designed to detect the presence of cell-cell interactions that give rise to collective behaviors in cell motility experiments. This inference method has been first successfully tested on synthetic motional data and then applied to two experiments. In the first experiment, cells migrate in a wound-healing model: When applied to this experiment, the inference method predicts the existence of cell-cell interactions, correctly mirroring the strong intercellular contacts that are present in the experiment. In the second experiment, dendritic cells migrate in a chemokine gradient. Our inference analysis does not provide evidence for interactions, indicating that cells migrate by sensing independently the chemokine source. According to this prediction, we speculate that mature dendritic cells disregard intercellular signals that could otherwise delay their arrival to lymph vessels.

Neural Networks with a Redundant Representation: Detecting the Undetectable.

We consider a three-layer Sejnowski machine and show that features learnt via contrastive divergence have a dual representation as patterns in a dense associative memory of order P=4. The latter is known to be able to Hebbian store an amount of patterns scaling as N^{P-1}, where N denotes the number of constituting binary neurons interacting P wisely. We also prove that, by keeping the dense associative network far from the saturation regime (namely, allowing for a number of patterns scaling only linearly with N, while P>2) such a system is able to perform pattern recognition far below the standard signal-to-noise threshold. In particular, a network with P=4 is able to retrieve information whose intensity is O(1) even in the presence of a noise O(sqrt[N]) in the large N limit. This striking skill stems from a redundancy representation of patterns-which is afforded given the (relatively) low-load information storage-and it contributes to explain the impressive abilities in pattern recognition exhibited by new-generation neural networks. The whole theory is developed rigorously, at the replica symmetric level of approximation, and corroborated by signal-to-noise analysis and Monte Carlo simulations.

Histone demethylase KDM5C is a SAHA-sensitive central hub at the crossroads of transcriptional axes involved in multiple neurodevelopmental disorders.

A disproportional large number of neurodevelopmental disorders (NDDs) is caused by variants in genes encoding transcription factors and chromatin modifiers. However, the functional interactions between the corresponding proteins are only partly known. Here, we show that KDM5C, encoding a H3K4 demethylase, is at the intersection of transcriptional axes under the control of three regulatory proteins ARX, ZNF711 and PHF8. Interestingly, mutations in all four genes (KDM5C, ARX, ZNF711 and PHF8) are associated with X-linked NDDs comprising intellectual disability as a core feature. in vitro analysis of the KDM5C promoter revealed that ARX and ZNF711 function as antagonist transcription factors that activate KDM5C expression and compete for the recruitment of PHF8. Functional analysis of mutations in these genes showed a correlation between phenotype severity and the reduction in KDM5C transcriptional activity. The KDM5C decrease was associated with a lack of repression of downstream target genes Scn2a, Syn1 and Bdnf in the embryonic brain of Arx-null mice. Aiming to correct the faulty expression of KDM5C, we studied the effect of the FDA-approved histone deacetylase inhibitor suberanilohydroxamic acid (SAHA). In Arx-KO murine ES-derived neurons, SAHA was able to rescue KDM5C depletion, recover H3K4me3 signalling and improve neuronal differentiation. Indeed, in ARX/alr-1-deficient Caenorhabditis elegans animals, SAHA was shown to counteract the defective KDM5C/rbr-2-H3K4me3 signalling, recover abnormal behavioural phenotype and ameliorate neuronal maturation. Overall, our studies indicate that KDM5C is a conserved and druggable effector molecule across a number of NDDs for whom the use of SAHA may be considered a potential therapeutic strategy.

A New Synthetic Spiroketal: Studies on Antitumor Activity on Murine Melanoma Model In Vivo and Mechanism of Action In Vitro.

BACKGROUND: In a previous study, we synthesised a new spiroketal derivative, inspired to natural products, that has shown high antiproliferative activity, potent telomerase inhibition and proapoptotic activity on several human cell lines. OBJECTIVE: This work focused on the study of in vivo antitumor effect of this synthetic spiroketal on a murine melanoma model. In order to shed additional light on the origin of the antitumor effect, in vitro studies were performed. METHODS: Spiroketal was administered to B16F10 melanoma mice at a dose of 5 mg/Kg body weight via intraperitoneum at alternate days for 15 days. Tumor volume measures were made every 2 days starting after 12 days from cells injection. The effects of the spiroketal on tumor growth inhibition, apoptosis induction, and cell cycle modification were investigated in vitro on B16 cells. HIF1α gene expression, the inhibition of cells migration and the changes induced in cytoskeleton conformation were evaluated. RESULTS: Spiroketal displayed proapoptotic activity and high antitumor activity in B16 cells with nanomolar IC50. Moreover it has shown to inhibit cell migration, to strongly reduce the HIF1α expression and to induce strongly deterioration of cytoskeleton structure. A potent dose-dependent antitumor efficacy in syngenic B16/C57BL/6J murine model of melanoma was observed with the suppression of tumor growth by an average of 90% at a dose of 5 mg/kg. CONCLUSION: The synthesized spiroketal shows high antitumor activity in the B16 cells in vitro at nM concentration and a dose-dependent antitumor efficacy in syngenic B16/C57BL/6J mice. The results suggest that this natural product inspired spiroketal may have a potential application in melanoma therapy.

Dreaming neural networks: Forgetting spurious memories and reinforcing pure ones.

The standard Hopfield model for associative neural networks accounts for biological Hebbian learning and acts as the harmonic oscillator for pattern recognition, however its maximal storage capacity is α∼0.14, far from the theoretical bound for symmetric networks, i.e. α=1. Inspired by sleeping and dreaming mechanisms in mammal brains, we propose an extension of this model displaying the standard on-line (awake) learning mechanism (that allows the storage of external information in terms of patterns) and an off-line (sleep) unlearning&consolidating mechanism (that allows spurious-pattern removal and pure-pattern reinforcement): this obtained daily prescription is able to saturate the theoretical bound α=1, remaining also extremely robust against thermal noise. The emergent neural and synaptic features are analyzed both analytically and numerically. In particular, beyond obtaining a phase diagram for neural dynamics, we focus on synaptic plasticity and we give explicit prescriptions on the temporal evolution of the synaptic matrix. We analytically prove that our algorithm makes the Hebbian kernel converge with high probability to the projection matrix built over the pure stored patterns. Furthermore, we obtain a sharp and explicit estimate for the "sleep rate" in order to ensure such a convergence. Finally, we run extensive numerical simulations (mainly Monte Carlo sampling) to check the approximations underlying the analytical investigations (e.g., we developed the whole theory at the so called replica-symmetric level, as standard in the Amit-Gutfreund-Sompolinsky reference framework) and possible finite-size effects, finding overall full agreement with the theory.

Towards the development of human immune-system-on-a-chip platforms.

Organ-on-a-chip (OoCs) platforms could revolutionize drug discovery and might ultimately become essential tools for precision therapy. Although many single-organ and interconnected systems have been described, the immune system has been comparatively neglected, despite its pervasive role in the body and the trend towards newer therapeutic products (i.e., complex biologics, nanoparticles, immune checkpoint inhibitors, and engineered T cells) that often cause, or are based on, immune reactions. In this review, we recapitulate some distinctive features of the immune system before reviewing microfluidic devices that mimic lymphoid organs or other organs and/or tissues with an integrated immune system component.

A new mechanical approach to handle generalized Hopfield neural networks.

We propose a modification of the cost function of the Hopfield model whose salient features shine in its Taylor expansion and result in more than pairwise interactions with alternate signs, suggesting a unified framework for handling both with deep learning and network pruning. In our analysis, we heavily rely on the Hamilton-Jacobi correspondence relating the statistical model with a mechanical system. In this picture, our model is nothing but the relativistic extension of the original Hopfield model (whose cost function is a quadratic form in the Mattis magnetization and mimics the non-relativistic counterpart, the so-called classical limit). We focus on the low-storage regime and solve the model analytically by taking advantage of the mechanical analogy, thus obtaining a complete characterization of the free energy and the associated self-consistency equations in the thermodynamic limit. Further, on the numerical side, we test the performances of our proposal with extensive Monte Carlo simulations, showing that the stability of spurious states (limiting the capabilities of the standard Hebbian construction) is sensibly reduced due to presence of unlearning contributions that prune them massively.

Phase diagram of restricted Boltzmann machines and generalized Hopfield networks with arbitrary priors.

Restricted Boltzmann machines are described by the Gibbs measure of a bipartite spin glass, which in turn can be seen as a generalized Hopfield network. This equivalence allows us to characterize the state of these systems in terms of their retrieval capabilities, both at low and high load, of pure states. We study the paramagnetic-spin glass and the spin glass-retrieval phase transitions, as the pattern (i.e., weight) distribution and spin (i.e., unit) priors vary smoothly from Gaussian real variables to Boolean discrete variables. Our analysis shows that the presence of a retrieval phase is robust and not peculiar to the standard Hopfield model with Boolean patterns. The retrieval region becomes larger when the pattern entries and retrieval units get more peaked and, conversely, when the hidden units acquire a broader prior and therefore have a stronger response to high fields. Moreover, at low load retrieval always exists below some critical temperature, for every pattern distribution ranging from the Boolean to the Gaussian case.