On PDE Characterization of Smooth Hierarchical Functions Computed by Neural Networks.

Neural networks are versatile tools for computation, having the ability to approximate a broad range of functions. An important problem in the theory of deep neural networks is expressivity; that is, we want to understand the functions that are computable by a given network. We study real, infinitely differentiable (smooth) hierarchical functions implemented by feedforward neural networks via composing simpler functions in two cases: (1) each constituent function of the composition has fewer inputs than the resulting function and (2) constituent functions are in the more specific yet prevalent form of a nonlinear univariate function (e.g., tanh) applied to a linear multivariate function. We establish that in each of these regimes, there exist nontrivial algebraic partial differential equations (PDEs) that are satisfied by the computed functions. These PDEs are purely in terms of the partial derivatives and are dependent only on the topology of the network. Conversely, we conjecture that such PDE constraints, once accompanied by appropriate nonsingularity conditions and perhaps certain inequalities involving partial derivatives, guarantee that the smooth function under consideration can be represented by the network. The conjecture is verified in numerous examples, including the case of tree architectures, which are of neuroscientific interest. Our approach is a step toward formulating an algebraic description of functional spaces associated with specific neural networks, and may provide useful new tools for constructing neural networks.

On Functions Computed on Trees.

Any function can be constructed using a hierarchy of simpler functions through compositions. Such a hierarchy can be characterized by a binary rooted tree. Each node of this tree is associated with a function that takes as inputs two numbers from its children and produces one output. Since thinking about functions in terms of computation graphs is becoming popular, we may want to know which functions can be implemented on a given tree. Here, we describe a set of necessary constraints in the form of a system of nonlinear partial differential equations that must be satisfied. Moreover, we prove that these conditions are sufficient in contexts of analytic and bit-valued functions. In the latter case, we explicitly enumerate discrete functions and observe that there are relatively few. Our point of view allows us to compare different neural network architectures in regard to their function spaces. Our work connects the structure of computation graphs with the functions they can implement and has potential applications to neuroscience and computer science.

Predicting Feeding Conditions of Premature Infants Through Non-Nutritive Sucking Skills Using a Sensitized Pacifier.

Due to the lack of enough physical or suck central pattern generator (SCPG) development, premature infants require assistance in improving their sucking skills as one of the first coordinated muscular activities in infants. Hence, we need to quantitatively measure their sucking abilities for future studies on their sucking interventions. Here, we present a new device that can measure both intraoral pressure (IP) and expression pressure (EP) as ororhithmic behavior parameters of non-nutritive sucking skills in infants. Our device is low-cost, easy-to-use, and accurate, which makes it appropriate for extensive studies. To showcase one of the applications of our device, we collected weekly data from 137 premature infants from 29 week-old to 36 week-old. Around half of the infants in our study needed intensive care even after they were 36 week-old. We call them full attainment of oral feeding (FAOF) infants. We then used the Non-nutritive sucking (NNS) features of EP and IP signals of infants recorded by our device to predict FAOF infants' sucking conditions. We found that our pipeline can predict FAOF infants several weeks before discharge from the hospital. Thus, this application of our device presents a robust and inexpensive alternative to monitor oral feeding ability in premature infants.

Evaluation of Apgar Scores and Non-Nutritive Sucking Skills in Infants Using a Novel Sensitized Non-Nutritive Sucking System.

One of the challenges in examining development of newborns is measuring activities which are correlated to their health. Oral feeding is the most important factor in an infant's healthy development. Here, we present a new device that can measure intraoral and expression pressures produced in a newborn's mouth by non-nutritive sucking. We then develop a method to extract time-intervals that a sucking has occurred. To show an application of this device, we use Apgar score as a reference of the general health of newborns, and we evaluate these scores with the non-nutritive sucking patterns demonstrated by the infants. We show that for the pairs of infant with the same background but different Apgar scores, those with lower Apgar scores have lower pressure amplitudes while sucking. Importance of non-nutritive sucking skills in the development of newborns and ease of using our device make it useful for clinical studies of infantile health.

Quantifying How Staining Methods Bias Measurements of Neuron Morphologies.

The process through which neurons are labeled is a key methodological choice in measuring neuron morphology. However, little is known about how this choice may bias measurements. To quantify this bias we compare the extracted morphology of neurons collected from the same rodent species, experimental condition, gender distribution, age distribution, brain region and putative cell type, but obtained with 19 distinct staining methods. We found strong biases on measured features of morphology. These were largest in features related to the coverage of the dendritic tree (e.g., the total dendritic tree length). Understanding measurement biases is crucial for interpreting morphological data.

The roles of supervised machine learning in systems neuroscience.

Over the last several years, the use of machine learning (ML) in neuroscience has been rapidly increasing. Here, we review ML's contributions, both realized and potential, across several areas of systems neuroscience. We describe four primary roles of ML within neuroscience: (1) creating solutions to engineering problems, (2) identifying predictive variables, (3) setting benchmarks for simple models of the brain, and (4) serving itself as a model for the brain. The breadth and ease of its applicability suggests that machine learning should be in the toolbox of most systems neuroscientists.