Gaussian Process-Gated Hierarchical Mixtures of Experts.

In this paper, we propose novel Gaussian process-gated hierarchical mixtures of experts (GPHMEs). Unlike other mixtures of experts with gating models linear in the input, our model employs gating functions built with Gaussian processes (GPs). These processes are based on random features that are non-linear functions of the inputs. Furthermore, the experts in our model are also constructed with GPs. The optimization of the GPHMEs is performed by variational inference. The proposed GPHMEs have several advantages. They outperform tree-based HME benchmarks that partition the data in the input space, and they achieve good performance with reduced complexity. Another advantage is the interpretability they provide for deep GPs, and more generally, for deep Bayesian neural networks. Our GPHMEs demonstrate excellent performance for large-scale data sets, even with quite modest sizes.

A quantitative model of the cerebral windkessel and its relevance to disorders of intracranial dynamics.

OBJECTIVE: Traditional models of intracranial dynamics fail to capture several important features of the intracranial pressure (ICP) pulse. Experiments show that, at a local amplitude minimum, the ICP pulse normally precedes the arterial blood pressure (ABP) pulse, and the cranium is a band-stop filter centered at the heart rate for the ICP pulse with respect to the ABP pulse, which is the cerebral windkessel mechanism. These observations are inconsistent with existing pressure-volume models. METHODS: To explore these issues, the authors modeled the ABP and ICP pulses by using a simple electrical tank circuit, and they compared the dynamics of the circuit to physiological data from dogs by using autoregressive with exogenous inputs (ARX) modeling. RESULTS: The authors' ARX analysis showed close agreement between the circuit and pulse suppression in the canine cranium, and they used the analogy between the circuit and the cranium to examine the dynamics that underlie this pulse suppression. CONCLUSIONS: This correspondence between physiological data and circuit dynamics suggests that the cerebral windkessel consists of the rhythmic motion of the brain parenchyma and CSF that continuously opposes systolic and diastolic blood flow. Such motion has been documented with flow-sensitive MRI. In thermodynamic terms, the direct current (DC) power of cerebral arterial perfusion drives smooth capillary flow and alternating current (AC) power shunts pulsatile energy through the CSF to the veins. This suggests that hydrocephalus and related disorders are disorders of CSF path impedance. Obstructive hydrocephalus is the consequence of high CSF path impedance due to high resistance. Normal pressure hydrocephalus (NPH) is the consequence of high CSF path impedance due to low inertance and high compliance. Low-pressure hydrocephalus is the consequence of high CSF path impedance due to high resistance and high compliance. Ventriculomegaly is an adaptive physiological response that increases CSF path volume and thereby reduces CSF path resistance and impedance. Pseudotumor cerebri is the consequence of high DC power with normal CSF path impedance. CSF diversion by shunting is an accessory windkessel-it drains energy (and thereby lowers ICP) and lowers CSF path resistance and impedance. Cushing's reflex is an accessory windkessel in extremis-it maintains DC power (arterial hypertension) and reduces AC power (bradycardia). The windkessel theory is a thermodynamic approach to the study of energy flow through the cranium, and it points to a new understanding of hydrocephalus and related disorders.

Microscale multicircuit brain stimulation: Achieving real-time brain state control for novel applications.

Neurological and psychiatric disorders typically result from dysfunction across multiple neural circuits. Most of these disorders lack a satisfactory neuromodulation treatment. However, deep brain stimulation (DBS) has been successful in a limited number of disorders; DBS typically targets one or two brain areas with single contacts on relatively large electrodes, allowing for only coarse modulation of circuit function. Because of the dysfunction in distributed neural circuits - each requiring fine, tailored modulation - that characterizes most neuropsychiatric disorders, this approach holds limited promise. To develop the next generation of neuromodulation therapies, we will have to achieve fine-grained, closed-loop control over multiple neural circuits. Recent work has demonstrated spatial and frequency selectivity using microstimulation with many small, closely-spaced contacts, mimicking endogenous neural dynamics. Using custom electrode design and stimulation parameters, it should be possible to achieve bidirectional control over behavioral outcomes, such as increasing or decreasing arousal during central thalamic stimulation. Here, we discuss one possible approach, which we term microscale multicircuit brain stimulation (MMBS). We discuss how machine learning leverages behavioral and neural data to find optimal stimulation parameters across multiple contacts, to drive the brain towards desired states associated with behavioral goals. We expound a mathematical framework for MMBS, where behavioral and neural responses adjust the model in real-time, allowing us to adjust stimulation in real-time. These technologies will be critical to the development of the next generation of neurostimulation therapies, which will allow us to treat problems like disorders of consciousness and cognition.

Electromyogram-Based Lip-Reading via Unobtrusive Dry Electrodes and Machine Learning Methods.

Lip-reading provides an effective speech communication interface for people with voice disorders and for intuitive human-machine interactions. Existing systems are generally challenged by bulkiness, obtrusiveness, and poor robustness against environmental interferences. The lack of a truly natural and unobtrusive system for converting lip movements to speech precludes the continuous use and wide-scale deployment of such devices. Here, the design of a hardware-software architecture to capture, analyze, and interpret lip movements associated with either normal or silent speech is presented. The system can recognize different and similar visemes. It is robust in a noisy or dark environment. Self-adhesive, skin-conformable, and semi-transparent dry electrodes are developed to track high-fidelity speech-relevant electromyogram signals without impeding daily activities. The resulting skin-like sensors can form seamless contact with the curvilinear and dynamic surfaces of the skin, which is crucial for a high signal-to-noise ratio and minimal interference. Machine learning algorithms are employed to decode electromyogram signals and convert them to spoken words. Finally, the applications of the developed lip-reading system in augmented reality and medical service are demonstrated, which illustrate the great potential in immersive interaction and healthcare applications.

Missingness patterns in a comprehensive instrument identifying psychosocial and substance use risk in antenatal care.

BACKGROUND: Psychosocial vulnerabilities (e.g. inadequate social support, financial insecurity, stress) and substance use elevate risks for adverse perinatal outcomes and maternal mental health morbidities. However, various barriers, including paucity of validated, simple and usable comprehensive instruments, impede execution of the recommendations to screen for such vulnerabilities in the first antenatal care visit. The current study presents findings from a newly implemented self-report tool created to overcome screening barriers in outpatient antenatal clinics. METHODS: This was a retrospective chart-review of 904 women who completed the Profile for Maternal & Obstetric Treatment Effectiveness (PROMOTE) during their first antenatal visit between June and December 2019. The PROMOTE includes the 4-item NIDA Quick Screen and 15 additional items that each assess a different psychosocial vulnerability. Statistical analysis included evaluation of missing data, and exploration of missing data patterns using univariate correlations and hierarchical clustering. RESULTS: Three quarters of women (70.0%) had no missing items. In the entire sample, all but four PROMOTE items (opioid use, planned pregnancy, educational level, and financial state) had < 5% missing values, suggesting good acceptability and feasibility. Several respondent-related characteristics such as lower education, less family support, and greater stress were associated with greater likelihood of missing items. Instrument-related characteristics associated with missing values were completing the PROMOTE in Spanish or question positioning at the end of the instrument. CONCLUSIONS AND IMPLICATIONS: Conducting a comprehensive screening of theoretically and clinically meaningful vulnerabilities in an outpatient setting is feasible. Study findings will inform modifications of the PROMOTE and subsequent digitisation.

Applying machine learning methods to psychosocial screening data to improve identification of prenatal depression: Implications for clinical practice and research.

We utilized machine learning (ML) methods on data from the PROMOTE, a novel psychosocial screening tool, to quantify risk for prenatal depression for individual patients and identify contributing factors that impart greater risk for depression. Random forest algorithms were used to predict likelihood for being at high risk for prenatal depression (Edinburgh Postnatal Depression Scale; EPDS ≥ 13 and/or positive self-injury item) using data from 1715 patients who completed the PROMOTE. Performance matrices were calculated to assess the ability of the PROMOTE to accurately classify patients. Probability for depression was calculated for individual patients. Finally, recursive feature elimination was used to evaluate the importance of each PROMOTE item in the classification of depression risk. PROMOTE data were successfully used to predict depression with acceptable performance matrices (accuracy = 0.80; sensitivity = 0.75; specificity = 0.81; positive predictive value = 0.79; negative predictive value = 0.97). Perceived stress, emotional problems, family support, age, major life events, partner support, unplanned pregnancy, current employment, lifetime abuse, and financial state were the most important PROMOTE items in the classification of depression risk. Results affirm the value of the PROMOTE as a psychosocial screening tool for prenatal depression and the benefit of using it in conjunction with ML methods. Using such methods can help detect underreported outcomes and identify what in patients' lives makes them more vulnerable, thus paving the way for effective individually tailored precision medicine.

BOOST ENSEMBLE LEARNING FOR CLASSIFICATION OF CTG SIGNALS.

During the process of childbirth, fetal distress caused by hypoxia can lead to various abnormalities. Cardiotocography (CTG), which consists of continuous recording of the fetal heart rate (FHR) and uterine contractions (UC), is routinely used for classifying the fetuses as hypoxic or non-hypoxic. In practice, we face highly imbalanced data, where the hypoxic fetuses are significantly underrepresented. We propose to address this problem by boost ensemble learning, where for learning, we use the distribution of classification error over the dataset. We then iteratively select the most informative majority data samples according to this distribution. In our work, in addition to addressing the imbalanced problem, we also experimented with features that are not commonly used in obstetrics. We extracted a large number of statistical features of fetal heart tracings and uterine activity signals and used only the most informative ones. For classification, we implemented several methods: Random Forest, AdaBoost, k-Nearest Neighbors, Support Vector Machine, and Decision Trees. The paper provides a comparison in the performance of these methods on fetal heart rate tracings available from a public database. Our results show that most applied methods improved their performances considerably when boost ensemble was used.

UNSUPERVISED CLUSTERING AND ANALYSIS OF CONTRACTION-DEPENDENT FETAL HEART RATE SEGMENTS.

The computer-aided interpretation of fetal heart rate (FHR) and uterine contraction (UC) has not been developed well enough for wide use in delivery rooms. The main challenges still lie in the lack of unclear and nonstandard labels for cardiotocography (CTG) recordings, and the timely prediction of fetal state during monitoring. Rather than traditional supervised approaches to FHR classification, this paper demonstrates a way to understand the UC-dependent FHR responses in an unsupervised manner. In this work, we provide a complete method for FHR-UC segment clustering and analysis via the Gaussian process latent variable model, and density-based spatial clustering. We map the UC-dependent FHR segments into a space with a visual dimension and propose a trajectory-based FHR interpretation method. Three metrics of FHR trajectory are defined and an open-access CTG database is used for testing the proposed method.

IMPROVING PHASE-RECTIFIED SIGNAL AVERAGING FOR FETAL HEART RATE ANALYSIS.

Low umbilical artery pH is a marker for neonatal acidosis and is associated with an increased risk for neonatal complications. The phase-rectified signal averaging (PRSA) features have demonstrated superior discriminatory or diagnostic ability and good interpretability in many biomedical applications including fetal heart rate analysis. However, the performance of PRSA method is sensitive to values of the selected parameters which are usually either chosen based on a grid search or empirically in the literature. In this paper, we examine PRSA method through the lens of dynamical systems theory and reveal the intrinsic connection between state space reconstruction and PRSA. From this perspective, we then introduce a new feature that can better characterize dynamical systems comparing with PRSA. Our experimental results on an open-access intrapartum Cardiotocography database demonstrate that the proposed feature outperforms state-of-the-art PRSA features in pH-based fetal heart rate analysis.

Why don't ventricles dilate in pseudotumor cerebri? A circuit model of the cerebral windkessel.

OBJECTIVE: Pseudotumor cerebri is a disorder of intracranial dynamics characterized by elevated intracranial pressure (ICP) and chronic cerebral venous hypertension without structural abnormalities. A perplexing feature of pseudotumor is the absence of the ventriculomegaly found in obstructive hydrocephalus, although both diseases are associated with increased resistance to cerebrospinal fluid (CSF) resorption. Traditionally, the pathophysiology of ventricular dilation and obstructive hydrocephalus has been attributed to the backup of CSF due to impaired absorption, and it is unclear why backup of CSF with resulting ventriculomegaly would not occur in pseudotumor. In this study, the authors used an electrical circuit model to simulate the cerebral windkessel effect and explain the presence of ventriculomegaly in obstructive hydrocephalus but not in pseudotumor cerebri. METHODS: The cerebral windkessel is a band-stop filter that dampens the arterial blood pressure pulse in the cranium. The authors used a tank circuit with parallel inductance and capacitance to model the windkessel. The authors distinguished the smooth flow of blood and CSF and the pulsatile flow of blood and CSF by using direct current (DC) and alternating current (AC) sources, respectively. The authors measured the dampening notch from ABP to ICP as the band-stop filter of the windkessel. RESULTS: In obstructive hydrocephalus, loss of CSF pathway volume impaired the flow of AC power in the cranium and caused windkessel impairment, to which ventriculomegaly is an adaptation. In pseudotumor, venous hypertension affected DC power flow in the capillaries but did not affect AC power or the windkessel, therefore obviating the need for adaptive ventriculomegaly. CONCLUSIONS: In pseudotumor, the CSF spaces are unaffected and the windkessel remains effective. Therefore, ventricles remain normal in size. In hydrocephalus, the windkessel, which depends on the flow of AC power in patent CSF spaces, is impaired, and the ventricles dilate as an adaptive process to restore CSF pathway volume. The windkessel model explains both ventriculomegaly in obstructive hydrocephalus and the lack of ventriculomegaly in pseudotumor. This model provides a novel understanding of the pathophysiology of disorders of CSF dynamics and has significant implications in clinical management.

Injury to thalamocortical projections following traumatic brain injury results in attractor dynamics for cortical networks.

Major theories of consciousness predict that complex electroencephalographic (EEG) activity is required for consciousness, yet it is not clear how such activity arises in the corticothalamic system. The thalamus is well-known to control cortical excitability via interlaminar projections, but whether thalamic input is needed for complexity is not known. We hypothesized that the thalamus facilitates complex activity by adjusting synaptic connectivity, thereby increasing the availability of different configurations of cortical neurons (cortical "states"), as well as the probability of state transitions. To test this hypothesis, we characterized EEG activity from prefrontal cortex (PFC) in traumatic brain injury (TBI) patients with and without injuries to thalamocortical projections, measured with diffusion tensor imaging (DTI). We found that injury to thalamic projections (especially from the mediodorsal thalamus) was strongly associated with unconsciousness and delta-band EEG activity. Using advanced signal processing techniques, we found that lack of thalamic input led to 1.) attractor dynamics for cortical networks with a tendency to visit the same states, 2.) a reduced repertoire of possible states, and 3.) high predictability of transitions between states. These results imply that complex PFC activity associated with consciousness depends on thalamic input. Our model implies that restoration of cortical connectivity is a critical function of the thalamus after brain injury. We draw a critical connection between thalamic input and complex cortical activity associated with consciousness.

Cardiotocography analysis by empirical dynamic modeling and Gaussian processes.

Introduction: During labor, fetal heart rate (FHR) and uterine activity (UA) can be continuously monitored using Cardiotocography (CTG). This is the most widely adopted approach for electronic fetal monitoring in hospitals. Both FHR and UA recordings are evaluated by obstetricians for assessing fetal well-being. Due to the complex and noisy nature of these recordings, the evaluation by obstetricians suffers from high interobserver and intraobserver variability. Machine learning is a field that has seen unprecedented advances in the past two decades and many efforts have been made in computerized analysis of CTG using machine learning methods. However, in the literature, the focus is often only on FHR signals unlike in evaluations performed by obstetricians where the UA signals are also taken into account. Methods: Machine learning is a field that has seen unprecedented advances in the past two decades and many efforts have been made in computerized analysis of CTG using machine learning methods. However, in the literature, the focus is often only on FHR signals unlike in evaluations performed by obstetricians where the UA signals are also taken into account. In this paper, we propose to model intrapartum CTG recordings from a dynamical system perspective using empirical dynamic modeling with Gaussian processes, which is a Bayesian nonparametric approach for estimation of functions. Results and Discussion: In the context of our paper, Gaussian processes are capable for simultaneous estimation of the dimensionality of attractor manifolds and reconstructing of attractor manifolds from time series data. This capacity of Gaussian processes allows for revealing causal relationships between the studied time series. Experimental results on real CTG recordings show that FHR and UA signals are causally related. More importantly, this causal relationship and estimated attractor manifolds can be exploited for several important applications in computerized analysis of CTG recordings including estimating missing FHR samples, recovering burst errors in FHR tracings and characterizing the interactions between FHR and UA signals.

Electrocorticography reveals thalamic control of cortical dynamics following traumatic brain injury.

The return of consciousness after traumatic brain injury (TBI) is associated with restoring complex cortical dynamics; however, it is unclear what interactions govern these complex dynamics. Here, we set out to uncover the mechanism underlying the return of consciousness by measuring local field potentials (LFP) using invasive electrophysiological recordings in patients recovering from TBI. We found that injury to the thalamus, and its efferent projections, on MRI were associated with repetitive and low complexity LFP signals from a highly structured phase space, resembling a low-dimensional ring attractor. But why do thalamic injuries in TBI patients result in a cortical attractor? We built a simplified thalamocortical model, which connotes that thalamic input facilitates the formation of cortical ensembles required for the return of cognitive function and the content of consciousness. These observations collectively support the view that thalamic input to the cortex enables rich cortical dynamics associated with consciousness.

IDENTIFICATION OF UTERINE CONTRACTIONS BY AN ENSEMBLE OF GAUSSIAN PROCESSES.

Identifying uterine contractions with the aid of machine learning methods is necessary vis-á-vis their use in combination with fetal heart rates and other clinical data for the assessment of a fetus wellbeing. In this paper, we study contraction identification by processing noisy signals due to uterine activities. We propose a complete four-step method where we address the imbalanced classification problem with an ensemble Gaussian process classifier, where the Gaussian process latent variable model is used as a decision-maker. The results of both simulation and real data show promising performance compared to existing methods.

CLASS-IMBALANCED CLASSIFIERS USING ENSEMBLES OF GAUSSIAN PROCESSES AND GAUSSIAN PROCESS LATENT VARIABLE MODELS.

Classification with imbalanced data is a common and challenging problem in many practical machine learning problems. Ensemble learning is a popular solution where the results from multiple base classifiers are synthesized to reduce the effect of a possibly skewed distribution of the training set. In this paper, binary classifiers based on Gaussian processes are chosen as bases for inferring the predictive distributions of test latent variables. We apply a Gaussian process latent variable model where the outputs of the Gaussian processes are used for making the final decision. The tests of the new method in both synthetic and real data sets show improved performance over standard approaches.

ADAPTIVE IMPORTANCE SAMPLING VIA AUTO-REGRESSIVE GENERATIVE MODELS AND GAUSSIAN PROCESSES.

The quality of importance distribution is vital to adaptive importance sampling, especially in high dimensional sampling spaces where the target distributions are sparse and hard to approximate. This requires that the proposal distributions are expressive and easily adaptable. Because of the need for weight calculation, point evaluation of the proposal distributions is also needed. The Gaussian process has been proven to be a highly expressive non-parametric model for conditional density estimation whose training process is also straightforward. In this paper, we introduce a class of adaptive importance sampling methods where the proposal distribution is constructed in a way that Gaussian processes are combined autoregressively. By numerical experiments of sampling from a high dimensional target distribution, we demonstrate that the method is accurate and efficient compared to existing methods.

Performance Limits of Direct Wideband Coherent 3D Localization in Distributed Massive MIMO Systems.

We address the accuracy of wideband direct position estimation of a radio transmitter via a distributed antenna array in 5G cellular systems. Our derivations are based only on the presence of spatially coherent line-of-sight (LoS) signal components, which is a realistic assumption in small cells, especially in the mmWave range. The system model considers collocated time and phase synchronized receiving front-ends with antennas distributed in 3D space at known locations and connected to the front-ends via calibrated coaxial cables or analog radio-frequency-over-fiber links. Furthermore, the signal model assumes spherical wavefronts. We derive the Cramér-Rao bounds (CRBs) for two implementations of the system: with (a) known signals and (b) random Gaussian signals. The results show how the bounds depend on the carrier frequency, number of samples used for estimation, and signal-to-noise ratios. They also show that increasing the number of antennas (such as in massive MIMO systems) considerably improves the accuracy and lowers the signal-to-noise threshold for localization even for non-cooperative transmitters. Finally, our derivations show that the square roots of the bounds are two to three orders of magnitude below the carrier wavelength for realistic system parameters.

Exploiting Causality for Improved Prediction of Patient Volumes by Gaussian Processes.

Estimating and surveillance volumes of patients are of great importance for public health and resource allocation. In many situations, the change of these volumes is correlated with many factors, e.g., seasonal environmental variables, medicine sales, and patient medical claims. It is often of interest to predict patient volumes and to that end, discovering causalities can improve the prediction accuracy. Correlations do not imply causations and they can be spurious, which in turn may entail deterioration of prediction performance if the prediction is based on them. By contrast, in this paper, we propose an approach for prediction based on causalities discovered by Gaussian processes. Our interest is in estimating volumes of patients that suffer from allergy and where the model and the results are highly interpretable. In selecting features, instead of only using correlation, we take causal information into account. Specifically, we adopt the Gaussian processes-based convergent cross mapping framework for causal discovery which is proven to be more reliable than the Granger causality when time series are coupled. Moreover, we introduce a novel method for selecting the history or look-back length of features from the perspective of a dynamical system in a principled manner. The quasi-periodicities that commonly exist in observations of volumes of patients and environment variables can readily be accommodated. Further, the proposed method performs well even in cases when the data are scarce. Also, the approach can be modified without much difficulty to forecast other types of patient volumes. We validate the method with synthetic and real-world datasets.

Unsupervised Detection of Anomalies in Fetal Heart Rate Tracings using Phase Space Reconstruction.

Detection of anomalies in time series is still a challenging problem. In this paper, we provide a new approach to unsupervised detection of anomalies in time series based on the concept of phase space reconstruction and manifolds. We propose a rotation-insensitive metric for quantifying the similarity of manifolds and a method that uses it for estimating the probability of an outlier. The proposed method does not rely on any features and can be used for signals with variable lengths. We tested it on both synthetic signals and real fetal heart rate tracings. The method has promising performance and can be used for interpreting the severity of fetal asphyxia.

Bayesian Nonparametric Dimensionality Reduction of Categorical Data for Predicting Severity of COVID-19 in Pregnant Women.

The coronavirus disease (COVID-19) has rapidly spread throughout the world and while pregnant women present the same adverse outcome rates, they are underrepresented in clinical research. We collected clinical data of 155 test-positive COVID-19 pregnant women at Stony Brook University Hospital. Many of these collected data are of multivariate categorical type, where the number of possible outcomes grows exponentially as the dimension of data increases. We modeled the data within the unsupervised Bayesian framework and mapped them into a lower dimensional space using latent Gaussian processes. The latent features in the lower dimensional space were further used for predicting if a pregnant woman would be admitted to a hospital due to COVID-19 or would remain with mild symptoms. We compared the prediction accuracy with the dummy/one-hot encoding of categorical data and found that the latent Gaussian process had better accuracy.