Deep latent variable joint cognitive modeling of neural signals and human behavior.

As the field of computational cognitive neuroscience continues to expand and generate new theories, there is a growing need for more advanced methods to test the hypothesis of brain-behavior relationships. Recent progress in Bayesian cognitive modeling has enabled the combination of neural and behavioral models into a single unifying framework. However, these approaches require manual feature extraction, and lack the capability to discover previously unknown neural features in more complex data. Consequently, this would hinder the expressiveness of the models. To address these challenges, we propose a Neurocognitive Variational Autoencoder (NCVA) to conjoin high-dimensional EEG with a cognitive model in both generative and predictive modeling analyses. Importantly, our NCVA enables both the prediction of EEG signals given behavioral data and the estimation of cognitive model parameters from EEG signals. This novel approach can allow for a more comprehensive understanding of the triplet relationship between behavior, brain activity, and cognitive processes.

Microbiome subcommunity learning with logistic-tree normal latent Dirichlet allocation.

Mixed-membership (MM) models such as latent Dirichlet allocation (LDA) have been applied to microbiome compositional data to identify latent subcommunities of microbial species. These subcommunities are informative for understanding the biological interplay of microbes and for predicting health outcomes. However, microbiome compositions typically display substantial cross-sample heterogeneities in subcommunity compositions-that is, the variability in the proportions of microbes in shared subcommunities across samples-which is not accounted for in prior analyses. As a result, LDA can produce inference, which is highly sensitive to the specification of the number of subcommunities and often divides a single subcommunity into multiple artificial ones. To address this limitation, we incorporate the logistic-tree normal (LTN) model into LDA to form a new MM model. This model allows cross-sample variation in the composition of each subcommunity around some "centroid" composition that defines the subcommunity. Incorporation of auxiliary Pólya-Gamma variables enables a computationally efficient collapsed blocked Gibbs sampler to carry out Bayesian inference under this model. By accounting for such heterogeneity, our new model restores the robustness of the inference in the specification of the number of subcommunities and allows meaningful subcommunities to be identified.

Informing Harmonization Decisions in Integrative Data Analysis: Exploring the Measurement Multiverse.

Combining datasets in an integrative data analysis (IDA) requires researchers to make a number of decisions about how best to harmonize item responses across datasets. This entails two sets of steps: logical harmonization, which involves combining items which appear similar across datasets, and analytic harmonization, which involves using psychometric models to find and account for cross-study differences in measurement. Embedded in logical and analytic harmonization are many decisions, from deciding whether items can be combined prima facie to how best to find covariate effects on specific items. Researchers may not have specific hypotheses about these decisions, and each individual choice may seem arbitrary, but the cumulative effects of these decisions are unknown. In the current study, we conducted an IDA of the relationship between alcohol use and delinquency using three datasets (total N = 2245). For analytic harmonization, we used moderated nonlinear factor analysis (MNLFA) to generate factor scores for delinquency. We conducted both logical and analytic harmonization 72 times, each time making a different set of decisions. We assessed the cumulative influence of these decisions on MNLFA parameter estimates, factor scores, and estimates of the relationship between delinquency and alcohol use. There were differences across paths in MNLFA parameter estimates, but fewer differences in estimates of factor scores and regression parameters linking delinquency to alcohol use. These results suggest that factor scores may be relatively robust to subtly different decisions in data harmonization, and measurement model parameters are less so.

Thinking thrice about sum scores, and then some more about measurement and analysis.

Measurement is fundamental to all research in psychology and should be accorded greater scrutiny than typically occurs. Among other claims, McNeish and Wolf (Thinking twice about sum scores. Behavior Research Methods, 52, 2287-2305) argued that use of sum scores (a) implies that a highly constrained latent variable model underlies items comprising a scale, and (b) may misrepresent or bias relations with other criteria. The central claim by McNeish and Wolf that use of sum scores requires the assumption that a parallel test model underlies item responses is incorrect and without psychometric merit. Instead, if a set of items is unidimensional, estimators of reliability are available even if the factor model underlying the set of items does not have a highly constrained form. Thus, dimensionality of a set of items is the key issue, and whether strict constraints on parameter estimates do or do not hold dictate the appropriate way to estimate reliability. McNeish and Wolf also claimed that more precise forms of scoring, such as estimating factor scores, would be preferable to sum scores. We provide analytic bases for reliability estimation and then provide several demonstrations of reliability estimation and the relative advantages of sum scores and factor scores. We contend that several claims by McNeish and Wolf are questionable and that, as a result, multiple recommendations they made and conclusions they drew are incorrect. The upshot is that, once the dimensional structure of a set of items is verified, sum scores often have a solid psychometric basis and therefore are frequently quite adequate for psychological research.

Mixed-type multivariate response regression with covariance estimation.

We propose a new method for multivariate response regression and covariance estimation when elements of the response vector are of mixed types, for example some continuous and some discrete. Our method is based on a model which assumes the observable mixed-type response vector is connected to a latent multivariate normal response linear regression through a link function. We explore the properties of this model and show its parameters are identifiable under reasonable conditions. We impose no parametric restrictions on the covariance of the latent normal other than positive definiteness, thereby avoiding assumptions about unobservable variables which can be difficult to verify in practice. To accommodate this generality, we propose a novel algorithm for approximate maximum likelihood estimation that works "off-the-shelf" with many different combinations of response types, and which scales well in the dimension of the response vector. Our method typically gives better predictions and parameter estimates than fitting separate models for the different response types and allows for approximate likelihood ratio testing of relevant hypotheses such as independence of responses. The usefulness of the proposed method is illustrated in simulations; and one biomedical and one genomic data example.

A nonlinear dynamic factor model of health and medical treatment.

Quantitative assessments of the relationship between health and medical treatment are of great importance to policy makers. To overcome endogeneity problems we formulate and estimate a tractable dynamic factor model where observed health outcomes are driven by the individual's latent health. The dynamics of latent health reflects both exogenous health deterioration and endogenous health investments. Our model allows us to investigate the effect of medical treatment on current health, as well as on future medical treatment and health outcomes. We estimate the model by maximum simulated likelihood and minimum distance methods using a rich longitudinal data set from Italy obtained by merging a number of administrative archives. These data contain detailed information on medical drug purchase, hospitalization, and mortality for a representative sample of elderly hypertensive patients. Our findings show that the observed autocorrelation in medical treatment reflects both permanent and time-varying observed and unobserved heterogeneity. They also show that medical drug purchase significantly maintains future health levels and prevents transitions to worse health. This suggests that policies aimed at increasing the awareness and the compliance of hypertensive patients help reduce cardiovascular risks and consequent hospitalization and mortality.

Representation Learning for Dynamic Functional Connectivities via Variational Dynamic Graph Latent Variable Models.

Latent variable models (LVMs) for neural population spikes have revealed informative low-dimensional dynamics about the neural data and have become powerful tools for analyzing and interpreting neural activity. However, these approaches are unable to determine the neurophysiological meaning of the inferred latent dynamics. On the other hand, emerging evidence suggests that dynamic functional connectivities (DFC) may be responsible for neural activity patterns underlying cognition or behavior. We are interested in studying how DFC are associated with the low-dimensional structure of neural activities. Most existing LVMs are based on a point process and fail to model evolving relationships. In this work, we introduce a dynamic graph as the latent variable and develop a Variational Dynamic Graph Latent Variable Model (VDGLVM), a representation learning model based on the variational information bottleneck framework. VDGLVM utilizes a graph generative model and a graph neural network to capture dynamic communication between nodes that one has no access to from the observed data. The proposed computational model provides guaranteed behavior-decoding performance and improves LVMs by associating the inferred latent dynamics with probable DFC.

Dynamic latent variable models for the analysis of cognitive abilities in the elderly population.

Cognitive functioning is a key indicator of overall individual health. Identifying factors related to cognitive status, especially in later life, is of major importance. We concentrate on the analysis of the temporal evolution of cognitive abilities in the elderly population. We propose to model the individual cognitive functioning as a multidimensional latent process that accounts also for the effects of individual-specific characteristics (gender, age, and years of education). The proposed model is specified within the generalized linear latent variable framework, and its efficient estimation is obtained using a recent approximation technique, called dimensionwise quadrature. It provides a fast and streamlined approximate inference for complex models, with better or no degradation in accuracy compared with standard techniques. The methodology is applied to the cognitive assessment data from the Health and Retirement Study combined with the Asset and Health Dynamic study in the years between 2006 and 2010. We evaluate the temporal relationship between two dimensions of cognitive functioning, that is, episodic memory and general mental status. We find a substantial influence of the former on the evolution of the latter, as well as evidence of severe consequences on both cognitive abilities among less-educated and older individuals.

History Marginalization Improves Forecasting in Variational Recurrent Neural Networks.

Deep probabilistic time series forecasting models have become an integral part of machine learning. While several powerful generative models have been proposed, we provide evidence that their associated inference models are oftentimes too limited and cause the generative model to predict mode-averaged dynamics. Mode-averaging is problematic since many real-world sequences are highly multi-modal, and their averaged dynamics are unphysical (e.g., predicted taxi trajectories might run through buildings on the street map). To better capture multi-modality, we develop variational dynamic mixtures (VDM): a new variational family to infer sequential latent variables. The VDM approximate posterior at each time step is a mixture density network, whose parameters come from propagating multiple samples through a recurrent architecture. This results in an expressive multi-modal posterior approximation. In an empirical study, we show that VDM outperforms competing approaches on highly multi-modal datasets from different domains.

Probabilistic Models with Deep Neural Networks.

Recent advances in statistical inference have significantly expanded the toolbox of probabilistic modeling. Historically, probabilistic modeling has been constrained to very restricted model classes, where exact or approximate probabilistic inference is feasible. However, developments in variational inference, a general form of approximate probabilistic inference that originated in statistical physics, have enabled probabilistic modeling to overcome these limitations: (i) Approximate probabilistic inference is now possible over a broad class of probabilistic models containing a large number of parameters, and (ii) scalable inference methods based on stochastic gradient descent and distributed computing engines allow probabilistic modeling to be applied to massive data sets. One important practical consequence of these advances is the possibility to include deep neural networks within probabilistic models, thereby capturing complex non-linear stochastic relationships between the random variables. These advances, in conjunction with the release of novel probabilistic modeling toolboxes, have greatly expanded the scope of applications of probabilistic models, and allowed the models to take advantage of the recent strides made by the deep learning community. In this paper, we provide an overview of the main concepts, methods, and tools needed to use deep neural networks within a probabilistic modeling framework.

Inference and Learning in a Latent Variable Model for Beta Distributed Interval Data.

Latent Variable Models (LVMs) are well established tools to accomplish a range of different data processing tasks. Applications exploit the ability of LVMs to identify latent data structure in order to improve data (e.g., through denoising) or to estimate the relation between latent causes and measurements in medical data. In the latter case, LVMs in the form of noisy-OR Bayes nets represent the standard approach to relate binary latents (which represent diseases) to binary observables (which represent symptoms). Bayes nets with binary representation for symptoms may be perceived as a coarse approximation, however. In practice, real disease symptoms can range from absent over mild and intermediate to very severe. Therefore, using diseases/symptoms relations as motivation, we here ask how standard noisy-OR Bayes nets can be generalized to incorporate continuous observables, e.g., variables that model symptom severity in an interval from healthy to pathological. This transition from binary to interval data poses a number of challenges including a transition from a Bernoulli to a Beta distribution to model symptom statistics. While noisy-OR-like approaches are constrained to model how causes determine the observables' mean values, the use of Beta distributions additionally provides (and also requires) that the causes determine the observables' variances. To meet the challenges emerging when generalizing from Bernoulli to Beta distributed observables, we investigate a novel LVM that uses a maximum non-linearity to model how the latents determine means and variances of the observables. Given the model and the goal of likelihood maximization, we then leverage recent theoretical results to derive an Expectation Maximization (EM) algorithm for the suggested LVM. We further show how variational EM can be used to efficiently scale the approach to large networks. Experimental results finally illustrate the efficacy of the proposed model using both synthetic and real data sets. Importantly, we show that the model produces reliable results in estimating causes using proofs of concepts and first tests based on real medical data and on images.

Identifying novel associations in GWAS by hierarchical Bayesian latent variable detection of differentially misclassified phenotypes.

BACKGROUND: Heterogeneity in the definition and measurement of complex diseases in Genome-Wide Association Studies (GWAS) may lead to misdiagnoses and misclassification errors that can significantly impact discovery of disease loci. While well appreciated, almost all analyses of GWAS data consider reported disease phenotype values as is without accounting for potential misclassification. RESULTS: Here, we introduce Phenotype Latent variable Extraction of disease misdiagnosis (PheLEx), a GWAS analysis framework that learns and corrects misclassified phenotypes using structured genotype associations within a dataset. PheLEx consists of a hierarchical Bayesian latent variable model, where inference of differential misclassification is accomplished using filtered genotypes while implementing a full mixed model to account for population structure and genetic relatedness in study populations. Through simulations, we show that the PheLEx framework dramatically improves recovery of the correct disease state when considering realistic allele effect sizes compared to existing methodologies designed for Bayesian recovery of disease phenotypes. We also demonstrate the potential of PheLEx for extracting new potential loci from existing GWAS data by analyzing bipolar disorder and epilepsy phenotypes available from the UK Biobank. From the PheLEx analysis of these data, we identified new candidate disease loci not previously reported for these datasets that have value for supplemental hypothesis generation. CONCLUSION: PheLEx shows promise in reanalyzing GWAS datasets to provide supplemental candidate loci that are ignored by traditional GWAS analysis methodologies.

Latent variable models for harmonization of test scores: A case study on memory.

Combining data from different studies has a long tradition within the scientific community. It requires that the same information is collected from each study to be able to pool individual data. When studies have implemented different methods or used different instruments (e.g., questionnaires) for measuring the same characteristics or constructs, the observed variables need to be harmonized in some way to obtain equivalent content information across studies. This paper formulates the main concepts for harmonizing test scores from different observational studies in terms of latent variable models. The concepts are formulated in terms of calibration, invariance, and exchangeability. Although similar ideas are present in measurement reliability and test equating, harmonization is different from measurement invariance and generalizes test equating. In addition, if a test score needs to be transformed to another test score, harmonization of variables is only possible under specific conditions. Observed test scores that connect all of the different studies, are necessary to be able to test the underlying assumptions of harmonization. The concepts of harmonization are illustrated on multiple memory test scores from three different Canadian studies.

Nonlinear Indicator-Level Moderation in Latent Variable Models.

Linear, nonlinear, and nonparametric moderated latent variable models have been developed to investigate possible interaction effects between a latent variable and an external continuous moderator on the observed indicators in the latent variable model. Most moderation models have focused on moderators that vary across persons but not across the indicators (e.g., moderators like age and socioeconomic status). However, in many applications, the values of the moderator may vary both across persons and across indicators (e.g., moderators like response times and confidence ratings). Indicator-level moderation models are available for categorical moderators and linear interaction effects. However, these approaches require respectively categorization of the continuous moderator and the assumption of linearity of the interaction effect. In this article, parametric nonlinear and nonparametric indicator-level moderation methods are developed. In a simulation study, we demonstrate the viability of these methods. In addition, the methods are applied to a real data set pertaining to arithmetic ability.

Methodological approaches to the analyses of elder abuse screening measures: Application of latent variable measurement modeling to the WC-RAPS.

While several elder abuse screens exist, few measure risk and none target long-term support services. The aims were to examine the psychometric properties of the Weinberg Center Risk and Abuse Prevention Screen (WC-RAPS), comparing approaches to modeling self-reported risk and abuse in relation to reported Adult Protective Services contacts. METHODS: The sample (n = 7,035), admissions to managed long-term care (79%) and short-term rehabilitation (20%), was primarily (66%) female, with mean age 77.6 (SD = 9.10); 7% each were African American and Latino and 12% Asian. Latent variable models were used to examine measurement properties of six indicators of abuse and five of risk. RESULTS: Good model fit and stable subscale measurement models were observed across analyses. Reliability was >0.80 across methods, and concurrent criterion validity estimates were as expected. CONCLUSION: Evidence supported the reliability and concurrent criterion validity of the risk and abuse subscales in an ethnically diverse cohort.

Order selection and sparsity in latent variable models via the ordered factor LASSO.

Generalized linear latent variable models (GLLVMs) offer a general framework for flexibly analyzing data involving multiple responses. When fitting such models, two of the major challenges are selecting the order, that is, the number of factors, and an appropriate structure for the loading matrix, typically a sparse structure. Motivated by the application of GLLVMs to study marine species assemblages in the Southern Ocean, we propose the Ordered Factor LASSO or OFAL penalty for order selection and achieving sparsity in GLLVMs. The OFAL penalty is the first penalty developed specifically for order selection in latent variable models, and achieves this by using a hierarchically structured group LASSO type penalty to shrink entire columns of the loading matrix to zero, while ensuring that non-zero loadings are concentrated on the lower-order factors. Simultaneously, individual element sparsity is achieved through the use of an adaptive LASSO. In conjunction with using an information criterion which promotes aggressive shrinkage, simulation shows that the OFAL penalty performs strongly compared with standard methods and penalties for order selection, achieving sparsity, and prediction in GLLVMs. Applying the OFAL penalty to the Southern Ocean marine species dataset suggests the available environmental predictors explain roughly half of the total covariation between species, thus leading to a smaller number of latent variables and increased sparsity in the loading matrix compared to a model without any covariates.

Revealing unobserved factors underlying cortical activity with a rectified latent variable model applied to neural population recordings.

The activity of sensory cortical neurons is not only driven by external stimuli but also shaped by other sources of input to the cortex. Unlike external stimuli, these other sources of input are challenging to experimentally control, or even observe, and as a result contribute to variability of neural responses to sensory stimuli. However, such sources of input are likely not "noise" and may play an integral role in sensory cortex function. Here we introduce the rectified latent variable model (RLVM) in order to identify these sources of input using simultaneously recorded cortical neuron populations. The RLVM is novel in that it employs nonnegative (rectified) latent variables and is much less restrictive in the mathematical constraints on solutions because of the use of an autoencoder neural network to initialize model parameters. We show that the RLVM outperforms principal component analysis, factor analysis, and independent component analysis, using simulated data across a range of conditions. We then apply this model to two-photon imaging of hundreds of simultaneously recorded neurons in mouse primary somatosensory cortex during a tactile discrimination task. Across many experiments, the RLVM identifies latent variables related to both the tactile stimulation as well as nonstimulus aspects of the behavioral task, with a majority of activity explained by the latter. These results suggest that properly identifying such latent variables is necessary for a full understanding of sensory cortical function and demonstrate novel methods for leveraging large population recordings to this end.NEW & NOTEWORTHY The rapid development of neural recording technologies presents new opportunities for understanding patterns of activity across neural populations. Here we show how a latent variable model with appropriate nonlinear form can be used to identify sources of input to a neural population and infer their time courses. Furthermore, we demonstrate how these sources are related to behavioral contexts outside of direct experimental control.

The general relationship between internalizing psychopathology and chronic physical health conditions: a population-based study.

Studies have consistently demonstrated a reciprocal relationship between internalizing disorders and several chronic physical health conditions. Yet, much of the extant literature fails to take into account the role of comorbidity among internalizing disorders when examining the relationship with poor physical health. The current study applied latent variable modelling to investigate the shared and specific relationships between internalizing (fear and distress factors) and a range of physical health conditions. Data comprised 8841 respondents aged 16-85 years who took part in the 2007 Australian National Survey of Mental Health and Wellbeing. Multiple indicator, multiple causes models were used to parse the shared and specific relationships between internalizing disorders and variables associated with poor physical health. The study found that several physical conditions were significantly related to mean levels of fear and distress. The results were broadly similar but minor differences emerged depending on whether lifetime or past 12 months indicators of mental disorders and physical conditions were utilized in the model. Finally, the results demonstrated that the association between individual mental disorders and physical health conditions are better accounted for by indirect relationships with broad transdiagnostic dimensions rather than including additional disorder-specific relationships. The results indicate that researchers should focus on common mechanisms across multiple internalizing disorders and poor physical health when developing prevention and treatment initiatives.

Nonlinear Analysis to Detect if Excellent Nursing Work Environments Have Highest Well-Being.

PURPOSE: To detect potentially nonlinear associations between nurses' work environment and nurse staffing on the one hand and nurse burnout on the other hand. DESIGN: A cross-sectional multicountry study for which data collection using a survey of 33,731 registered nurses in 12 European countries took place during 2009 to 2010. METHODS: A semiparametric latent variable model that describes both linear and potentially nonlinear associations between burnout (Maslach Burnout Inventory: emotional exhaustion, depersonalization, personal accomplishment) and work environment (Practice Environment Scale of the Nursing Work Index: managerial support for nursing, doctor-nurse collegial relations, promotion of care quality) and staffing (patient-to-nurse ratio). FINDINGS: Similar conclusions are reached from linear and nonlinear models estimating the association between work environment and burnout. For staffing, an increase in the patient-to-nurse ratio is associated with an increase in emotional exhaustion. At about 15 patients per nurse, no further increase in emotional exhaustion is seen. CONCLUSIONS: Absence of evidence for diminishing returns of improving work environments suggests that continuous improvement and achieving excellence in nurse work environments pays off strongly in terms of lower nurse-reported burnout rates. Nurse staffing policy would benefit from a larger number of studies that identify specific minimum as well as maximum thresholds at which inputs affect nurse and patient outcomes. CLINICAL RELEVANCE: Nurse burnout is omnipresent and has previously been shown to be related to worse patient outcomes. Additional increments in characteristics of excellent work environments, up to the highest possible standard, correspond to lower nurse burnout.

Measuring individual differences in responses to date-rape vignettes using latent variable models.

Vignette methodology can be a flexible and powerful way to examine individual differences in response to dangerous real-life scenarios. However, most studies underutilize the usefulness of such methodology by analyzing only one outcome, which limits the ability to track event-related changes (e.g., vacillation in risk perception). The current study was designed to illustrate the dynamic influence of risk perception on exit point from a date-rape vignette. Our primary goal was to provide an illustrative example of how to use latent variable models for vignette methodology, including latent growth curve modeling with piecewise slopes, as well as latent variable measurement models. Through the combination of a step-by-step exposition in this text and corresponding model syntax available electronically, we detail an alternative statistical "blueprint" to enhance future violence research efforts using vignette methodology. Aggr. Behav. 43:60-73, 2017. © 2016 Wiley Periodicals, Inc.