HMM for discovering decision-making dynamics using reinforcement learning experiments.

Major depressive disorder (MDD), a leading cause of years of life lived with disability, presents challenges in diagnosis and treatment due to its complex and heterogeneous nature. Emerging evidence indicates that reward processing abnormalities may serve as a behavioral marker for MDD. To measure reward processing, patients perform computer-based behavioral tasks that involve making choices or responding to stimulants that are associated with different outcomes, such as gains or losses in the laboratory. Reinforcement learning (RL) models are fitted to extract parameters that measure various aspects of reward processing (e.g. reward sensitivity) to characterize how patients make decisions in behavioral tasks. Recent findings suggest the inadequacy of characterizing reward learning solely based on a single RL model; instead, there may be a switching of decision-making processes between multiple strategies. An important scientific question is how the dynamics of strategies in decision-making affect the reward learning ability of individuals with MDD. Motivated by the probabilistic reward task within the Establishing Moderators and Biosignatures of Antidepressant Response in Clinical Care (EMBARC) study, we propose a novel RL-HMM (hidden Markov model) framework for analyzing reward-based decision-making. Our model accommodates decision-making strategy switching between two distinct approaches under an HMM: subjects making decisions based on the RL model or opting for random choices. We account for continuous RL state space and allow time-varying transition probabilities in the HMM. We introduce a computationally efficient Expectation-maximization (EM) algorithm for parameter estimation and use a nonparametric bootstrap for inference. Extensive simulation studies validate the finite-sample performance of our method. We apply our approach to the EMBARC study to show that MDD patients are less engaged in RL compared to the healthy controls, and engagement is associated with brain activities in the negative affect circuitry during an emotional conflict task.

Change in food choice during acute treatment and the effect on longer-term outcome in patients with anorexia nervosa.

BACKGROUND: Restriction of food intake is a central pathological feature of anorexia nervosa (AN). Maladaptive eating behavior and, specifically, limited intake of calorie-dense foods are resistant to change and contribute to poor long-term outcomes. This study is a preliminary examination of whether change in food choices during inpatient treatment is related to longer-term clinical course. METHODS: Individuals with AN completed a computerized Food Choice Task at the beginning and end of inpatient treatment to determine changes in high-fat and self-controlled food choices. Linear regression and longitudinal analyses tested whether change in task behavior predicted short-term outcome (body mass index [BMI] at discharge) and longer-term outcome (BMI and eating disorder psychopathology). RESULTS: Among 88 patients with AN, BMI improved significantly with hospital treatment (p < 0.001), but Food Choice Task outcomes did not change significantly. Change in high-fat and self-controlled choices was not associated with BMI at discharge (r = 0.13, p = 0.22 and r = 0.10, p = 0.39, respectively). An increase in the proportion of high-fat foods selected (ß = 0.91, p = 0.02) and a decrease in the use of self-control (ß = -1.50, p = 0.001) predicted less decline in BMI over 3 years after discharge. CONCLUSIONS: Short-term treatment is associated with improvement in BMI but with no significant change, on average, in choices made in a task known to predict actual eating. However, the degree to which individuals increased high-fat choices during treatment and decreased the use of self-control over food choice were associated with reduced weight loss over the following 3 years, underscoring the need to focus on changing eating behavior in treatment of AN.

Identifying temporal pathways using biomarkers in the presence of latent non-Gaussian components.

Time-series data collected from a network of random variables are useful for identifying temporal pathways among the network nodes. Observed measurements may contain multiple sources of signals and noises, including Gaussian signals of interest and non-Gaussian noises, including artifacts, structured noise, and other unobserved factors (eg, genetic risk factors, disease susceptibility). Existing methods, including vector autoregression (VAR) and dynamic causal modeling do not account for unobserved non-Gaussian components. Furthermore, existing methods cannot effectively distinguish contemporaneous relationships from temporal relations. In this work, we propose a novel method to identify latent temporal pathways using time-series biomarker data collected from multiple subjects. The model adjusts for the non-Gaussian components and separates the temporal network from the contemporaneous network. Specifically, an independent component analysis (ICA) is used to extract the unobserved non-Gaussian components, and residuals are used to estimate the contemporaneous and temporal networks among the node variables based on method of moments. The algorithm is fast and can easily scale up. We derive the identifiability and the asymptotic properties of the temporal and contemporaneous networks. We demonstrate superior performance of our method by extensive simulations and an application to a study of attention-deficit/hyperactivity disorder (ADHD), where we analyze the temporal relationships between brain regional biomarkers. We find that temporal network edges were across different brain regions, while most contemporaneous network edges were bilateral between the same regions and belong to a subset of the functional connectivity network.

Temporal generative models for learning heterogeneous group dynamics of ecological momentary assessment data.

One of the goals of precision psychiatry is to characterize mental disorders in an individualized manner, taking into account the underlying dynamic processes. Recent advances in mobile technologies have enabled the collection of ecological momentary assessments that capture multiple responses in real-time at high frequency. However, ecological momentary assessment data are often multi-dimensional, correlated, and hierarchical. Mixed-effect models are commonly used but may require restrictive assumptions about the fixed and random effects and the correlation structure. The recurrent temporal restricted Boltzmann machine (RTRBM) is a generative neural network that can be used to model temporal data, but most existing RTRBM approaches do not account for the potential heterogeneity of group dynamics within a population based on available covariates. In this paper, we propose a new temporal generative model, the HDRBM, to learn the heterogeneous group dynamics and demonstrate the effectiveness of this approach on simulated and real-world ecological momentary assessment datasets. We show that by incorporating covariates, HDRBM can improve accuracy and interpretability, explore the underlying drivers of the group dynamics of participants, and serve as a generative model for ecological momentary assessment studies.

Learning optimal biomarker-guided treatment policy for chronic disorders.

Electroencephalogram (EEG) provides noninvasive measures of brain activity and is found to be valuable for the diagnosis of some chronic disorders. Specifically, pre-treatment EEG signals in the alpha and theta frequency bands have demonstrated some association with antidepressant response, which is well-known to have a low response rate. We aim to design an integrated pipeline that improves the response rate of patients with major depressive disorder by developing a treatment policy guided by the resting state pre-treatment EEG recordings and other treatment effects modifiers. First, we design an innovative automatic site-specific EEG preprocessing pipeline to extract features with stronger signals than raw data. We then estimate the conditional average treatment effect (CATE) using causal forests and use a doubly robust technique to improve efficiency in the estimation of the average treatment effect. We present evidence of heterogeneity in the treatment effect and the modifying power of the EEG features, as well as a significant average treatment effect, a result that cannot be obtained with conventional methods. Finally, we employ an efficient policy learning algorithm to learn an optimal depth-2 treatment assignment decision tree and compare its performance with Q-Learning and outcome-weighted learning via simulation studies and an application to a large multi-site, double-blind, randomized controlled clinical trial, EMBARC.

Dynamic undirected graphical models for time-varying clinical symptom and neuroimaging networks.

In this work, we propose methods to examine how the complex interrelationships between clinical symptoms and, separately, brain imaging biomarkers change over time leading up to the diagnosis of a disease in subjects with a known genetic near-certainty of disease. We propose a time-dependent undirected graphical model that ensures temporal and structural smoothness across time-specific networks to examine the trajectories of interactions between markers aligned at the time of disease onset. Specifically, we anchor subjects relative to the time of disease diagnosis (anchoring time) as in a revival process, and we estimate networks at each time point of interest relative to the anchoring time. To use all available data, we apply kernel weights to borrow information across observations that are close to the time of interest. Adaptive lasso weights are introduced to encourage temporal smoothness in edge strength, while a novel elastic fused- l 0 $$ {l}_0 $$ penalty removes spurious edges and encourages temporal smoothness in network structure. Our approach can handle practical complications such as unbalanced visit times. We conduct simulation studies to compare our approach with existing methods. We then apply our method to data from PREDICT-HD, a large prospective observational study of pre-manifest Huntington's disease (HD) patients, to identify symptom and imaging network changes that precede clinical diagnosis of HD.

Eligibility for Substance Use Clinical Trials Among Emergency Psychiatry Patients: The Impact of Exclusion Criteria.

Objective: The first objective was to identify common exclusion criteria used in clinical trials. The second objective was to quantify the degree to which these criteria exclude emergency psychiatry patients. Methods: Qualitative Content Analysis was used for the first objective, identifying common exclusion criteria used in recent high-impact substance use clinical trials. A retrospective record review was used for the second objective, which examined the frequency of these exclusion criteria in a 1-month sample of adults receiving psychiatric evaluation in an emergency department. Results: Most trials had exclusions for co-occurring psychiatric problems (76.6%), medical problems (74.0%), prior or current treatment (72.7%), motivation for change (61.1%), pregnancy or lactation (57.1%), or using other specified substances of abuse (54.6%). In the clinical sample, exclusions for co-occurring psychiatric problems would make 94.7% of patients ineligible. Other exclusions had a combined effect of making 76% of patients ineligible. Conclusions: Clinical trials using typical exclusion criteria exclude nearly all emergency psychiatry patients with substance use problems.

Multilayer Exponential Family Factor models for integrative analysis and learning disease progression.

Current diagnosis of neurological disorders often relies on late-stage clinical symptoms, which poses barriers to developing effective interventions at the premanifest stage. Recent research suggests that biomarkers and subtle changes in clinical markers may occur in a time-ordered fashion and can be used as indicators of early disease. In this article, we tackle the challenges to leverage multidomain markers to learn early disease progression of neurological disorders. We propose to integrate heterogeneous types of measures from multiple domains (e.g., discrete clinical symptoms, ordinal cognitive markers, continuous neuroimaging, and blood biomarkers) using a hierarchical Multilayer Exponential Family Factor (MEFF) model, where the observations follow exponential family distributions with lower-dimensional latent factors. The latent factors are decomposed into shared factors across multiple domains and domain-specific factors, where the shared factors provide robust information to perform extensive phenotyping and partition patients into clinically meaningful and biologically homogeneous subgroups. Domain-specific factors capture remaining unique variations for each domain. The MEFF model also captures nonlinear trajectory of disease progression and orders critical events of neurodegeneration measured by each marker. To overcome computational challenges, we fit our model by approximate inference techniques for large-scale data. We apply the developed method to Parkinson's Progression Markers Initiative data to integrate biological, clinical, and cognitive markers arising from heterogeneous distributions. The model learns lower-dimensional representations of Parkinson's disease (PD) and the temporal ordering of the neurodegeneration of PD.

Semiparametric regression analysis of bivariate censored events in a family study of Alzheimer's disease.

Assessing disease comorbidity patterns in families represents the first step in gene mapping for diseases and is central to the practice of precision medicine. One way to evaluate the relative contributions of genetic risk factor and environmental determinants of a complex trait (e.g., Alzheimer's disease [AD]) and its comorbidities (e.g., cardiovascular diseases [CVD]) is through familial studies, where an initial cohort of subjects are recruited, genotyped for specific loci, and interviewed to provide extensive disease history in family members. Because of the retrospective nature of obtaining disease phenotypes in family members, the exact time of disease onset may not be available such that current status data or interval-censored data are observed. All existing methods for analyzing these family study data assume single event subject to right-censoring so are not applicable. In this article, we propose a semiparametric regression model for the family history data that assumes a family-specific random effect and individual random effects to account for the dependence due to shared environmental exposures and unobserved genetic relatedness, respectively. To incorporate multiple events, we jointly model the onset of the primary disease of interest and a secondary disease outcome that is subject to interval-censoring. We propose nonparametric maximum likelihood estimation and develop a stable Expectation-Maximization (EM) algorithm for computation. We establish the asymptotic properties of the resulting estimators and examine the performance of the proposed methods through simulation studies. Our application to a real world study reveals that the main contribution of comorbidity between AD and CVD is due to genetic factors instead of environmental factors.

A general framework of nonparametric feature selection in high-dimensional data.

Nonparametric feature selection for high-dimensional data is an important and challenging problem in the fields of statistics and machine learning. Most of the existing methods for feature selection focus on parametric or additive models which may suffer from model misspecification. In this paper, we propose a new framework to perform nonparametric feature selection for both regression and classification problems. Under this framework, we learn prediction functions through empirical risk minimization over a reproducing kernel Hilbert space. The space is generated by a novel tensor product kernel, which depends on a set of parameters that determines the importance of the features. Computationally, we minimize the empirical risk with a penalty to estimate the prediction and kernel parameters simultaneously. The solution can be obtained by iteratively solving convex optimization problems. We study the theoretical property of the kernel feature space and prove the oracle selection property and Fisher consistency of our proposed method. Finally, we demonstrate the superior performance of our approach compared to existing methods via extensive simulation studies and applications to two real studies.

A latent state space model for estimating brain dynamics from electroencephalogram (EEG) data.

Modern neuroimaging technologies have substantially advanced the measurement of brain activity. Electroencephalogram (EEG) as a noninvasive neuroimaging technique measures changes in electrical voltage on the scalp induced by brain cortical activity. With its high temporal resolution, EEG has emerged as an increasingly useful tool to study brain connectivity. Challenges with modeling EEG signals of complex brain activity include interactions among unknown sources, low signal-to-noise ratio, and substantial between-subject heterogeneity. In this work, we propose a state space model that jointly analyzes multichannel EEG signals and learns dynamics of different sources corresponding to brain cortical activity. Our model borrows strength from spatially correlated measurements and uses low-dimensional latent states to explain all observed channels. The model can account for patient heterogeneity and quantify the effect of a subject's covariates on the latent space. The EM algorithm, Kalman filtering, and bootstrap resampling are used to fit the state space model and provide comparisons between patient diagnostic groups. We apply the developed approach to a case-control study of alcoholism and reveal significant attenuation of brain activity in response to visual stimuli in alcoholic subjects compared to healthy controls.

Estimating disease onset from change points of markers measured with error.

Huntington disease is an autosomal dominant, neurodegenerative disease without clearly identified biomarkers for when motor-onset occurs. Current standards to determine motor-onset rely on a clinician's subjective judgment that a patient's extrapyramidal signs are unequivocally associated with Huntington disease. This subjectivity can lead to error which could be overcome using an objective, data-driven metric that determines motor-onset. Recent studies of motor-sign decline-the longitudinal degeneration of motor-ability in patients-have revealed that motor-onset is closely related to an inflection point in its longitudinal trajectory. We propose a nonlinear location-shift marker model that captures this motor-sign decline and assesses how its inflection point is linked to other markers of Huntington disease progression. We propose two estimating procedures to estimate this model and its inflection point: one is a parametric method using nonlinear mixed effects model and the other one is a multi-stage nonparametric approach, which we developed. In an empirical study, the parametric approach was sensitive to correct specification of the mean structure of the longitudinal data. In contrast, our multi-stage nonparametric procedure consistently produced unbiased estimates regardless of the true mean structure. Applying our multi-stage nonparametric estimator to Neurobiological Predictors of Huntington Disease, a large observational study of Huntington disease, leads to earlier prediction of motor-onset compared to the clinician's subjective judgment.

Conditional Gaussian graphical model for estimating personalized disease symptom networks.

The co-occurrence of symptoms may result from the direct interactions between these symptoms and the symptoms can be treated as a system. In addition, subject-specific risk factors (eg, genetic variants, age) can also exert external influence on the system. In this work, we develop a covariate-dependent conditional Gaussian graphical model to obtain personalized symptom networks. The strengths of network connections are modeled as a function of covariates to capture the heterogeneity among individuals and subgroups of individuals. We assess the performance of our proposed method by simulation studies and an application to a large natural history study of Huntington's disease to investigate the networks of symptoms in multiple clinical domains (motor, cognitive, psychiatric) and identify important brain imaging biomarkers that are associated with the connections. We show that the symptoms in the same clinical domain interact more often with each other than cross domains and the psychiatric subnetwork is the densest network. We validate the findings using the subjects' symptom measurements at follow-up visits.

Self-matched learning to construct treatment decision rules from electronic health records.

Electronic health records (EHRs) collected from large-scale health systems provide rich subject-specific information on a broad patient population at a lower cost compared to randomized controlled trials. Thus, EHRs may serve as a complementary resource to provide real-world data to construct individualized treatment rules (ITRs) and achieve precision medicine. However, in the absence of randomization, inferring treatment rules from EHR data may suffer from unmeasured confounding. In this article, we propose a self-matched learning method inspired by the self-controlled case series (SCCS) design to mitigate this challenge. We alleviate unmeasured time-invariant confounding between patients by matching different periods of treatments within the same patient (self-controlled matching) to infer the optimal ITRs. The proposed method constructs a within-subject matched value function for optimizing ITRs and bears similarity to the SCCS design. We examine assumptions that ensure Fisher consistency, and show that our method requires weaker assumptions on unmeasured confounding than alternative methods. Through extensive simulation studies, we demonstrate that self-matched learning has comparable performance to other existing methods when there are no unmeasured confounders, but performs markedly better when unobserved time-invariant confounders are present, which is often the case for EHRs. Sensitivity analyses show that the proposed method is robust under different scenarios. Finally, we apply self-matched learning to estimate the optimal ITRs from type 2 diabetes patient EHRs, which shows our estimated decision rules lead to greater advantages in reducing patients' diabetes-related complications.

Evaluating effectiveness of public health intervention strategies for mitigating COVID-19 pandemic.

Coronavirus disease 2019 (COVID-19) pandemic is an unprecedented global public health challenge. In the United States (US), state governments have implemented various non-pharmaceutical interventions (NPIs), such as physical distance closure (lockdown), stay-at-home order, mandatory facial mask in public in response to the rapid spread of COVID-19. To evaluate the effectiveness of these NPIs, we propose a nested case-control design with propensity score weighting under the quasi-experiment framework to estimate the average intervention effect on disease transmission across states. We further develop a method to test for factors that moderate intervention effect to assist precision public health intervention. Our method takes account of the underlying dynamics of disease transmission and balance state-level pre-intervention characteristics. We prove that our estimator provides causal intervention effect under assumptions. We apply this method to analyze US COVID-19 incidence cases to estimate the effects of six interventions. We show that lockdown has the largest effect on reducing transmission and reopening bars significantly increase transmission. States with a higher percentage of non-White population are at greater risk of increased R t $$ {R}_t $$ associated with reopening bars.

Optimizing relapse prevention and changing habits (REACH+) in anorexia nervosa.

INTRODUCTION: Relapse rates in anorexia nervosa (AN) are high, even after full weight restoration. This study aims to develop a relapse prevention treatment that specifically addresses persistent maladaptive behaviors (habits). Relapse Prevention and Changing Habits (REACH+) aims to support patients in developing routines that promote weight maintenance, encourage health, and challenge habits that perpetuate illness. The clinical trial design uses the Multiphase Optimization STrategy (MOST) framework to efficiently identify which components of treatment contribute to positive outcomes. METHODS: Participants will be 60 adults with AN who have achieved weight restoration in an inpatient setting. Treatment will consist of 6 months of outpatient telehealth sessions. REACH+ consists of behavior, cognitive, and motivation components, as well as food monitoring and a skill consolidation phase. A specialized online platform extends therapy between sessions. Participants will be randomly assigned to different versions of each component in a fractional factorial design. Outcomes will focus on maintenance of remission, measured by rate of weight loss and end-of-trial status. Interventions that contribute to remission will be included in an optimized treatment package, suitable for a large-scale clinical trial of relapse prevention in AN.

Constructing disease onset signatures using multi-dimensional network-structured biomarkers.

Potential disease-modifying therapies for neurodegenerative disorders need to be introduced prior to the symptomatic stage in order to be effective. However, current diagnosis of neurological disorders mostly rely on measurements of clinical symptoms and thus only identify symptomatic subjects in their late disease course. Thus, it is of interest to select and integrate biomarkers that may reflect early disease-related pathological changes for earlier diagnosis and recruiting pre-sypmtomatic subjects in a prevention clinical trial. Two sources of biological information are relevant to the construction of biomarker signatures for time to disease onset that is subject to right censoring. First, biomarkers' effects on disease onset may vary with a subject's baseline disease stage indicated by a particular marker. Second, biomarkers may be connected through networks, and their effects on disease may be informed by this network structure. To leverage these information, we propose a varying-coefficient hazards model to induce double smoothness over the dimension of the disease stage and over the space of network-structured biomarkers. The distinctive feature of the model is a non-parametric effect that captures non-linear change according to the disease stage and similarity among the effects of linked biomarkers. For estimation and feature selection, we use kernel smoothing of a regularized local partial likelihood and derive an efficient algorithm. Numeric simulations demonstrate significant improvements over existing methods in performance and computational efficiency. Finally, the methods are applied to our motivating study, a recently completed study of Huntington's disease (HD), where structural brain imaging measures are used to inform age-at-onset of HD and assist clinical trial design. The analysis offers new insights on the structural network signatures for premanifest HD subjects.

Age of Onset of Huntington's Disease in Carriers of Reduced Penetrance Alleles.

BACKGROUND: Age of manifest Huntington's disease (HD) onset correlates with number of CAG repeats in the huntingtin gene. Little is known about onset with 36 to 39 repeats, the "reduced penetrance" (RP) range. OBJECTIVES: We provide allele-specific estimates of HD penetrance (diagnostic confidence level of 4) for RP allele carriers. METHODS: We analyzed 431 pre-manifest RP allele carriers from Enroll-HD, the largest prospective observational HD study. Cumulative penetrance (CP) was estimated from Kaplan-Meier curves. RESULTS: No one with 36 repeats (n = 25) phenoconverted. CP for 38 repeats (n = 120) was 32% (95% confidence interval [CI] 0%-55%) and 51% (CI, 10%-73%) by ages 70 and 75, respectively, and 68% (CI, 46%-81%) and 81% (CI, 58%-92%) by ages 70 and 75 for 39 repeats (n = 253). CP was not estimable at those ages for 37 repeats (n = 33). CONCLUSIONS: Differences by RP-range repeat length did not reach significance with a 3-year median follow-up duration among censored individuals. © 2021 International Parkinson and Movement Disorder Society.

Learning latent heterogeneity for type 2 diabetes patients using longitudinal health markers in electronic health records.

Electronic health records (EHRs) from type 2 diabetes (T2D) patients consist of longitudinally and sparsely measured health markers at clinical encounters. Our goal is to use such data to learn latent patterns that can inform patient's health status related to T2D while accounting for challenges in retrospectively collected EHRs. To handle challenges such as correlated longitudinal measurements, irregular and informative encounter times, and mixed marker types, we propose multivariate generalized linear models to learn latent patient subgroups. In our model, covariate effects were time-dependent and latent Gaussian processes were introduced to model between-marker correlations over time. Using inferred latent processes, we integrated the irregularly measured health markers of mixed types into composite scores and applied hierarchical clustering to learn latent subgroup structures among T2D patients. Application to an EHR dataset of T2D patients showed different trends of age, sex, and race effects on hypertension/high blood pressure, total cholesterol, glycated hemoglobin, high-density lipoprotein, and medications. The associations among these markers varied over time during the study window. Clustering results revealed four subgroups, each with distinct health status. The same patterns were further confirmed using new EHR records of the same cohort. We developed a novel latent model to integrate longitudinal health markers in EHRs and characterize patient latent heterogeneities. Analysis indicated that there were distinct subgroups of T2D patients, suggesting that effective healthcare managements for these patients should be performed separately for each subgroup.

Liver Disease: Induction, Progression, Immunological Mechanisms, and Therapeutic Interventions.

The liver is an organ with impressive regenerative potential and has been shown to heal sizable portions after their removal. However, certain diseases can overstimulate its potential to self-heal and cause excessive cellular matrix and collagen buildup. Decompensation of liver fibrosis leads to cirrhosis, a buildup of fibrotic ECM that impedes the liver's ability to efficiently exchange fluid. This review summarizes the complex immunological activities in different liver diseases, and how failure to maintain liver homeostasis leads to progressive fibrotic tissue development. We also discuss a variety of pathologies that lead to liver cirrhosis, such as alcoholic liver disease and chronic hepatitis B virus (HBV). Mesenchymal stem cells are widely studied for their potential in tissue replacement and engineering. Herein, we discuss the potential of MSCs to regulate immune response and alter the disease state. Substantial efforts have been performed in preclinical animal testing, showing promising results following inhibition of host immunity. Finally, we outline the current state of clinical trials with mesenchymal stem cells and other cellular and non-cellular therapies as they relate to the detection and treatment of liver cirrhosis.