Visualization, benchmarking and characterization of nested single-cell heterogeneity as dynamic forest mixtures.

A major topic of debate in developmental biology centers on whether development is continuous, discontinuous, or a mixture of both. Pseudo-time trajectory models, optimal for visualizing cellular progression, model cell transitions as continuous state manifolds and do not explicitly model real-time, complex, heterogeneous systems and are challenging for benchmarking with temporal models. We present a data-driven framework that addresses these limitations with temporal single-cell data collected at discrete time points as inputs and a mixture of dependent minimum spanning trees (MSTs) as outputs, denoted as dynamic spanning forest mixtures (DSFMix). DSFMix uses decision-tree models to select genes that account for variations in multimodality, skewness and time. The genes are subsequently used to build the forest using tree agglomerative hierarchical clustering and dynamic branch cutting. We first motivate the use of forest-based algorithms compared to single-tree approaches for visualizing and characterizing developmental processes. We next benchmark DSFMix to pseudo-time and temporal approaches in terms of feature selection, time correlation, and network similarity. Finally, we demonstrate how DSFMix can be used to visualize, compare and characterize complex relationships during biological processes such as epithelial-mesenchymal transition, spermatogenesis, stem cell pluripotency, early transcriptional response from hormones and immune response to coronavirus disease. Our results indicate that the expression of genes during normal development exhibits a high proportion of non-uniformly distributed profiles that are mostly right-skewed and multimodal; the latter being a characteristic of major steady states during development. Our study also identifies and validates gene signatures driving complex dynamic processes during somatic or germline differentiation.

Linking within- and between-host scales for understanding the evolutionary dynamics of quantitative antimicrobial resistance.

Understanding both the epidemiological and evolutionary dynamics of antimicrobial resistance is a major public health concern. In this paper, we propose a nested model, explicitly linking the within- and between-host scales, in which the level of resistance of the bacterial population is viewed as a continuous quantitative trait. The within-host dynamics is based on integro-differential equations structured by the resistance level, while the between-host scale is additionally structured by the time since infection. This model simultaneously captures the dynamics of the bacteria population, the evolutionary transient dynamics which lead to the emergence of resistance, and the epidemic dynamics of the host population. Moreover, we precisely analyze the model proposed by particularly performing the uniform persistence and global asymptotic results. Finally, we discuss the impact of the treatment rate of the host population in controlling both the epidemic outbreak and the average level of resistance, either if the within-host scale therapy is a success or failure. We also explore how transitions between infected populations (treated and untreated) can impact the average level of resistance, particularly in a scenario where the treatment is successful at the within-host scale.

A Modified Net Reclassification Improvement Statistic.

The continuous net reclassification improvement (NRI) statistic is a popular model change measure that was developed to assess the incremental value of new factors in a risk prediction model. Two prominent statistical issues identified in the literature call the utility of this measure into question: (1) it is not a proper scoring function and (2) it has a high false positive rate when testing whether new factors contribute to the risk model. For binary response regression models, these subjects are interrogated and a modification of the continuous NRI, guided by the likelihood-based score residual, is proposed to address these issues. Within a nested model framework, the modified NRI may be viewed as a distance measure between two risk models. An application of the modified NRI is illustrated using prostate cancer data.

Comparing logit & probit coefficients between nested models.

Social scientists are often interested in seeing how the estimated effects of variables change once other variables are controlled for. For example, a simple analysis may reveal that income differs by race - but why does it differ? To answer such a question, a researcher might estimate a model where race is the only independent variable, and then add variables such as education to subsequent models. If the original estimated effect of race declines, this may be because race affects education, which in turn affects income. What is not universally realized is that the interpretation of such nested models can be problematic when logit or probit techniques are employed with binary dependent variables. Naïve comparisons of coefficients between models can indicate differences where none exist, hide differences that do exist, and even show differences in the opposite direction of what actually exists. We discuss why problems occur and illustrate their potential consequences. Proposed solutions, such as Linear Probability Models, Y-standardization, the Karlson/Holm/Breen method, and marginal effects, are explained and evaluated.

Instrumental variable estimation for a time-varying treatment and a time-to-event outcome via structural nested cumulative failure time models.

BACKGROUND: In many applications of instrumental variable (IV) methods, the treatments of interest are intrinsically time-varying and outcomes of interest are failure time outcomes. A common example is Mendelian randomization (MR), which uses genetic variants as proposed IVs. In this article, we present a novel application of g-estimation of structural nested cumulative failure models (SNCFTMs), which can accommodate multiple measures of a time-varying treatment when modelling a failure time outcome in an IV analysis. METHODS: A SNCFTM models the ratio of two conditional mean counterfactual outcomes at time k under two treatment strategies which differ only at an earlier time m. These models can be extended to accommodate inverse probability of censoring weights, and can be applied to case-control data. We also describe how the g-estimates of the SNCFTM parameters can be used to calculate marginal cumulative risks under nondynamic treatment strategies. We examine the performance of this method using simulated data, and present an application of these models by conducting an MR study of alcohol intake and endometrial cancer using longitudinal observational data from the Nurses' Health Study. RESULTS: Our simulations found that estimates from SNCFTMs which used an IV approach were similar to those obtained from SNCFTMs which adjusted for confounders, and similar to those obtained from the g-formula approach when the outcome was rare. In our data application, the cumulative risk of endometrial cancer from age 45 to age 72 under the "never drink" strategy (4.0%) was similar to that under the "always ½ drink per day" strategy (4.3%). CONCLUSIONS: SNCFTMs can be used to conduct MR and other IV analyses with time-varying treatments and failure time outcomes.

The added value of new covariates to the brier score in cox survival models.

Calibration is an important measure of the predictive accuracy for a prognostic risk model. A widely used measure of calibration when the outcome is survival time is the expected Brier score. In this paper, methodology is developed to accurately estimate the difference in expected Brier scores derived from nested survival models and to compute an accompanying variance estimate of this difference. The methodology is applicable to time invariant and time-varying coefficient Cox survival models. The nested survival model approach is often applied to the scenario where the full model consists of conventional and new covariates and the subset model contains the conventional covariates alone. A complicating factor in the methodologic development is that the Cox model specification cannot, in general, be simultaneously satisfied for nested models. The problem has been resolved by projecting the properly specified full survival model onto the lower dimensional space of conventional markers alone. Simulations are performed to examine the method's finite sample properties and a prostate cancer data set is used to illustrate its application.

Model selection for G-estimation of dynamic treatment regimes.

Dynamic treatment regimes (DTRs) aim to formalize personalized medicine by tailoring treatment decisions to individual patient characteristics. G-estimation for DTR identification targets the parameters of a structural nested mean model, known as the blip function, from which the optimal DTR is derived. Despite its potential, G-estimation has not seen widespread use in the literature, owing in part to its often complex presentation and implementation, but also due to the necessity for correct specification of the blip. Using a quadratic approximation approach inspired by iteratively reweighted least squares, we derive a quasi-likelihood function for G-estimation within the DTR framework, and show how it can be used to form an information criterion for blip model selection. We outline the theoretical properties of this model selection criterion and demonstrate its application in a variety of simulation studies as well as in data from the Sequenced Treatment Alternatives to Relieve Depression study.

Accounting for established predictors with the multistep elastic net.

Multivariable models for prediction or estimating associations with an outcome are rarely built in isolation. Instead, they are based upon a mixture of covariates that have been evaluated in earlier studies (eg, age, sex, or common biomarkers) and covariates that were collected specifically for the current study (eg, a panel of novel biomarkers or other hypothesized risk factors). For that context, we present the multistep elastic net (MSN), which considers penalized regression with variables that can be qualitatively grouped based upon their degree of prior research support: established predictors vs unestablished predictors. The MSN chooses between uniform penalization of all predictors (the standard elastic net) and weaker penalization of the established predictors in a cross-validated framework and includes the option to impose zero penalty on the established predictors. In simulation studies that reflect the motivating context, we show the comparability or superiority of the MSN over the standard elastic net, the Integrative LASSO with Penalty Factors, the sparse group lasso, and the group lasso, and we investigate the importance of not penalizing the established predictors at all. We demonstrate the MSN to update a prediction model for pediatric ECMO patient mortality.

Statistical Approach to Identify Threshold and Point of Departure in Dose-Response Data.

The study presents an integrated, rigorous statistical approach to define the likelihood of a threshold and point of departure (POD) based on dose-response data using nested family of bent-hyperbola models. The family includes four models: the full bent-hyperbola model, which allows for transition between two linear regiments with various levels of smoothness; a bent-hyperbola model reduced to a spline model, where the transition is fixed to a knot; a bent-hyperbola model with a restricted negative asymptote slope of zero, named hockey-stick with arc (HS-Arc); and spline model reduced further to a hockey-stick type model (HS), where the first linear segment has a slope of zero. A likelihood-ratio test is used to discriminate between the models and determine if the more flexible versions of the model provide better or significantly better fit than a hockey-stick type model. The full bent-hyperbola model can accommodate both threshold and nonthreshold behavior, can take on concave up and concave down shapes with various levels of curvature, can approximate the biochemically relevant Michaelis-Menten model, and even be reduced to a straight line. Therefore, with the use of this model, the presence or absence of a threshold may even become irrelevant and the best fit of the full bent-hyperbola model be used to characterize the dose-response behavior and risk levels, with no need for mode of action (MOA) information. Point of departure (POD), characterized by exposure level at which some predetermined response is reached, can be defined using the full model or one of the better fitting reduced models.

Inference for the difference in the area under the ROC curve derived from nested binary regression models.

The area under the curve (AUC) statistic is a common measure of model performance in a binary regression model. Nested models are used to ascertain whether the AUC statistic increases when new factors enter the model. The regression coefficient estimates used in the AUC statistics are computed using the maximum rank correlation methodology. Typically, inference for the difference in AUC statistics from nested models is derived under asymptotic normality. In this work, it is demonstrated that the asymptotic normality is true only when at least one of the new factors is associated with the binary outcome. When none of the new factors are associated with the binary outcome, the asymptotic distribution for the difference in AUC statistics is a linear combination of chi-square random variables. Further, when at least one new factor is associated with the outcome and the population difference is small, a variance stabilizing reparameterization improves the asymptotic normality of the AUC difference statistic. A confidence interval using this reparameterization is developed and simulations are generated to determine their coverage properties. The derived confidence interval provides information on the magnitude of the added value of new factors and enables investigators to weigh the size of the improvement against potential costs associated with the new factors. A pancreatic cancer data example is used to illustrate this approach.

Direct estimation for adaptive treatment length policies: Methods and application to evaluating the effect of delayed PEG insertion.

Dysphagia is a primary cause of death among patients diagnosed with amyotrophic lateral sclerosis (ALS), and percutaneous endoscopic gastrostomy (PEG) is a procedure to insert a tube into the stomach to assist or replace oral feeding. It is believed that PEG is beneficial and, generally, earlier insertion is preferable to later. However, gathering clinical evidence to support these beliefs on the use and timing of PEG is challenging because controlled clinical trials are not feasible and clinical endpoints are confounded with PEG in observational data. Moreover, the confounders are time-varying and time to PEG insertion may be only partially observed. We show how one can view this problem as an adaptive treatment length policy and propose a new estimator via g-computation. We show that our estimator is consistent and asymptotically normal for the causal estimand and explore its finite sample properties in simulation studies. Finally, using more than 10 years of data from Emory ALS clinic registry, we found no evidence to suggest that earlier PEG reduced 4-year mortality; thus, our results do not support the hypothesis and belief that initiating palliative care earlier extends life, on average. At the same, we cannot be certain that all important confounding variables are collected and observed to ensure our modeling assumptions are correct, so more work is needed to address these important end-of-life questions for ALS patients.

MIMICKING COUNTERFACTUAL OUTCOMES TO ESTIMATE CAUSAL EFFECTS.

In observational studies, treatment may be adapted to covariates at several times without a fixed protocol, in continuous time. Treatment influences covariates, which influence treatment, which influences covariates, and so on. Then even time-dependent Cox-models cannot be used to estimate the net treatment effect. Structural nested models have been applied in this setting. Structural nested models are based on counterfactuals: the outcome a person would have had had treatment been withheld after a certain time. Previous work on continuous-time structural nested models assumes that counterfactuals depend deterministically on observed data, while conjecturing that this assumption can be relaxed. This article proves that one can mimic counterfactuals by constructing random variables, solutions to a differential equation, that have the same distribution as the counterfactuals, even given past observed data. These "mimicking" variables can be used to estimate the parameters of structural nested models without assuming the treatment effect to be deterministic.

Mediation Analysis for Health Disparities Research.

Social epidemiologists often seek to determine the mechanisms that underlie health disparities. This work is typically based on mediation procedures that may not be justified with exposures of common interest in social epidemiology. In this analysis, we explored the consequences of using standard approaches, referred to as the difference and generalized product methods, when mediator-outcome confounders are associated with the exposure. We compared these with inverse probability-weighted marginal structural models, the structural transformation method, doubly robust g-estimation of a structural nested model, and doubly robust targeted minimum loss-based estimation. We used data on 900,726 births from 2003 to 2007 in the Penn Moms study, conducted in Pennsylvania, to assess the extent to which breastfeeding prior to hospital discharge explained the racial disparity in infant mortality. Overall, for every 1,000 births, 3.36 more infant deaths occurred among non-Hispanic black women relative to all other women (95% confidence interval: 2.78, 3.93). Using the difference and generalized product methods to assess the disparity that would remain if everyone breastfed prior to discharge suggested a complete elimination of the disparity (risk difference = -0.87 per 1,000 births; 95% confidence interval: -1.39, -0.35). In contrast, doubly robust methods suggested a reduction in the disparity to 2.45 (95% confidence interval: 2.20, 2.71) more infant deaths per 1,000 births among non-Hispanic black women. Standard approaches for mediation analysis in health disparities research can yield misleading results.

Using structural-nested models to estimate the effect of cluster-level adherence on individual-level outcomes with a three-armed cluster-randomized trial.

Much attention has been paid to estimating the causal effect of adherence to a randomized protocol using instrumental variables to adjust for unmeasured confounding. Researchers tend to use the instrumental variable within one of the three main frameworks: regression with an endogenous variable, principal stratification, or structural-nested modeling. We found in our literature review that even in simple settings, causal interpretations of analyses with endogenous regressors can be ambiguous or rely on a strong assumption that can be difficult to interpret. Principal stratification and structural-nested modeling are alternative frameworks that render unambiguous causal interpretations based on assumptions that are, arguably, easier to interpret. Our interest stems from a wish to estimate the effect of cluster-level adherence on individual-level binary outcomes with a three-armed cluster-randomized trial and polytomous adherence. Principal stratification approaches to this problem are quite challenging because of the sheer number of principal strata involved. Therefore, we developed a structural-nested modeling approach and, in the process, extended the methodology to accommodate cluster-randomized trials with unequal probability of selecting individuals. Furthermore, we developed a method to implement the approach with relatively simple programming. The approach works quite well, but when the structural-nested model does not fit the data, there is no solution to the estimating equation. We investigate the performance of the approach using simulated data, and we also use the approach to estimate the effect on pupil absence of school-level adherence to a randomized water, sanitation, and hygiene intervention in western Kenya.

Causal inference in occupational epidemiology: accounting for the healthy worker effect by using structural nested models.

In a recent issue of the Journal, Kirkeleit et al. (Am J Epidemiol. 2013;177(11):1218-1224) provided empirical evidence for the potential of the healthy worker effect in a large cohort of Norwegian workers across a range of occupations. In this commentary, we provide some historical context, define the healthy worker effect by using causal diagrams, and use simulated data to illustrate how structural nested models can be used to estimate exposure effects while accounting for the healthy worker survivor effect in 4 simple steps. We provide technical details and annotated SAS software (SAS Institute, Inc., Cary, North Carolina) code corresponding to the example analysis in the Web Appendices, available at http://aje.oxfordjournals.org/.

Applications of a Kullback-Leibler Divergence for Comparing Non-nested Models.

Wang and Ghosh (2011) proposed a Kullback-Leibler divergence (KLD) which is asymptotically equivalent to the KLD by Goutis and Robert (1998) when the reference model (in comparison with a competing fitted model) is correctly specified and when certain regularity conditions hold true. While properties of the KLD by Wang and Ghosh (2011) have been investigated in the Bayesian framework, this paper further explores the property of this KLD in the frequentist framework using four application examples, each fitted by two competing non-nested models.

Correlations of demographic factors and hygiene factors with face mask wearing during the COVID-19 pandemic and suggestion for future research: A cross-sectional study of adults in Malaysia.

Background: Despite the many touted benefits of community-wide face mask wearing, numerous communication campaigns and mandates, some people still refuse or fail to wear face masks in public settings. Hence, exposing themselves and others to the risk of infection by the severe acute respiratory syndrome coronavirus 2 and raise the potential for public healthcare systems to become overwhelmed once again. This study investigates demographic and hygiene factors related to propensity of face mask wearing in public settings. Design and methods: The self-administered online questionnaire contained the independent variables (demographic and hygiene factors) and the outcome variable (frequency of face mask wearing). Participants were recruited through convenience and snowball sampling techniques. Seven hundred and eight responses were collected from Malaysian adults between May and June 2020. The demographic characteristics of participants, differences in the frequency of face mask wearing across demographic factors and hierarchical multiple regression were analyzed. Results: The propensity of face mask wearing differs by gender. The hierarchical multiple regression revealed that being female, having personal protective equipment available and frequently washing hands were positively correlated with the frequency of face mask wearing. Moreover, the availability of personal protective equipment and the frequency of hand washing accounted for greater variation of the frequency of face mask wearing than gender. Conclusion: Future studies should adopt established psychosocial models in conjunction with normative and cultural factors for a better understanding of underlying motivations to engage in preventive health behaviors to shape improved hygienic and societal precautionary protective behaviors in different contexts.

Understanding the impact of correlation within pair-bonds on Cormack-Jolly-Seber models.

The Cormack-Jolly-Seber (CJS) model and its extensions have been widely applied to the study of animal survival rates in open populations. The model assumes that individuals within the population of interest have independent fates. It is, however, highly unlikely that a pair of animals which have formed a long-term pairing have dissociated fates.We examine a model extension which allows animals who have formed a pair-bond to have correlated survival and recapture fates. Using the proposed extension to generate data, we conduct a simulation study exploring the impact that correlated fate data has on inference from the CJS model. We compute Monte Carlo estimates for the bias, range, and standard errors of the parameters of the CJS model for data with varying degrees of survival correlation between mates. Furthermore, we study the likelihood ratio test of sex effects within the CJS model by simulating densities of the deviance. Finally, we estimate the variance inflation factor c ^ for CJS models that incorporate sex-specific heterogeneity.Our study shows that correlated fates between mated animals may result in underestimated standard errors for parsimonious models, significantly deflated likelihood ratio test statistics, and underestimated values of c ^ for models taking sex-specific effects into account.Underestimated standard errors can result in lowered coverage of confidence intervals. Moreover, deflated test statistics will provide overly conservative test results. Finally, underestimated variance inflation factors can lead researchers to make incorrect conclusions about the level of extra-binomial variation present in their data.

Nested logistic regression models and ΔAUC applications: Change-point analysis.

The area under the receiver operating characteristic curve (AUC) is one of the most popular measures for evaluating the performance of a predictive model. In nested models, the change in AUC (ΔAUC) can be a discriminatory measure of whether the newly added predictors provide significant improvement in terms of predictive accuracy. Recently, several authors have shown rigorously that ΔAUC can be degenerate and its asymptotic distribution is no longer normal when the reduced model is true, but it could be the distribution of a linear combination of some χ12 random variables [1,2]. Hence, the normality assumption and existing variance estimate cannot be applied directly for developing a statistical test under the nested models. In this paper, we first provide a brief review on the use of ΔAUC for comparing nested logistic models and the difficulty of retrieving the reference distribution behind. Then, we present a special case of the nested logistic regression models that the newly added predictor to the reduced model contains a change-point in its effects. A new test statistic based on ΔAUC is proposed in this setting. A simple resampling scheme is proposed to approximate the critical values for the test statistic. The inference of the change-point parameter is done via m-out-of-n bootstrap. Large-scale simulation is conducted to evaluate the finite-sample performance of the ΔAUC test for the change-point model. The proposed method is applied to two real-life datasets for illustration.

Controlling the error probabilities of model selection information criteria using bootstrapping.

The Akaike Information Criterion (AIC) and related information criteria are powerful and increasingly popular tools for comparing multiple, non-nested models without the specification of a null model. However, existing procedures for information-theoretic model selection do not provide explicit and uniform control over error rates for the choice between models, a key feature of classical hypothesis testing. We show how to extend notions of Type-I and Type-II error to more than two models without requiring a null. We then present the Error Control for Information Criteria (ECIC) method, a bootstrap approach to controlling Type-I error using Difference of Goodness of Fit (DGOF) distributions. We apply ECIC to empirical and simulated data in time series and regression contexts to illustrate its value for parametric Neyman-Pearson classification. An R package implementing the bootstrap method is publicly available.