Discussion of "the evolution of boosting algorithms" and "extending statistical boosting".

This article is part of a For-Discussion-Section of Methods of Information in Medicine about the papers "The Evolution of Boosting Algorithms - From Machine Learning to Statistical Modelling" and "Extending Statistical Boosting - An Overview of Recent Methodological Developments", written by Andreas Mayr and co-authors. It is introduced by an editorial. This article contains the combined commentaries invited to independently comment on the Mayr et al. papers. In subsequent issues the discussion can continue through letters to the editor.

Regularization for generalized additive mixed models by likelihood-based boosting.

OBJECTIVE: With the emergence of semi- and nonparametric regression the generalized linear mixed model has been extended to account for additive predictors. However, available fitting methods fail in high dimensional settings where many explanatory variables are present. We extend the concept of boosting to generalized additive mixed models and present an appropriate algorithm that uses two different approaches for the fitting procedure of the variance components of the random effects. METHODS: The main tool developed is likelihood-based componentwise boosting that enforces variable selection in generalized additive mixed models. In contrast to common procedures they can be used in high-dimensional settings where many covariates are available and the form of the influence is unknown. The complexity of the resulting estimators is determined by information criteria. The performance of the methods is investigated in simulation studies for binary and Poisson responses with comparisons to alternative approaches and it is applied to clinical real world data. RESULTS: Simulations show that the proposed methods are considerably more stable and more accurate in estimating the regression function than the conventional approach, especially when a large number of predictors is available. The methods also produce reasonable results in applications to real data sets, which is illustrated by the Multicenter AIDS Cohort Study. CONCLUSIONS: The boosting algorithm allows to extract relevant predictors in generalized additive mixed models. It works in high-dimensional settings and is very stable.

A boosting approach to flexible semiparametric mixed models.

In linear mixed models the influence of covariates is restricted to a strictly parametric form. With the rise of semi- and non-parametric regression also the mixed model has been expanded to allow for additive predictors. The common approach uses the representation of additive models as mixed models. An alternative approach that is proposed in the present paper is likelihood based boosting. Boosting originates in the machine learning community where it has been proposed as a technique to improve classification procedures by combining estimates with reweighted observations. Likelihood based boosting is a general method which may be seen as an extension of L2 boost. In additive mixed models the advantage of boosting techniques in the form of componentwise boosting is that it is suitable for high dimensional settings where many explanatory variables are present. It allows to fit additive models for many covariates with implicit selection of relevant variables and automatic selection of smoothing parameters. Moreover, boosting techniques may be used to incorporate the subject-specific variation of smooth influence functions by specifying 'random slopes' on smooth effects. This results in flexible semiparametric mixed models which are appropriate in cases where a simple random intercept is unable to capture the variation of effects across subjects.

Flexible modelling of discrete failure time including time-varying smooth effects.

Discrete survival models have been extended in several ways. More flexible models are obtained by including time-varying coefficients and covariates which determine the hazard rate in an additive but not further specified form. In this paper, a general model is considered which comprises both types of covariate effects. An additional extension is the incorporation of smooth interaction between time and covariates. Thus, in the linear predictor smooth effects of covariates which may vary across time are allowed. It is shown how simple duration models produce artefacts which may be avoided by flexible models. For the general model which includes parametric terms, time-varying coefficients in parametric terms and time-varying smooth effects estimation procedures are derived which are based on the regularized expansion of smooth effects in basis functions. The approach is used to model the sojourn time in a psychiatric hospital. It is demonstrated how initial conditions which have non-linear influence are damped over time.

Nonparametric estimation of discrete hazard functions.

A smoothing procedure for discrete time failure data is proposed which allows for the inclusion of covariates. This purely nonparametric method is based on discrete or continuous kernel smoothing techniques that gives a compromise between the data and smoothness. The method may be used as an exploratory tool to uncover the underlying structure or as an alternative to parametric methods when prediction is the primary objective. Confidence intervals are considered and alternative techniques of cross validation based choices of smoothing parameters are investigated.