Negative variance components and intercept-slope correlations greater than one in magnitude: How do such "non-regular" random intercept and slope models arise, and what should be done when they do?

Statistical models with random intercepts and slopes (RIAS models) are commonly used to analyze longitudinal data. Fitting such models sometimes results in negative estimates of variance components or estimates on parameter space boundaries. This can be an unlucky chance occurrence, but can also occur because certain marginal distributions are mathematically identical to those from RIAS models with negative intercept and/or slope variance components and/or intercept-slope correlations greater than one in magnitude. We term such parameters "pseudo-variances" and "pseudo-correlations," and the models "non-regular." We use eigenvalue theory to explore how and when such non-regular RIAS models arise, showing: (i) A small number of measurements, short follow-up, and large residual variance increase the parameter space for which data (with a positive semidefinite marginal variance-covariance matrix) are compatible with non-regular RIAS models. (ii) Non-regular RIAS models can arise from model misspecification, when non-linearity in fixed effects is ignored or when random effects are omitted. (iii) A non-regular RIAS model can sometimes be interpreted as a regular linear mixed model with one or more additional random effects, which may not be identifiable from the data. (iv) Particular parameterizations of non-regular RIAS models have no generality for all possible numbers of measurements over time. Because of this lack of generality, we conclude that non-regular RIAS models can only be regarded as plausible data-generating mechanisms in some situations. Nevertheless, fitting a non-regular RIAS model can be acceptable, allowing unbiased inference on fixed effects where commonly recommended alternatives such as dropping the random slope result in bias.

How important is the linearity assumption in a sample size calculation for a randomised controlled trial where treatment is anticipated to affect a rate of change?

BACKGROUND: For certain conditions, treatments aim to lessen deterioration over time. A trial outcome could be change in a continuous measure, analysed using a random slopes model with a different slope in each treatment group. A sample size for a trial with a particular schedule of visits (e.g. annually for three years) can be obtained using a two-stage process. First, relevant (co-) variances are estimated from a pre-existing dataset e.g. an observational study conducted in a similar setting. Second, standard formulae are used to calculate sample size. However, the random slopes model assumes linear trajectories with any difference in group means increasing proportionally to follow-up time. The impact of these assumptions failing is unclear. METHODS: We used simulation to assess the impact of a non-linear trajectory and/or non-proportional treatment effect on the proposed trial's power. We used four trajectories, both linear and non-linear, and simulated observational studies to calculate sample sizes. Trials of this size were then simulated, with treatment effects proportional or non-proportional to time. RESULTS: For a proportional treatment effect and a trial visit schedule matching the observational study, powers are close to nominal even for non-linear trajectories. However, if the schedule does not match the observational study, powers can be above or below nominal levels, with the extent of this depending on parameters such as the residual error variance. For a non-proportional treatment effect, using a random slopes model can lead to powers far from nominal levels. CONCLUSIONS: If trajectories are suspected to be non-linear, observational data used to inform power calculations should have the same visit schedule as the proposed trial where possible. Additionally, if the treatment effect is expected to be non-proportional, the random slopes model should not be used. A model allowing trajectories to vary freely over time could be used instead, either as a second line analysis method (bearing in mind that power will be lost) or when powering the trial.

Power and sample-size calculations for trials that compare slopes over time: Introducing the slopepower command.

Trials of interventions that aim to slow disease progression may analyze a continuous outcome by comparing its change over time-its slope-between the treated and the untreated group using a linear mixed model. To perform a sample-size calculation for such a trial, one must have estimates of the parameters that govern the between- and within-subject variability in the outcome, which are often unknown. The algebra needed for the sample-size calculation can also be complex for such trial designs. We have written a new user-friendly command, slopepower, that performs sample-size or power calculations for trials that compare slope outcomes. The package is based on linear mixed-model methodology, described for this setting by Frost, Kenward, and Fox (2008, Statistics in Medicine 27: 3717-3731). In the first stage of this approach, slopepower obtains estimates of mean slopes together with variances and covariances from a linear mixed model fit to previously collected user-supplied data. In the second stage, these estimates are combined with user input about the target effectiveness of the treatment and design of the future trial to give an estimate of either a sample size or a statistical power. In this article, we present the slopepower command, briefly explain the methodology behind it, and demonstrate how it can be used to help plan a trial and compare the sample sizes needed for different trial designs.

The association between anthropometric measures of adiposity and the progression of carotid atherosclerosis.

BACKGROUND: Few reports are available on the contribution of general and abdominal obesity to the progression of carotid atherosclerosis in late adulthood. This study investigated the impact of four simple anthropometric measures of general and abdominal obesity on the progression of carotid atherosclerosis and the extent to which the association between adiposity and the progression of plaque burden is mediated by cardiometabolic markers. METHODS: Four thousand three hundred forty-five adults (median age 60) from the population-based Tromsø Study were followed over 7 years from the first carotid ultrasound screening to the next. The progression of carotid atherosclerosis was measured in three ways: incidence of plaques in previously plaque-free participants; change in the number of plaques; and total plaque area (TPA). We used generalised linear models to investigate the association between each adiposity measure - body mass index (BMI), waist circumference (WC), waist-to-hip ratio (WHR), and waist-to-height ratio (WHtR) - and each outcome. Models were adjusted for potential confounders (age, sex, smoking, education, physical activity). The pathways through which any associations observed might operate were investigated by further adjusting for cardiometabolic mediators (systolic blood pressure, cholesterol, and HbA1c). RESULTS: There was little evidence that adiposity was related to the formation of new plaques during follow-up. However, abdominal adiposity was associated with TPA progression. WHtR showed the largest effect size (mean change in TPA per one standard deviation (SD) increase in WHtR of 0.665 mm2, 95% confidence interval 0.198, 1.133) while BMI showed the smallest. Effect sizes were substantially reduced after the adjustment for potential mediators. CONCLUSIONS: Abdominal obesity indirectly measured with WC seems more strongly associated with the progression of TPA than general obesity. These associations appear to be largely mediated by known cardiometabolic markers.

Interpreting population reach of a large, successful physical activity trial delivered through primary care.

BACKGROUND: Failure to include socio-economically deprived or ethnic minority groups in physical activity (PA) trials may limit representativeness and could lead to implementation of interventions that then increase health inequalities. Randomised intervention trials often have low recruitment rates and rarely assess recruitment bias. A previous trial by the same team using similar methods recruited 30% of the eligible population but was in an affluent setting with few non-white residents and was limited to those over 60 years of age. METHODS: PACE-UP is a large, effective, population-based walking trial in inactive 45-75 year-olds that recruited through seven London general practices. Anonymised practice demographic data were available for all those invited, enabling investigation of inequalities in trial recruitment. Non-participants were invited to complete a questionnaire. RESULTS: From 10,927 postal invitations, 1150 (10.5%) completed baseline assessment. Participation rate ratios (95% CI), adjusted for age and gender as appropriate, were lower in men 0.59 (0.52, 0.67) than women, in those under 55 compared with those ≥65, 0.60 (0.51, 0.71), in the most deprived quintile compared with the least deprived 0.52 (0.39, 0.70) and in Asian individuals compared with whites 0.62 (0.50, 0.76). Black individuals were equally likely to participate as white individuals. Participation was also associated with having a co-morbidity or some degree of health limitation. The most common reasons for non-participation were considering themselves as being too active or lack of time. CONCLUSIONS: Conducting the trial in this diverse setting reduced overall response, with lower response in socio-economically deprived and Asian sub-groups. Trials with greater reach are likely to be more expensive in terms of recruitment and gains in generalizability need to be balanced with greater costs. Differential uptake of successful trial interventions may increase inequalities in PA levels and should be monitored. TRIAL REGISTRATION: ISRCTN.com ISRCTN98538934 . Registered 2nd March 2012.

Increasing the Availability of Psychological Treatments: A Multinational Study of a Scalable Method for Training Therapists.

BACKGROUND: One of the major barriers to the dissemination and implementation of psychological treatments is the scarcity of suitably trained therapists. A highly scalable form of Web-centered therapist training, undertaken without external support, has recently been shown to have promise in promoting therapist competence. OBJECTIVE: The aim of this study was to conduct an evaluation of the acceptability and effectiveness of a scalable independent form of Web-centered training in a multinational sample of therapists and investigate the characteristics of those most likely to benefit. METHODS: A cohort of eligible therapists was recruited internationally and offered access to Web-centered training in enhanced cognitive behavioral therapy, a multicomponent, evidence-based, psychological treatment for any form of eating disorder. No external support was provided during training. Therapist competence was assessed using a validated competence measure before training and after 20 weeks. RESULTS: A total of 806 therapists from 33 different countries expressed interest in the study, and 765 (94.9%) completed a pretraining assessment. The median number of training modules completed was 15 out of a possible 18 (interquartile range, IQR: 4-18), and 87.9% (531/604) reported that they treated at least one patient during training as recommended. Median pretraining competence score was 7 (IQR: 5-10, range: 0-19; N=765), and following training, it was 12 (IQR: 9-15, range: 0-20; N=577). The expected change in competence scores from pretraining to posttraining was 3.5 (95% CI 3.1-3.8; P<.001). After training, 52% (300/574) of therapists with complete competence data met or exceeded the competence threshold, and 45% (95% CI 41-50) of those who had not met this threshold before training did so after training. Compliance with training predicted both an increase in competence scores and meeting or exceeding the competence threshold. Expected change in competence score increased for each extra training module completed (0.19, 95% CI 0.13-0.25), and those who treated a suitable patient during training had an expected change in competence score 1.2 (95% CI 0.4-2.1) points higher than those who did not. Similarly, there was an association between meeting the competence threshold after training and the number of modules completed (odds ratio, OR=1.11, 95% CI 1.07-1.15), and treating at least one patient during training was associated with competence after training (OR=2.2, 95% CI 1.2-4.1). CONCLUSIONS: Independent Web-centered training can successfully train large numbers of therapists dispersed across a wide geographical area. This finding is of importance because the availability of a highly scalable method of training potentially increases the number of people who might receive effective psychological treatments.

Choosing appropriate analysis methods for cluster randomised cross-over trials with a binary outcome.

In cluster randomised cross-over (CRXO) trials, clusters receive multiple treatments in a randomised sequence over time. In such trials, there is usual correlation between patients in the same cluster. In addition, within a cluster, patients in the same period may be more similar to each other than to patients in other periods. We demonstrate that it is necessary to account for these correlations in the analysis to obtain correct Type I error rates. We then use simulation to compare different methods of analysing a binary outcome from a two-period CRXO design. Our simulations demonstrated that hierarchical models without random effects for period-within-cluster, which do not account for any extra within-period correlation, performed poorly with greatly inflated Type I errors in many scenarios. In scenarios where extra within-period correlation was present, a hierarchical model with random effects for cluster and period-within-cluster only had correct Type I errors when there were large numbers of clusters; with small numbers of clusters, the error rate was inflated. We also found that generalised estimating equations did not give correct error rates in any scenarios considered. An unweighted cluster-level summary regression performed best overall, maintaining an error rate close to 5% for all scenarios, although it lost power when extra within-period correlation was present, especially for small numbers of clusters. Results from our simulation study show that it is important to model both levels of clustering in CRXO trials, and that any extra within-period correlation should be accounted for. Copyright © 2016 John Wiley & Sons, Ltd.

Using the Internet to Train Therapists: Randomized Comparison of Two Scalable Methods.

BACKGROUND: One of the major barriers to the dissemination and implementation of psychological treatments is the scarcity of suitably trained therapists. The currently accepted method of training is not scalable. Recently, a scalable form of training, Web-centered training, has been shown to have promise. OBJECTIVE: The goal of our research was to conduct a randomized comparison of the relative effects of independent and supported Web-centered training on therapist competence and investigate the persistence of the effects. METHODS: Eligible therapists were recruited from across the United States and Canada. They were randomly assigned to 1 of 2 forms of training in enhanced cognitive behavior therapy (CBT-E), a multicomponent evidence-based psychological treatment for any form of eating disorder. Independent training was undertaken autonomously, while supported training was accompanied by support from a nonspecialist worker. Therapist competence was assessed using a validated competence measure before training, after 20 weeks of training, and 6 months after the completion of training. RESULTS: A total of 160 therapists expressed interest in the study, and 156 (97.5%) were randomized to the 2 forms of training (81 to supported training and 75 to independent training). Mixed effects analysis showed an increase in competence scores in both groups. There was no difference between the 2 forms of training, with mean difference for the supported versus independent group being -0.06 (95% Cl -1.29 to 1.16, P=.92). A total of 58 participants (58/114, 50.9%) scored above the competence threshold; three-quarters (43/58, 74%) had not met this threshold before training. There was no difference between the 2 groups in the odds of scoring over the competence threshold (odds ratio [OR] 1.02, 95% CI 0.52 to 1.99; P=.96). At follow-up, there was no significant difference between the 2 training groups (mean difference 0.19, 95% Cl -1.27 to 1.66, P=.80). Overall, change in competence score from end of training to follow-up was not significant (mean difference -0.70, 95% CI -1.52 to 0.11, P=.09). There was also no difference at follow-up between the training groups in the odds of scoring over the competence threshold (OR 0.95, 95% Cl 0.34 to 2.62; P=.92). CONCLUSIONS: Web-centered training was equally effective whether undertaken independently or accompanied by support, and its effects were sustained. The independent form of Web-centered training is particularly attractive as it provides a means of training large numbers of geographically dispersed therapists at low cost, thereby overcoming several obstacles to the widespread dissemination of psychological treatments.

Reflection on modern methods: calculating a sample size for a repeatability sub-study to correct for measurement error in a single continuous exposure.

Using a continuous exposure variable that is measured with random error in a univariable linear regression model leads to regression dilution bias: the observed association between the exposure and outcome is smaller than it would be if the true value of the exposure could be used. A repeatability sub-study, where a sample of study participants have their data measured again, can be used to correct for this bias. It is important to perform a sample size calculation for such a sub-study, to ensure that correction factors can be estimated with sufficient precision. We describe how a previously published method can be used to calculate the sample size from the anticipated size of the correction factor and its desired precision, and demonstrate this approach using the example of the cross-sectional studies conducted as part of the International Project on Cardiovascular Disease in Russia study. We also provide correction factors calculated from repeat data from the UK Biobank study, which can be used to help plan future repeatability studies.

Cluster randomized trials with a small number of clusters: which analyses should be used?

Background: Cluster randomized trials (CRTs) are increasingly used to assess the effectiveness of health interventions. Three main analysis approaches are: cluster-level analyses, mixed-models and generalized estimating equations (GEEs). Mixed models and GEEs can lead to inflated type I error rates with a small number of clusters, and numerous small-sample corrections have been proposed to circumvent this problem. However, the impact of these methods on power is still unclear. Methods: We performed a simulation study to assess the performance of 12 analysis approaches for CRTs with a continuous outcome and 40 or fewer clusters. These included weighted and unweighted cluster-level analyses, mixed-effects models with different degree-of-freedom corrections, and GEEs with and without a small-sample correction. We assessed these approaches across different values of the intraclass correlation coefficient (ICC), numbers of clusters and variability in cluster sizes. Results: Unweighted and variance-weighted cluster-level analysis, mixed models with degree-of-freedom corrections, and GEE with a small-sample correction all maintained the type I error rate at or below 5% across most scenarios, whereas uncorrected approaches lead to inflated type I error rates. However, these analyses had low power (below 50% in some scenarios) when fewer than 20 clusters were randomized, with none reaching the expected 80% power. Conclusions: Small-sample corrections or variance-weighted cluster-level analyses are recommended for the analysis of continuous outcomes in CRTs with a small number of clusters. The use of these corrections should be incorporated into the sample size calculation to prevent studies from being underpowered.

The contribution of obesity to carotid atherosclerotic plaque burden in a general population sample in Norway: The Tromsø Study.

BACKGROUND AND AIMS: Few studies have investigated the association of different measures of adiposity with carotid plaque. We aimed to investigate and compare the associations of four measures of adiposity: body mass index (BMI), waist circumference (WC), waist-to-hip ratio (WHR), and waist-to-height ratio (WHtR) with the presence of carotid plaque and total plaque area (TPA) in the right carotid artery. METHODS: We included 4906 individuals aged 31-88 years who participated in a population-based study with ultrasonography of the right carotid artery. Adiposity measures were converted to sex-specific SD units to allow comparison of effect sizes. TPA was log transformed due to its skewed distribution. Logistic and linear regression models were used respectively to investigate the association of each adiposity measure with the presence of plaque and with log-transformed TPA. Estimates were adjusted for potential confounders and mediators such as blood pressure and lipids. RESULTS: After adjustment for age, sex, smoking, and education level, there was strong evidence of an association between all adiposity measures and log-transformed TPA, whereas only WHR was weakly associated with presence of plaque. WHR showed the largest adjusted effect size for both log-transformed TPA (beta 0.055, 95%CI 0.028-0.081) and the presence of plaque (OR 1.07, 95%CI 1.01-1.15). Adjustment for mediators led to appreciable attenuation of observed effects. CONCLUSIONS: Adiposity is more consistently associated with extent of plaque burden than with whether an individual does or does not have any plaque. There was evidence that established biomarkers mediate much of this association. Abdominal adiposity appears to show the strongest effect.

The quality of reporting in cluster randomised crossover trials: proposal for reporting items and an assessment of reporting quality.

BACKGROUND: The cluster randomised crossover (CRXO) design is gaining popularity in trial settings where individual randomisation or parallel group cluster randomisation is not feasible or practical. Our aim is to stimulate discussion on the content of a reporting guideline for CRXO trials and to assess the reporting quality of published CRXO trials. METHODS: We undertook a systematic review of CRXO trials. Searches of MEDLINE, EMBASE, and CINAHL Plus as well as citation searches of CRXO methodological articles were conducted to December 2014. Reporting quality was assessed against both modified items from 2010 CONSORT and 2012 cluster trials extension and other proposed quality measures. RESULTS: Of the 3425 records identified through database searching, 83 trials met the inclusion criteria. Trials were infrequently identified as "cluster randomis(z)ed crossover" in title (n = 7, 8%) or abstract (n = 21, 25%), and a rationale for the design was infrequently provided (n = 20, 24%). Design parameters such as the number of clusters and number of periods were well reported. Discussion of carryover took place in only 17 trials (20%). Sample size methods were only reported in 58% (n = 48) of trials. A range of approaches were used to report baseline characteristics. The analysis method was not adequately reported in 23% (n = 19) of trials. The observed within-cluster within-period intracluster correlation and within-cluster between-period intracluster correlation for the primary outcome data were not reported in any trial. The potential for selection, performance, and detection bias could be evaluated in 30%, 81%, and 70% of trials, respectively. CONCLUSIONS: There is a clear need to improve the quality of reporting in CRXO trials. Given the unique features of a CRXO trial, it is important to develop a CONSORT extension. Consensus amongst trialists on the content of such a guideline is essential.

Appropriate statistical methods were infrequently used in cluster-randomized crossover trials.

OBJECTIVE: To assess the design and statistical methods used in cluster-randomized crossover (CRXO) trials. STUDY DESIGN AND SETTING: We undertook a systematic review of CRXO trials. Searches of MEDLINE, EMBASE, and CINAHL Plus; and citation searches of CRXO methodological articles were conducted to December 2014. We extracted data on design characteristics and statistical methods for sample size, data analysis, and handling of missing data. RESULTS: Ninety-one trials including 139 end point analyses met the inclusion criteria. Trials had a median of nine clusters [interquartile range (IQR), 4-21] and median cluster-period size of 30 individuals (IQR, 14-77); 58 (69%) trials had two periods, and 27 trials (30%) included the same individuals in all periods. A rationale for the design was reported in only 25 trials (27%). A sample size justification was provided in 53 (58%) trials. Only nine (10%) trials accounted appropriately for the design in their sample size calculation. Ten of the 12 cluster-level analyses used a method that accounted for the clustering and multiple-period aspects of the design. In contrast, only 4 of the 127 individual-level analyses used a potentially appropriate method. CONCLUSIONS: There is a need for improved application of appropriate analysis and sample size methods, and reporting, in CRXO trials.

The use of the cluster randomized crossover design in clinical trials: protocol for a systematic review.

BACKGROUND: The cluster randomized crossover (CRXO) design is gaining popularity in trial settings where individual randomization or parallel group cluster randomization is not feasible or practical. In a CRXO trial, not only are clusters of individuals rather than individuals themselves randomized to trial arms, but also each cluster participates in each arm of the trial at least once in separate periods of time.We will review publications of clinical trials undertaken in humans that have used the CRXO design. The aim of this systematic review is to summarize, as reported: the motivations for using the CRXO design, the values of the CRXO design parameters, the justification and methodology for the sample size calculations and analyses, and the quality of reporting the CRXO design aspects. METHODS/DESIGN: We will identify reports of CRXO trials by systematically searching MEDLINE, PubMed, Cochrane Methodology Register, EMBASE, and CINAHL Plus. In addition, we will search for methodological articles that describe the CRXO design and conduct citation searches to identify any further CRXO trials. The references of all eligible trials will also be searched. We will screen the identified abstracts, and retrieve and assess for inclusion the full text for any potentially relevant articles. Data will be extracted from the full text independently by two reviewers. Descriptive summary statistics will be presented for the extracted data. DISCUSSION: This systematic review will inform both researchers addressing CRXO methodology and trialists considering implementing the design. The results will allow focused methodological research of the CRXO design, provide practical examples for researchers of how CRXO trials have been conducted, including any shortcomings, and highlight areas where reporting and conduct may be improved.