Evaluating analytic models for individually randomized group treatment trials with complex clustering in nested and crossed designs.

Many individually randomized group treatment (IRGT) trials randomly assign individuals to study arms but deliver treatments via shared agents, such as therapists, surgeons, or trainers. Post-randomization interactions induce correlations in outcome measures between participants sharing the same agent. Agents can be nested in or crossed with trial arm, and participants may interact with a single agent or with multiple agents. These complications have led to ambiguity in choice of models but there have been no systematic efforts to identify appropriate analytic models for these study designs. To address this gap, we undertook a simulation study to examine the performance of candidate analytic models in the presence of complex clustering arising from multiple membership, single membership, and single agent settings, in both nested and crossed designs and for a continuous outcome. With nested designs, substantial type I error rate inflation was observed when analytic models did not account for multiple membership and when analytic model weights characterizing the association with multiple agents did not match the data generating mechanism. Conversely, analytic models for crossed designs generally maintained nominal type I error rates unless there was notable imbalance in the number of participants that interact with each agent.

Design and Analytic Methods to Evaluate Multilevel Interventions to Reduce Health Disparities: Rigorous Methods Are Available.

In June 2022, the NIH Office of Disease Prevention (ODP) issued a Call for Papers for a Supplemental Issue to Prevention Science on Design and Analytic Methods to Evaluate Multilevel Interventions to Reduce Health Disparities. ODP sought to bring together current thinking and new ideas about design and analytic methods for studies aimed at reducing health disparities, including strategies for balancing methodological rigor with design feasibility, acceptability, and ethical considerations. ODP was particularly interested in papers on design and analytic methods for parallel group- or cluster-randomized trials (GRTs), stepped-wedge GRTs, group-level regression discontinuity trials, and other methods appropriate for evaluating multilevel interventions. In this issue, we include 12 papers that report new methods, provide examples of strong applications of existing methods, or provide guidance on developing multilevel interventions to reduce health disparities. These papers provide examples showing that rigorous methods are available for the design and analysis of multilevel interventions to reduce health disparities.

Influential methods reports for group-randomized trials and related designs.

BACKGROUND: This article identifies the most influential methods reports for group-randomized trials and related designs published through 2020. Many interventions are delivered to participants in real or virtual groups or in groups defined by a shared interventionist so that there is an expectation for positive correlation among observations taken on participants in the same group. These interventions are typically evaluated using a group- or cluster-randomized trial, an individually randomized group treatment trial, or a stepped wedge group- or cluster-randomized trial. These trials face methodological issues beyond those encountered in the more familiar individually randomized controlled trial. METHODS: PubMed was searched to identify candidate methods reports; that search was supplemented by reports known to the author. Candidate reports were reviewed by the author to include only those focused on the designs of interest. Citation counts and the relative citation ratio, a new bibliometric tool developed at the National Institutes of Health, were used to identify influential reports. The relative citation ratio measures influence at the article level by comparing the citation rate of the reference article to the citation rates of the articles cited by other articles that also cite the reference article. RESULTS: In total, 1043 reports were identified that were published through 2020. However, 55 were deemed to be the most influential based on their relative citation ratio or their citation count using criteria specific to each of the three designs, with 32 group-randomized trial reports, 7 individually randomized group treatment trial reports, and 16 stepped wedge group-randomized trial reports. Many of the influential reports were early publications that drew attention to the issues that distinguish these designs from the more familiar individually randomized controlled trial. Others were textbooks that covered a wide range of issues for these designs. Others were "first reports" on analytic methods appropriate for a specific type of data (e.g. binary data, ordinal data), for features commonly encountered in these studies (e.g. unequal cluster size, attrition), or for important variations in study design (e.g. repeated measures, cohort versus cross-section). Many presented methods for sample size calculations. Others described how these designs could be applied to a new area (e.g. dissemination and implementation research). Among the reports with the highest relative citation ratios were the CONSORT statements for each design. CONCLUSIONS: Collectively, the influential reports address topics of great interest to investigators who might consider using one of these designs and need guidance on selecting the most appropriate design for their research question and on the best methods for design, analysis, and sample size.

NIH Primary and Secondary Prevention Research in Humans: a Portfolio Analysis of Study Designs Used in 2012-2019.

We can learn a great deal about the research questions being addressed in a field by examining the study designs used in that field. This manuscript examines the research questions being addressed in prevention research by characterizing the distribution and trends of study designs included in primary and secondary prevention research supported by the National Institutes of Health through grants and cooperative agreements, together with the types of prevention research, populations, rationales, exposures, and outcomes associated with each type of design. The Office of Disease Prevention developed a taxonomy to classify new extramural NIH-funded research projects and created a database with a representative sample of 14,523 research projects for fiscal years 2012-2019. The data were weighted to represent the entirety of the extramural research portfolio. Leveraging this dataset, the Office of Disease Prevention characterized the study designs proposed in NIH-funded primary and secondary prevention research applications. The most common study designs proposed in new NIH-supported prevention research applications during FY12-19 were observational designs (63.3%, 95% CI 61.5%-65.0%), analysis of existing data (44.5%, 95% CI: 42.7-46.3), methods research (23.9%, 95% CI: 22.3-25.6), and randomized interventions (17.2%, 95% CI: 16.1%-18.4%). Observational study designs dominated primary prevention research, while intervention designs were more common in secondary prevention research. Observational designs were more common for exposures that would be difficult to manipulate (e.g., genetics, chemical toxin, and infectious disease (not pneumonia/influenza or HIV/AIDS)), while intervention designs were more common for exposures that would be easier to manipulate (e.g., education/counseling, medication/device, diet/nutrition, and healthcare delivery). Intervention designs were not common for outcomes that are rare or have a long latency (e.g., cancer, neurological disease, Alzheimer's disease) and more common for outcomes that are more common or where effects would be expected earlier (e.g., healthcare delivery, health related quality of life, substance use, and medication/device). Observational designs and analyses of existing data dominated, suggesting that much of the prevention research funded by NIH continues to focus on questions of association and on questions of identification of risk and protective factors. Randomized and non-randomized intervention designs were included far less often, suggesting that a much smaller fraction of the NIH prevention research portfolio is focused on questions of whether interventions can be used to modify risk or protective factors or to change some other health-related biomedical or behavioral outcome. The much heavier focus on observational studies is surprising given how much we know already about the leading risk factors for death and disability in the USA, because those risk factors account for 74% of the county-level mortality in the USA, and because they play such a vital role in the development of clinical and public health guidelines, whose developers often weigh results from randomized trials much more heavily than results from observational studies. Improvements in death and disability nationwide are more likely to derive from guidelines based on intervention research to address the leading risk factors than from additional observational studies.

Essential Ingredients and Innovations in the Design and Analysis of Group-Randomized Trials.

This article reviews the essential ingredients and innovations in the design and analysis of group-randomized trials. The methods literature for these trials has grown steadily since they were introduced to the biomedical research community in the late 1970s, and we summarize those developments. We review, in addition to the group-randomized trial, methods for two closely related designs, the individually randomized group treatment trial and the stepped-wedge group-randomized trial. After describing the essential ingredients for these designs, we review the most important developments in the evolution of their methods using a new bibliometric tool developed at the National Institutes of Health. We then discuss the questions to be considered when selecting from among these designs or selecting the traditional randomized controlled trial. We close with a review of current methods for the analysis of data from these designs, a case study to illustrate each design, and a brief summary.

The Importance of Evaluating Health Disparities Research.

Health disparity populations are socially disadvantaged, and the multiple levels of discrimination they often experience mean that their characteristics and attributes differ from those of the mainstream. Programs and policies targeted at reducing health disparities or improving minority health must consider these differences. Despite the importance of evaluating health disparities research to produce high-quality data that can guide decision-making, it is not yet a customary practice. Although health disparities evaluations incorporate the same scientific methods as all evaluations, they have unique components such as population characteristics, sociocultural context, and the lack of health disparity common indicators and metrics that must be considered in every phase of the research. This article describes evaluation strategies grouped into 3 components: formative (needs assessments and process), design and methodology (multilevel designs used in real-world settings), and summative (outcomes, impacts, and cost). Each section will describe the standards for each component, discuss the unique health disparity aspects, and provide strategies from the National Institute on Minority Health and Health Disparities Metrics and Measures Visioning Workshop (April 2016) to advance the evaluation of health disparities research.

Design and analysis of group-randomized trials in cancer: A review of current practices.

The purpose of this paper is to summarize current practices for the design and analysis of group-randomized trials involving cancer-related risk factors or outcomes and to offer recommendations to improve future trials. We searched for group-randomized trials involving cancer-related risk factors or outcomes that were published or online in peer-reviewed journals in 2011-15. During 2016-17, in Bethesda MD, we reviewed 123 articles from 76 journals to characterize their design and their methods for sample size estimation and data analysis. Only 66 (53.7%) of the articles reported appropriate methods for sample size estimation. Only 63 (51.2%) reported exclusively appropriate methods for analysis. These findings suggest that many investigators do not adequately attend to the methodological challenges inherent in group-randomized trials. These practices can lead to underpowered studies, to an inflated type 1 error rate, and to inferences that mislead readers. Investigators should work with biostatisticians or other methodologists familiar with these issues. Funders and editors should ensure careful methodological review of applications and manuscripts. Reviewers should ensure that studies are properly planned and analyzed. These steps are needed to improve the rigor and reproducibility of group-randomized trials. The Office of Disease Prevention (ODP) at the National Institutes of Health (NIH) has taken several steps to address these issues. ODP offers an online course on the design and analysis of group-randomized trials. ODP is working to increase the number of methodologists who serve on grant review panels. ODP has developed standard language for the Application Guide and the Review Criteria to draw investigators' attention to these issues. Finally, ODP has created a new Research Methods Resources website to help investigators, reviewers, and NIH staff better understand these issues.

Review of Recent Methodological Developments in Group-Randomized Trials: Part 2-Analysis.

In 2004, Murray et al. reviewed methodological developments in the design and analysis of group-randomized trials (GRTs). We have updated that review with developments in analysis of the past 13 years, with a companion article to focus on developments in design. We discuss developments in the topics of the earlier review (e.g., methods for parallel-arm GRTs, individually randomized group-treatment trials, and missing data) and in new topics, including methods to account for multiple-level clustering and alternative estimation methods (e.g., augmented generalized estimating equations, targeted maximum likelihood, and quadratic inference functions). In addition, we describe developments in analysis of alternative group designs (including stepped-wedge GRTs, network-randomized trials, and pseudocluster randomized trials), which require clustering to be accounted for in their design and analysis.

Review of Recent Methodological Developments in Group-Randomized Trials: Part 1-Design.

In 2004, Murray et al. reviewed methodological developments in the design and analysis of group-randomized trials (GRTs). We have highlighted the developments of the past 13 years in design with a companion article to focus on developments in analysis. As a pair, these articles update the 2004 review. We have discussed developments in the topics of the earlier review (e.g., clustering, matching, and individually randomized group-treatment trials) and in new topics, including constrained randomization and a range of randomized designs that are alternatives to the standard parallel-arm GRT. These include the stepped-wedge GRT, the pseudocluster randomized trial, and the network-randomized GRT, which, like the parallel-arm GRT, require clustering to be accounted for in both their design and analysis.

An evaluation of constrained randomization for the design and analysis of group-randomized trials with binary outcomes.

Group-randomized trials are randomized studies that allocate intact groups of individuals to different comparison arms. A frequent practical limitation to adopting such research designs is that only a limited number of groups may be available, and therefore, simple randomization is unable to adequately balance multiple group-level covariates between arms. Therefore, covariate-based constrained randomization was proposed as an allocation technique to achieve balance. Constrained randomization involves generating a large number of possible allocation schemes, calculating a balance score that assesses covariate imbalance, limiting the randomization space to a prespecified percentage of candidate allocations, and randomly selecting one scheme to implement. When the outcome is binary, a number of statistical issues arise regarding the potential advantages of such designs in making inference. In particular, properties found for continuous outcomes may not directly apply, and additional variations on statistical tests are available. Motivated by two recent trials, we conduct a series of Monte Carlo simulations to evaluate the statistical properties of model-based and randomization-based tests under both simple and constrained randomization designs, with varying degrees of analysis-based covariate adjustment. Our results indicate that constrained randomization improves the power of the linearization F-test, the KC-corrected GEE t-test (Kauermann and Carroll, 2001, Journal of the American Statistical Association 96, 1387-1396), and two permutation tests when the prognostic group-level variables are controlled for in the analysis and the size of randomization space is reasonably small. We also demonstrate that constrained randomization reduces power loss from redundant analysis-based adjustment for non-prognostic covariates. Design considerations such as the choice of the balance metric and the size of randomization space are discussed.

Effectiveness of Two Community Health Worker Models of Tobacco Dependence Treatment Among Community Residents of Ohio Appalachia.

INTRODUCTION: Community health workers (CHW) may be effective in the delivery of tobacco dependence treatment with underserved groups. This study evaluated two evidence-based CHW models of treatment. It was hypothesized that smokers assigned to a CHW face-to-face condition would have higher abstinence at 12-month posttreatment than smokers enrolled in CHW referral to a state-sponsored quitline condition. Intrapersonal and treatment-related factors associated with abstinence at 12 months were determined. METHODS: A group-randomized trial was conducted with residents of 12 Ohio Appalachian counties with counties (n = 6) randomized to either a CHW face-to-face (F2F) or CHW quitline (QL) condition. Both conditions included behavioral counseling and free nicotine replacement therapy for 8 weeks. Follow-up data were collected at 3-, 6-, and 12-month posttreatment. Biochemically validated abstinence at 12 months served as the primary outcome. RESULTS: Seven hundred and seven participants were enrolled (n = 353 CHWF2F; n = 354 CHWQL). Baseline sample characteristics did not differ by condition. Using an intent-to-treat analysis (85.4% retention at 12 months), 13.3% of CHWF2F participants were abstinent at 12 months, compared to 10.7% of CHWQL members (OR = 1.28; 95% confidence interval [CI] = 0.810, 2.014; p = .292). No differences in abstinence were noted at 3 or 6 months by condition. Age, marital status, and baseline levels of cigarette consumption, depressive symptoms, and self-efficacy for quitting in positive settings were associated with abstinence, as was counseling dose during treatment. CONCLUSIONS: This research adds to the body of science evaluating the effectiveness of CHW models of tobacco dependence treatment. Both approaches may offer promise in low-resource settings and underserved regions. IMPLICATIONS: This 12-county community-based group-randomized trial in Ohio Appalachia adds to the body of science evaluating the effectiveness of CHW models of tobacco dependence treatment. Both CHW approaches may offer promise in low-resource settings and underserved regions. These findings are useful to national, state, and local tobacco control agencies, as they expand delivery of preventive health care services postadoption of the Affordable Care Act in the United States.

Teaching Medical Students to Help Patients Quit Smoking: Outcomes of a 10-School Randomized Controlled Trial.

BACKGROUND: Early in medical education, physicians must develop competencies needed for tobacco dependence treatment. OBJECTIVE: To assess the effect of a multi-modal tobacco dependence treatment curriculum on medical students' counseling skills. DESIGN: A group-randomized controlled trial (2010-2014) included ten U.S. medical schools that were randomized to receive either multi-modal tobacco treatment education (MME) or traditional tobacco treatment education (TE). SETTING/PARTICIPANTS: Students from the classes of 2012 and 2014 at ten medical schools participated. Students from the class of 2012 (N = 1345) completed objective structured clinical examinations (OSCEs), and 50 % (N = 660) were randomly selected for pre-intervention evaluation. A total of 72.9 % of eligible students (N = 1096) from the class of 2014 completed an OSCE and 69.7 % (N = 1047) completed pre and post surveys. INTERVENTIONS: The MME included a Web-based course, a role-play classroom demonstration, and a clerkship booster session. Clerkship preceptors in MME schools participated in an academic detailing module and were encouraged to be role models for third-year students. MEASUREMENTS: The primary outcome was student tobacco treatment skills using the 5As measured by an objective structured clinical examination (OSCE) scored on a 33-item behavior checklist. Secondary outcomes were student self-reported skills for performing 5As and pharmacotherapy counseling. RESULTS: Although the difference was not statistically significant, MME students completed more tobacco counseling behaviors on the OSCE checklist (mean 8.7 [SE 0.6] vs. mean 8.0 [SE 0.6], p = 0.52) than TE students. Several of the individual Assist and Arrange items were significantly more likely to have been completed by MME students, including suggesting behavioral strategies (11.8 % vs. 4.5 %, p < 0.001) and providing information regarding quitline (21.0 % vs. 3.8 %, p < 0.001). MME students reported higher self-efficacy for Assist, Arrange, and Pharmacotherapy counseling items (ps ≤0.05). LIMITATIONS: Inclusion of only ten schools limits generalizability. CONCLUSIONS: Subsequent interventions should incorporate lessons learned from this first randomized controlled trial of a multi-modal longitudinal tobacco treatment curriculum in multiple U.S. medical schools. NIH Trial Registry Number: NCT01905618.

An evaluation of constrained randomization for the design and analysis of group-randomized trials.

In group-randomized trials, a frequent practical limitation to adopting rigorous research designs is that only a small number of groups may be available, and therefore, simple randomization cannot be relied upon to balance key group-level prognostic factors across the comparison arms. Constrained randomization is an allocation technique proposed for ensuring balance and can be used together with a permutation test for randomization-based inference. However, several statistical issues have not been thoroughly studied when constrained randomization is considered. Therefore, we used simulations to evaluate key issues including the following: the impact of the choice of the candidate set size and the balance metric used to guide randomization; the choice of adjusted versus unadjusted analysis; and the use of model-based versus randomization-based tests. We conducted a simulation study to compare the type I error and power of the F-test and the permutation test in the presence of group-level potential confounders. Our results indicate that the adjusted F-test and the permutation test perform similarly and slightly better for constrained randomization relative to simple randomization in terms of power, and the candidate set size does not substantially affect their power. Under constrained randomization, however, the unadjusted F-test is conservative, while the unadjusted permutation test carries the desired type I error rate as long as the candidate set size is not too small; the unadjusted permutation test is consistently more powerful than the unadjusted F-test and gains power as candidate set size changes. Finally, we caution against the inappropriate specification of permutation distribution under constrained randomization. An ongoing group-randomized trial is used as an illustrative example for the constrained randomization design.

Statistical lessons learned for designing cluster randomized pragmatic clinical trials from the NIH Health Care Systems Collaboratory Biostatistics and Design Core.

BACKGROUND/AIMS: Pragmatic clinical trials embedded within health care systems provide an important opportunity to evaluate new interventions and treatments. Networks have recently been developed to support practical and efficient studies. Pragmatic trials will lead to improvements in how we deliver health care and promise to more rapidly translate research findings into practice. METHODS: The National Institutes of Health (NIH) Health Care Systems Collaboratory was formed to conduct pragmatic clinical trials and to cultivate collaboration across research areas and disciplines to develop best practices for future studies. Through a two-stage grant process including a pilot phase (UH2) and a main trial phase (UH3), investigators across the Collaboratory had the opportunity to work together to improve all aspects of these trials before they were launched and to address new issues that arose during implementation. Seven Cores were created to address the various considerations, including Electronic Health Records; Phenotypes, Data Standards, and Data Quality; Biostatistics and Design Core; Patient-Reported Outcomes; Health Care Systems Interactions; Regulatory/Ethics; and Stakeholder Engagement. The goal of this article is to summarize the Biostatistics and Design Core's lessons learned during the initial pilot phase with seven pragmatic clinical trials conducted between 2012 and 2014. RESULTS: Methodological issues arose from the five cluster-randomized trials, also called group-randomized trials, including consideration of crossover and stepped wedge designs. We outlined general themes and challenges and proposed solutions from the pilot phase including topics such as study design, unit of randomization, sample size, and statistical analysis. Our findings are applicable to other pragmatic clinical trials conducted within health care systems. CONCLUSION: Pragmatic clinical trials using the UH2/UH3 funding mechanism provide an opportunity to ensure that all relevant design issues have been fully considered in order to reliably and efficiently evaluate new interventions and treatments. The integrity and generalizability of trial results can only be ensured if rigorous designs and appropriate analysis choices are an essential part of their research protocols.

Evaluation of Community-Level Effects of Communities That Care on Adolescent Drug Use and Delinquency Using a Repeated Cross-Sectional Design.

The Communities That Care (CTC) prevention system has shown effects on reducing incidence and prevalence of problem behaviors among a panel of youth followed from 5th through 12th grade. The present report examines whether similar intervention effects could be observed using a repeated cross-sectional design in the same study. Data were from a community-randomized trial of 24 US towns. Cross-sectional samples of sixth, eighth, and tenth graders were surveyed at four waves. Two-stage ANCOVA analyses estimated differences between CTC and control communities in community-level prevalence of problem behaviors for each grade, adjusting for baseline prevalence. No statistically significant reductions in prevalence of problem behaviors were observed at any grade in CTC compared to control communities. Secondary analyses examined intervention effects within a "pseudo cohort" where cross-sectional data were used from sixth graders at baseline and tenth graders 4 years later. When examining effects within the pseudo cohort, CTC compared to control communities showed a significantly slower increase from sixth to tenth grade in lifetime smokeless tobacco use but not for other outcomes. Exploratory analyses showed significantly slower increases in lifetime problem behaviors within the pseudo cohort for CTC communities with high, but not low, prevention program saturation compared to control communities. Although CTC demonstrated effects in a longitudinal panel from the same community-randomized trial, we did not find similar effects on problem behaviors using a repeated cross-sectional design. These differences may be due to a reduced ability to detect effects because of potential cohort effects, accretion of those who were not exposed, and attrition of those who were exposed to CTC programming in the repeated cross-sectional sample.

Multivitamin-mineral use is associated with reduced risk of cardiovascular disease mortality among women in the United States.

BACKGROUND: Multivitamin-mineral (MVM) products are the most commonly used supplements in the United States, followed by multivitamin (MV) products. Two randomized clinical trials (RCTs) did not show an effect of MVMs or MVs on cardiovascular disease (CVD) mortality; however, no clinical trial data are available for women with MVM supplement use and CVD mortality. OBJECTIVE: The objective of this research was to examine the association between MVM and MV use and CVD-specific mortality among US adults without CVD. METHODS: A nationally representative sample of adults from the restricted data NHANES III (1988-1994; n = 8678; age ≥40 y) were matched with mortality data reported by the National Death Index through 2011 to examine associations between MVM and MV use and CVD mortality by using Cox proportional hazards models, adjusting for multiple potential confounders. RESULTS: We observed no significant association between CVD mortality and users of MVMs or MVs compared with nonusers; however, when users were classified by the reported length of time products were used, a significant association was found with MVM use of >3 y compared with nonusers (HR: 0.65; 95% CI: 0.49, 0.85). This finding was largely driven by the significant association among women (HR: 0.56; 95% CI: 0.37, 0.85) but not men (HR: 0.79; 95% CI: 0.44, 1.42). No significant association was observed for MV products and CVD mortality in fully adjusted models. CONCLUSIONS: In this nationally representative data set with detailed information on supplement use and CVD mortality data ∼20 y later, we found an association between MVM use of >3 y and reduced CVD mortality risk for women when models controlled for age, race, education, body mass index, alcohol, aspirin use, serum lipids, blood pressure, and blood glucose/glycated hemoglobin. Our results are consistent with the 1 available RCT in men, indicating no relation with MVM use and CVD mortality.

Enhancing the Quality of Prevention Research Supported by the National Institutes of Health.

As the nation's premier biomedical research agency, the National Institutes of Health (NIH) has supported most of the research that underlies the prevention services that are provided to citizens in the United States and around the world. Within the NIH, the Office of Disease Prevention (ODP) has as its mission to improve the public health by increasing the scope, quality, dissemination, and effect of prevention research supported by the NIH. In today's environment, the ODP needs to focus its efforts to address this mission. To do so, the ODP has developed a strategic plan for 2014 to 2018. We provide background on the ODP and key points from the strategic plan.

Recommendations for choosing an analysis method that controls Type I error for unbalanced cluster sample designs with Gaussian outcomes.

We used theoretical and simulation-based approaches to study Type I error rates for one-stage and two-stage analytic methods for cluster-randomized designs. The one-stage approach uses the observed data as outcomes and accounts for within-cluster correlation using a general linear mixed model. The two-stage model uses the cluster specific means as the outcomes in a general linear univariate model. We demonstrate analytically that both one-stage and two-stage models achieve exact Type I error rates when cluster sizes are equal. With unbalanced data, an exact size α test does not exist, and Type I error inflation may occur. Via simulation, we compare the Type I error rates for four one-stage and six two-stage hypothesis testing approaches for unbalanced data. With unbalanced data, the two-stage model, weighted by the inverse of the estimated theoretical variance of the cluster means, and with variance constrained to be positive, provided the best Type I error control for studies having at least six clusters per arm. The one-stage model with Kenward-Roger degrees of freedom and unconstrained variance performed well for studies having at least 14 clusters per arm. The popular analytic method of using a one-stage model with denominator degrees of freedom appropriate for balanced data performed poorly for small sample sizes and low intracluster correlation. Because small sample sizes and low intracluster correlation are common features of cluster-randomized trials, the Kenward-Roger method is the preferred one-stage approach.

Analytic methods for individually randomized group treatment trials and group-randomized trials when subjects belong to multiple groups.

Participants in trials may be randomized either individually or in groups and may receive their treatment either entirely individually, entirely in groups, or partially individually and partially in groups. This paper concerns cases in which participants receive their treatment either entirely or partially in groups, regardless of how they were randomized. Participants in group-randomized trials are randomized in groups, and participants in individually randomized group treatment trials are individually randomized, but participants in both types of trials receive part or all of their treatment in groups or through common change agents. Participants who receive part or all of their treatment in a group are expected to have positively correlated outcome measurements. This paper addresses a situation that occurs in group-randomized trials and individually randomized group treatment trials-participants receive treatment through more than one group. As motivation, we consider trials in The Childhood Obesity Prevention and Treatment Research Consortium, in which each child participant receives treatment in at least two groups. In simulation studies, we considered several possible analytic approaches over a variety of possible group structures. A mixed model with random effects for both groups provided the only consistent protection against inflated type I error rates and did so at the cost of only moderate loss of power when intraclass correlations were not large. We recommend constraining variance estimates to be positive and using the Kenward-Roger adjustment for degrees of freedom; this combination provided additional power but maintained type I error rates at the nominal level.