Correction to: To develop a regional ICU mortality prediction model during the first 24 h of ICU admission utilizing MODS and NEMS with six other independent variables from the critical care information system (CCIS) Ontario, Canada.

[This corrects the article DOI: 10.1186/s40560-016-0143-6.].

On the estimation of intracluster correlation for time-to-event outcomes in cluster randomized trials.

Cluster randomized trials (CRTs) involve the random assignment of intact social units rather than independent subjects to intervention groups. Time-to-event outcomes often are endpoints in CRTs. Analyses of such data need to account for the correlation among cluster members. The intracluster correlation coefficient (ICC) is used to assess the similarity among binary and continuous outcomes that belong to the same cluster. However, estimating the ICC in CRTs with time-to-event outcomes is a challenge because of the presence of censored observations. The literature suggests that the ICC may be estimated using either censoring indicators or observed event times. A simulation study explores the effect of administrative censoring on estimating the ICC. Results show that ICC estimators derived from censoring indicators or observed event times are negatively biased. Analytic work further supports these results. Observed event times are preferred to estimate the ICC under minimum frequency of administrative censoring. To our knowledge, the existing literature provides no practical guidance on the estimation of ICC when substantial amount of administrative censoring is present. The results from this study corroborate the need for further methodological research on estimating the ICC for correlated time-to-event outcomes. Copyright © 2016 John Wiley & Sons, Ltd.

To develop a regional ICU mortality prediction model during the first 24 h of ICU admission utilizing MODS and NEMS with six other independent variables from the Critical Care Information System (CCIS) Ontario, Canada.

BACKGROUND: Intensive care unit (ICU) scoring systems or prediction models evolved to meet the desire of clinical and administrative leaders to assess the quality of care provided by their ICUs. The Critical Care Information System (CCIS) is province-wide data information for all Ontario, Canada level 3 and level 2 ICUs collected for this purpose. With the dataset, we developed a multivariable logistic regression ICU mortality prediction model during the first 24 h of ICU admission utilizing the explanatory variables including the two validated scores, Multiple Organs Dysfunctional Score (MODS) and Nine Equivalents Nursing Manpower Use Score (NEMS) followed by the variables age, sex, readmission to the ICU during the same hospital stay, admission diagnosis, source of admission, and the modified Charlson Co-morbidity Index (CCI) collected through the hospital health records. METHODS: This study is a single-center retrospective cohort review of 8822 records from the Critical Care Trauma Centre (CCTC) and Medical-Surgical Intensive Care Unit (MSICU) of London Health Sciences Centre (LHSC), Ontario, Canada between 1 Jan 2009 to 30 Nov 2012. Multivariable logistic regression on training dataset (n = 4321) was used to develop the model and validate by bootstrapping method on the testing dataset (n = 4501). Discrimination, calibration, and overall model performance were also assessed. RESULTS: The predictors significantly associated with ICU mortality included: age (p < 0.001), source of admission (p < 0.0001), ICU admitting diagnosis (p < 0.0001), MODS (p < 0.0001), and NEMS (p < 0.0001). The variables sex and modified CCI were not significantly associated with ICU mortality. The training dataset for the developed model has good discriminating ability between patients with high risk and those with low risk of mortality (c-statistic 0.787). The Hosmer and Lemeshow goodness-of-fit test has a strong correlation between the observed and expected ICU mortality (χ (2) = 5.48; p > 0.31). The overall optimism of the estimation between the training and testing data set ΔAUC = 0.003, indicating a stable prediction model. CONCLUSIONS: This study demonstrates that CCIS data available after the first 24 h of ICU admission at LHSC can be used to create a robust mortality prediction model with acceptable fit statistic and internal validity for valid benchmarking and monitoring ICU performance.

A comparison of confidence interval methods for the intraclass correlation coefficient in community-based cluster randomization trials with a binary outcome.

BACKGROUND: Many investigators rely on previously published point estimates of the intraclass correlation coefficient rather than on their associated confidence intervals to determine the required size of a newly planned cluster randomized trial. Although confidence interval methods for the intraclass correlation coefficient that can be applied to community-based trials have been developed for a continuous outcome variable, fewer methods exist for a binary outcome variable. The aim of this study is to evaluate confidence interval methods for the intraclass correlation coefficient applied to binary outcomes in community intervention trials enrolling a small number of large clusters. Existing methods for confidence interval construction are examined and compared to a new ad hoc approach based on dividing clusters into a large number of smaller sub-clusters and subsequently applying existing methods to the resulting data. METHODS: Monte Carlo simulation is used to assess the width and coverage of confidence intervals for the intraclass correlation coefficient based on Smith's large sample approximation of the standard error of the one-way analysis of variance estimator, an inverted modified Wald test for the Fleiss-Cuzick estimator, and intervals constructed using a bootstrap-t applied to a variance-stabilizing transformation of the intraclass correlation coefficient estimate. In addition, a new approach is applied in which clusters are randomly divided into a large number of smaller sub-clusters with the same methods applied to these data (with the exception of the bootstrap-t interval, which assumes large cluster sizes). These methods are also applied to a cluster randomized trial on adolescent tobacco use for illustration. RESULTS: When applied to a binary outcome variable in a small number of large clusters, existing confidence interval methods for the intraclass correlation coefficient provide poor coverage. However, confidence intervals constructed using the new approach combined with Smith's method provide nominal or close to nominal coverage when the intraclass correlation coefficient is small (<0.05), as is the case in most community intervention trials. CONCLUSION: This study concludes that when a binary outcome variable is measured in a small number of large clusters, confidence intervals for the intraclass correlation coefficient may be constructed by dividing existing clusters into sub-clusters (e.g. groups of 5) and using Smith's method. The resulting confidence intervals provide nominal or close to nominal coverage across a wide range of parameters when the intraclass correlation coefficient is small (<0.05). Application of this method should provide investigators with a better understanding of the uncertainty associated with a point estimator of the intraclass correlation coefficient used for determining the sample size needed for a newly designed community-based trial.

Adjusted inference procedures for the interobserver agreement in twin studies.

We propose adjusted inference procedures for evaluating the agreement/disagreement of two raters in a clustered setting involving twins or paired body parts. These procedures include the construction of a confidence interval for the kappa statistic, a related test of statistical significance and a formula that facilitates sample size estimation. The results of a simulation study suggest that a simple adjustment using an estimated design effect will provide valid inferences. The methods proposed are illustrated using an example from the literature.

Reliability among central readers in the evaluation of endoscopic findings from patients with Crohn's disease.

OBJECTIVE: The Crohn's Disease Endoscopic Index of Severity (CDEIS) and Simple Endoscopic Score for Crohn's Disease (SES-CD) are commonly used to assess Crohn's disease (CD) activity; however, neither instrument has been fully validated. We assessed intra-rater and inter-rater reliability of these indices. DESIGN: Video recordings of colonoscopies obtained from 50 patients with CD who participated in an induction trial of a biological therapy were triplicated and reviewed in random order by four central readers. Data were used to assess intra-rater and inter-rater reliability for CDEIS, SES-CD and a global evaluation of lesion severity (GELS). Subsequently, readers participated in a consensus process that identified common sources of disagreement. RESULTS: Intraclass correlation coefficients (ICCs) for intra-rater reliability for CDEIS, SES-CD and GELS (95% CIs) were 0.89 (0.86 to 0.93), 0.91 (0.89 to 0.95) and 0.81 (0.77 to 0.89), respectively, with standard error of measurement (SEM) of 2.10, 2.42 and 1.15. The corresponding ICCs for inter-rater reliability were 0.71 (0.63 to 0.76), 0.83 (0.75 to 0.88) and 0.62 (0.52 to 0.70), with SEM of 3.42, 3.07 and 1.63, respectively. Correlation between CDEIS and GELS was 0.75, between SES-CD and GELS was 0.74 and between CDEIS and SES-CD was 0.92. The most common sources of disagreement were interpretation of superficial ulceration, definition of disease site at the ileocolonic anastomosis, assessment of anorectal lesions and grading severity of stenosis. CONCLUSIONS: Central reading of CDEIS and SES-CD had 'substantial' to 'almost perfect' intra-rater and inter-rater reliability; however, the responsiveness of these instruments is yet to be determined. TRIAL REGISTRATION NUMBER: Clinicaltrials.gov NCT01466374.

Early combined immunosuppression for the management of Crohn's disease (REACT): a cluster randomised controlled trial.

BACKGROUND: Conventional management of Crohn's disease features incremental use of therapies. However, early combined immunosuppression (ECI), with a TNF antagonist and antimetabolite might be a more effective strategy. We compared the efficacy of ECI with that of conventional management for treatment of Crohn's disease. METHODS: In this open-label cluster randomised controlled trial (Randomised Evaluation of an Algorithm for Crohn's Treatment, REACT), we included community gastroenterology practices from Belgium and Canada that were willing to be assigned to either of the study groups, participate in all aspects of the study, and provide data on up to 60 patients with Crohn's disease. These practices were randomly assigned (1:1) to either ECI or conventional management. The computer-generated randomisation was minimised by country and practice size. Up to 60 consecutive adult patients were assessed within practices. Patients who were aged 18 years or older; documented to have Crohn's disease; able to speak or understand English, French, or Dutch; able to access a telephone; and able to provide written informed consent were followed up for 2 years. The primary outcome was the proportion of patients in corticosteroid-free remission (Harvey-Bradshaw Index score ≤ 4) at 12 months at the practice level. This trial is registered with ClinicalTrials.gov, number NCT01030809. FINDINGS: This study took place between March 15, 2010, and Oct 1, 2013. Of the 60 practices screened, 41 were randomly assigned to either ECI (n=22) or conventional management (n=19). Two practices (one in each group) discontinued because of insufficient resources. 921 (85%) of the 1084 patients at ECI practices and 806 (90%) of 898 patients at conventional management practices completed 12 months follow-up and were included in an intention-to-treat analysis. The 12 month practice-level remission rates were similar at ECI and conventional management practices (66·0% [SD 14·0] and 61·9% [16·9]; adjusted difference 2·5%, 95% CI -5·2% to 10·2%, p=0·5169). The 24 month patient-level composite rate of major adverse outcomes defined as occurrence of surgery, hospital admission, or serious disease-related complications was lower at ECI practices than at conventional management practices (27·7% and 35·1%, absolute difference [AD] 7·3%, hazard ratio [HR]: 0·73, 95% CI 0·62 to 0·86, p=0·0003). There were no differences in serious drug-related adverse events. INTERPRETATION: Although ECI was not more effective than conventional management for controlling Crohn's disease symptoms, the risk of major adverse outcomes was lower. The latter finding should be considered hypothesis-generating for future trials. ECI was not associated with an increased risk of serious drug-related adverse events or mortality. FUNDING: AbbVie Pharmaceuticals.

Propensity score methods for estimating relative risks in cluster randomized trials with low-incidence binary outcomes and selection bias.

Despite randomization, selection bias may occur in cluster randomized trials. Classical multivariable regression usually allows for adjusting treatment effect estimates with unbalanced covariates. However, for binary outcomes with low incidence, such a method may fail because of separation problems. This simulation study focused on the performance of propensity score (PS)-based methods to estimate relative risks from cluster randomized trials with binary outcomes with low incidence. The results suggested that among the different approaches used (multivariable regression, direct adjustment on PS, inverse weighting on PS, and stratification on PS), only direct adjustment on the PS fully corrected the bias and moreover had the best statistical properties.

Variability in research ethics review of cluster randomized trials: a scenario-based survey in three countries.

BACKGROUND: Cluster randomized trials (CRTs) present unique ethical challenges. In the absence of a uniform standard for their ethical design and conduct, problems such as variability in procedures and requirements by different research ethics committees will persist. We aimed to assess the need for ethics guidelines for CRTs among research ethics chairs internationally, investigate variability in procedures for research ethics review of CRTs within and among countries, and elicit research ethics chairs' perspectives on specific ethical issues in CRTs, including the identification of research subjects. The proper identification of research subjects is a necessary requirement in the research ethics review process, to help ensure, on the one hand, that subjects are protected from harm and exploitation, and on the other, that reviews of CRTs are completed efficiently. METHODS: A web-based survey with closed- and open-ended questions was administered to research ethics chairs in Canada, the United States, and the United Kingdom. The survey presented three scenarios of CRTs involving cluster-level, professional-level, and individual-level interventions. For each scenario, a series of questions was posed with respect to the type of review required (full, expedited, or no review) and the identification of research subjects at cluster and individual levels. RESULTS: A total of 189 (35%) of 542 chairs responded. Overall, 144 (84%, 95% CI 79 to 90%) agreed or strongly agreed that there is a need for ethics guidelines for CRTs and 158 (92%, 95% CI 88 to 96%) agreed or strongly agreed that research ethics committees could be better informed about distinct ethical issues surrounding CRTs. There was considerable variability among research ethics chairs with respect to the type of review required, as well as the identification of research subjects. The cluster-cluster and professional-cluster scenarios produced the most disagreement. CONCLUSIONS: Research ethics committees identified a clear need for ethics guidelines for CRTs and education about distinct ethical issues in CRTs. There is disagreement among committees, even within the same countries, with respect to key questions in the ethics review of CRTs. This disagreement reflects variability of opinion and practices pointing toward possible gaps in knowledge, and supports the need for explicit guidelines for the ethical conduct and review of CRTs.