Stroke of Consistency: Streamlining Multicenter Protocols for Enhanced Reproducibility of Infarct Volumes in Preclinical Stroke Research.

BACKGROUND: The discrepancy between experimental research and clinical trial outcomes is a persistent challenge in preclinical studies, particularly in stroke research. A possible factor contributing to this issue is the lack of standardization across experimental stroke models, leading to poor reproducibility in multicenter studies. This study addresses this gap by aiming to enhance reproducibility and the efficacy of multicenter studies through the harmonization of protocols and training of involved personnel. METHODS: We established a set of standard operating procedures for various stroke models and the Neuroscore. These standard operating procedures were implemented across multiple research centers, followed by specialized, in-person training for all participants. We measured the variability in infarct volume both before and after the implementation of these standardized protocols and training sessions. RESULTS: The standardization process led to a significant reduction in variability of infarct volume across different stroke models (40%-50% reduction), demonstrating the effectiveness of our harmonized protocols and training. Additionally, the implementation of the Neuroscore system across centers showed low variability and consistent results up to 28 days poststroke, underscoring its utility in chronic phase evaluations. CONCLUSIONS: The harmonization of protocols and surgeon training significantly reduced variability in experimental outcomes across different centers. This improvement can increase the comparability of data between research groups and enhance the statistical power of multicenter studies. Our findings also establish the Neuroscore as a reliable tool for long-term assessment in stroke research, paving the way for more consistent and impactful multicenter preclinical studies.

Mega-analysis methods in ENIGMA: The experience of the generalized anxiety disorder working group.

The ENIGMA group on Generalized Anxiety Disorder (ENIGMA-Anxiety/GAD) is part of a broader effort to investigate anxiety disorders using imaging and genetic data across multiple sites worldwide. The group is actively conducting a mega-analysis of a large number of brain structural scans. In this process, the group was confronted with many methodological challenges related to study planning and implementation, between-country transfer of subject-level data, quality control of a considerable amount of imaging data, and choices related to statistical methods and efficient use of resources. This report summarizes the background information and rationale for the various methodological decisions, as well as the approach taken to implement them. The goal is to document the approach and help guide other research groups working with large brain imaging data sets as they develop their own analytic pipelines for mega-analyses.

Reproducibility in the absence of selective reporting: An illustration from large-scale brain asymmetry research.

The problem of poor reproducibility of scientific findings has received much attention over recent years, in a variety of fields including psychology and neuroscience. The problem has been partly attributed to publication bias and unwanted practices such as p-hacking. Low statistical power in individual studies is also understood to be an important factor. In a recent multisite collaborative study, we mapped brain anatomical left-right asymmetries for regional measures of surface area and cortical thickness, in 99 MRI datasets from around the world, for a total of over 17,000 participants. In the present study, we revisited these hemispheric effects from the perspective of reproducibility. Within each dataset, we considered that an effect had been reproduced when it matched the meta-analytic effect from the 98 other datasets, in terms of effect direction and significance threshold. In this sense, the results within each dataset were viewed as coming from separate studies in an "ideal publishing environment," that is, free from selective reporting and p hacking. We found an average reproducibility rate of 63.2% (SD = 22.9%, min = 22.2%, max = 97.0%). As expected, reproducibility was higher for larger effects and in larger datasets. Reproducibility was not obviously related to the age of participants, scanner field strength, FreeSurfer software version, cortical regional measurement reliability, or regional size. These findings constitute an empirical illustration of reproducibility in the absence of publication bias or p hacking, when assessing realistic biological effects in heterogeneous neuroscience data, and given typically-used sample sizes.

Mitigating site effects in covariance for machine learning in neuroimaging data.

To acquire larger samples for answering complex questions in neuroscience, researchers have increasingly turned to multi-site neuroimaging studies. However, these studies are hindered by differences in images acquired across multiple sites. These effects have been shown to bias comparison between sites, mask biologically meaningful associations, and even introduce spurious associations. To address this, the field has focused on harmonizing data by removing site-related effects in the mean and variance of measurements. Contemporaneously with the increase in popularity of multi-center imaging, the use of machine learning (ML) in neuroimaging has also become commonplace. These approaches have been shown to provide improved sensitivity, specificity, and power due to their modeling the joint relationship across measurements in the brain. In this work, we demonstrate that methods for removing site effects in mean and variance may not be sufficient for ML. This stems from the fact that such methods fail to address how correlations between measurements can vary across sites. Data from the Alzheimer's Disease Neuroimaging Initiative is used to show that considerable differences in covariance exist across sites and that popular harmonization techniques do not address this issue. We then propose a novel harmonization method called Correcting Covariance Batch Effects (CovBat) that removes site effects in mean, variance, and covariance. We apply CovBat and show that within-site correlation matrices are successfully harmonized. Furthermore, we find that ML methods are unable to distinguish scanner manufacturer after our proposed harmonization is applied, and that the CovBat-harmonized data retain accurate prediction of disease group.

Predicting alcohol dependence from multi-site brain structural measures.

To identify neuroimaging biomarkers of alcohol dependence (AD) from structural magnetic resonance imaging, it may be useful to develop classification models that are explicitly generalizable to unseen sites and populations. This problem was explored in a mega-analysis of previously published datasets from 2,034 AD and comparison participants spanning 27 sites curated by the ENIGMA Addiction Working Group. Data were grouped into a training set used for internal validation including 1,652 participants (692 AD, 24 sites), and a test set used for external validation with 382 participants (146 AD, 3 sites). An exploratory data analysis was first conducted, followed by an evolutionary search based feature selection to site generalizable and high performing subsets of brain measurements. Exploratory data analysis revealed that inclusion of case- and control-only sites led to the inadvertent learning of site-effects. Cross validation methods that do not properly account for site can drastically overestimate results. Evolutionary-based feature selection leveraging leave-one-site-out cross-validation, to combat unintentional learning, identified cortical thickness in the left superior frontal gyrus and right lateral orbitofrontal cortex, cortical surface area in the right transverse temporal gyrus, and left putamen volume as final features. Ridge regression restricted to these features yielded a test-set area under the receiver operating characteristic curve of 0.768. These findings evaluate strategies for handling multi-site data with varied underlying class distributions and identify potential biomarkers for individuals with current AD.

INSTRuCT: Protocol, Infrastructure, and Governance.

Background and Purpose: Very few large scale multicentric stroke clinical trials have been done in India. The Indian Council of Medical Research funded INSTRuCT (Indian Stroke Clinical Trial Network) as a task force project with the objectives to establish a state-of-the-art stroke clinical trial network and to conduct pharmacological and nonpharmacological stroke clinical trials relevant to the nation and globally. The purpose of the article is to enumerate the structure of multicentric stroke network, with emphasis on its scope, challenges and expectations in India. Methods: Multiple expert group meetings were conducted by Indian Council of Medical Research to understand the scope of network to perform stroke clinical trials in the country. Established stroke centers with annual volume of 200 patients with stroke with prior experience of conducting clinical trials were included. Central coordinating center, standard operating procedures, data and safety monitoring board were formed. Discussion: In first phase, 2 trials were initiated namely, SPRINT (Secondary Prevention by Structured Semi-Interactive Stroke Prevention Package in India) and Ayurveda treatment in the rehabilitation of patients with ischemic stroke in India (RESTORE [Rehabilitation of Ischemic stroke Patients in India: A Randomized controlled trial]). In second phase, 4 trials have been approved. SPRINT trial was the first to be initiated. SPRINT trial randomized first patient on April 28, 2018; recruited 3048 patients with an average of 128.5 per month so far. The first follow-up was completed on May 27, 2019. RESTORE trial randomized first patient on May 22, 2019; recruited 49 patients with an average of 3.7 per month so far. The first follow-up was completed on August 30, 2019. Conclusions: In next 5 years, INSTRuCT will be able to complete high-quality large scale stroke trials which are relevant globally. REGISTRATION: URL: http://www.ctri.nic.in/; Unique Identifier: CTRI/2017/05/008507.

Resting state fMRI scanner instabilities revealed by longitudinal phantom scans in a multi-center study.

Quality assurance (QA) is crucial in longitudinal and/or multi-site studies, which involve the collection of data from a group of subjects over time and/or at different locations. It is important to regularly monitor the performance of the scanners over time and at different locations to detect and control for intrinsic differences (e.g., due to manufacturers) and changes in scanner performance (e.g., due to gradual component aging, software and/or hardware upgrades, etc.). As part of the Ontario Neurodegenerative Disease Research Initiative (ONDRI) and the Canadian Biomarker Integration Network in Depression (CAN-BIND), QA phantom scans were conducted approximately monthly for three to four years at 13 sites across Canada with 3T research MRI scanners. QA parameters were calculated for each scan using the functional Biomarker Imaging Research Network's (fBIRN) QA phantom and pipeline to capture between- and within-scanner variability. We also describe a QA protocol to measure the full-width-at-half-maximum (FWHM) of slice-wise point spread functions (PSF), used in conjunction with the fBIRN QA parameters. Variations in image resolution measured by the FWHM are a primary source of variance over time for many sites, as well as between sites and between manufacturers. We also identify an unexpected range of instabilities affecting individual slices in a number of scanners, which may amount to a substantial contribution of unexplained signal variance to their data. Finally, we identify a preliminary preprocessing approach to reduce this variance and/or alleviate the slice anomalies, and in a small human data set show that this change in preprocessing can have a significant impact on seed-based connectivity measurements for some individual subjects. We expect that other fMRI centres will find this approach to identifying and controlling scanner instabilities useful in similar studies.

Common functional localizers to enhance NHP & cross-species neuroscience imaging research.

Functional localizers are invaluable as they can help define regions of interest, provide cross-study comparisons, and most importantly, allow for the aggregation and meta-analyses of data across studies and laboratories. To achieve these goals within the non-human primate (NHP) imaging community, there is a pressing need for the use of standardized and validated localizers that can be readily implemented across different groups. The goal of this paper is to provide an overview of the value of localizer protocols to imaging research and we describe a number of commonly used or novel localizers within NHPs, and keys to implement them across studies. As has been shown with the aggregation of resting-state imaging data in the original PRIME-DE submissions, we believe that the field is ready to apply the same initiative for task-based functional localizers in NHP imaging. By coming together to collect large datasets across research group, implementing the same functional localizers, and sharing the localizers and data via PRIME-DE, it is now possible to fully test their robustness, selectivity and specificity. To do this, we reviewed a number of common localizers and we created a repository of well-established localizer that are easily accessible and implemented through the PRIME-RE platform.

Comparison of traveling-subject and ComBat harmonization methods for assessing structural brain characteristics.

Multisite magnetic resonance imaging (MRI) is increasingly used in clinical research and development. Measurement biases-caused by site differences in scanner/image-acquisition protocols-negatively influence the reliability and reproducibility of image-analysis methods. Harmonization can reduce bias and improve the reproducibility of multisite datasets. Herein, a traveling-subject (TS) dataset including 56 T1-weighted MRI scans of 20 healthy participants in three different MRI procedures-20, 19, and 17 subjects in Procedures 1, 2, and 3, respectively-was considered to compare the reproducibility of TS-GLM, ComBat, and TS-ComBat harmonization methods. The minimum participant count required for harmonization was determined, and the Cohen's d between different MRI procedures was evaluated as a measurement-bias indicator. The measurement-bias reduction realized with different methods was evaluated by comparing test-retest scans for 20 healthy participants. Moreover, the minimum subject count for harmonization was determined by comparing test-retest datasets. The results revealed that TS-GLM and TS-ComBat reduced measurement bias by up to 85 and 81.3%, respectively. Meanwhile, ComBat showed a reduction of only 59.0%. At least 6 TSs were required to harmonize data obtained from different MRI scanners, complying with the imaging protocol predetermined for multisite investigations and operated with similar scan parameters. The results indicate that TS-based harmonization outperforms ComBat for measurement-bias reduction and is optimal for MRI data in well-prepared multisite investigations. One drawback is the small sample size used, potentially limiting the applicability of ComBat. Investigation on the number of subjects needed for a large-scale study is an interesting future problem.

Utility-Weighted Modified Rankin Scale Scores for the Assessment of Stroke Outcome: Pooled Analysis of 20 000+ Patients.

BACKGROUND AND PURPOSE: Patient-centered care prioritizes patient beliefs and values towards wellbeing. We aimed to map functional status (modified Rankin Scale [mRS] scores) and health-related quality of life on the European Quality of Life 5-dimensional questionnaire (EQ-5D) to derive utility-weighted (UW) stroke outcome measures and test their statistical properties and construct validity. METHODS: UW-mRS scores were derived using linear regression, with mRS as a discrete ordinal explanatory response variable in 8 large international acute stroke trials. Linear regression models were used to validate UW-mRS scores by assessing differences in mean UW-mRS scores between the treatment groups of each trial. To explore the variability in EQ-5D between individual mRS categories, we generated receiver operator characteristic curves for EQ-5D to differentiate between sequential mRS categories and misclassification matrix to classify individual patients into a matched mRS category based on the closest UW-mRS value to their observed individual EQ-5D value. RESULTS: Among 22 946 acute stroke patients, derived UW-mRS across mRS scores 0 to 6 were 0.96, 0.83, 0.72, 0.54, 0.22, -0.18, and 0, respectively. Both UW-mRS and ordinal mRS scores captured divergent treatment effects across all 8 acute stroke trials. The sample sizes required to detect the treatment effects using UW-mRS scores as a continuous variable were almost half that required in trials for a binary cut point on the mRS. Area under receiver operator characteristic curves based on EQ-5D utility values varied from 0.66 to 0.81. Misclassification matrix showed moderate agreement between actual and matched mRS scores (kappa, 0.68 [95% CI, 0.67-0.68]). CONCLUSIONS: Medical strategies that target avoiding dependency may provide maximum benefit in terms of poststroke health-related quality of life. Despite variable differences with mRS scores, the UW-mRS provides efficiency gains as a smaller sample size is required to detect a treatment effect in acute stroke trials through use of continuous scores. Registration: URL: https://www.clinicaltrials.gov; Unique identifiers: NCT00226096, NCT00716079, NCT01422616, NCT02162017, NCT00120003, NCT02123875. URL: http://ctri.nic.in; Unique identifier: CTRI/2013/04/003557. URL: https://www.isrctn.com; Unique identifier: ISRCTN89712435.

Multivariate consistency of resting-state fMRI connectivity maps acquired on a single individual over 2.5 years, 13 sites and 3 vendors.

Studies using resting-state functional magnetic resonance imaging (rsfMRI) are increasingly collecting data at multiple sites in order to speed up recruitment or increase sample size. The main objective of this study was to assess the long-term consistency of rsfMRI connectivity maps derived at multiple sites and vendors using the Canadian Dementia Imaging Protocol (CDIP, www.cdip-pcid.ca). Nine to 10 min of functional BOLD images were acquired from an adult cognitively healthy volunteer scanned repeatedly at 13 Canadian sites on three scanner makes (General Electric, Philips and Siemens) over the course of 2.5 years. The consistency (spatial Pearson's correlation) of rsfMRI connectivity maps for seven canonical networks ranged from 0.3 to 0.8, with a negligible effect of time, but significant site and vendor effects. We noted systematic differences in data quality (i.e. head motion, number of useable time frames, temporal signal-to-noise ratio) across vendors, which may also confound some of these results, and could not be disentangled in this sample. We also pooled the long-term longitudinal data with a single-site, short-term (1 month) data sample acquired on 26 subjects (10 scans per subject), called HNU1. Using randomly selected pairs of scans from each subject, we quantified the ability of a data-driven unsupervised cluster analysis to match two scans of the same subjects. In this "fingerprinting" experiment, we found that scans from the Canadian subject (Csub) could be matched with high accuracy intra-site (>95% for some networks), but that the accuracy decreased substantially for scans drawn from different sites and vendors, even falling outside of the range of accuracies observed in HNU1. Overall, our results demonstrate good multivariate stability of rsfMRI measures over several years, but substantial impact of scanning site and vendors. How detrimental these effects are will depend on the application, yet our results demonstrate that new methods for harmonizing multisite analysis represent an important area for future work.

Improving the evaluation of COPD exacerbation treatment effects by accounting for early treatment discontinuations: a post-hoc analysis of randomized clinical trials.

BACKGROUND: Chronic obstructive pulmonary disease (COPD) clinical trials aimed at evaluating treatment effects on exacerbations often suffer from early discontinuations of randomized treatment. Treatment discontinuations imply a loss of information and should ideally be considered in the statistical analysis of trial results, particularly if the discontinuations are related to the disease or treatment itself. Here, we explore this issue by investigating (1) whether there exists an association between the risks of exacerbation and treatment discontinuation in COPD clinical trials and (2) whether disregarding this association can cause bias in exacerbation treatment effect estimates. We focus on the hypothetical estimand, i.e. the treatment effect that would have been observed had all subjects completed the trial as planned. METHODS: The association between exacerbation and discontinuation risks was analysed by applying a joint frailty (random effect) model - allowing for the simultaneous analysis of multiple types of correlated events - to data from five Phase III-IV COPD clinical trials. Specifically, the impact of the association on exacerbation treatment effect estimates was assessed by comparing the treatment hazard ratios of the joint frailty model to the rate/hazard ratios of two related statistical models (the negative binomial and shared frailty models), which both assume discontinuations to be unrelated to the trial outcome. The models were also compared using simulated data. RESULTS: A statistically significant (p < 0.0001), positive association between exacerbation and discontinuation risks was found in all trials. Importantly, simulations confirmed that - with such an association - models disregarding the association risk producing biased results (> 5 percentage point difference in hazard/rate ratio). For some treatment comparisons in the clinical trials, the difference in treatment effect estimates between the joint frailty and the other models was as high as 10-15 percentage points. The difference was affected by the strength of the exacerbation-discontinuation association, the population heterogeneity in exacerbation risk, and the difference in discontinuation rates between treatment arms. CONCLUSIONS: We have identified an association between the risks of exacerbation and treatment discontinuation in five COPD clinical trials. We recommend using the joint frailty model to account for this association when estimating exacerbation treatment effects, particularly when targeting the hypothetical estimand.

Development and reliability of questionnaires for the assessment of diet and physical activity behaviors in a multi-country sample in Europe the Feel4Diabetes Study.

BACKGROUND: Assessment of diet and physical activity and their determinants still remains a demanding task, especially when the objective is to evaluate the efficacy of lifestyle interventions. In the context of the Feel4Diabetes study (a European community based intervention study in families with school aged children and at high risk of developing diabetes), we aimed to develop questionnaires for the assessment of food-frequency and eating behaviors, and physical activity and sedentary behaviors in both parents and school-aged children and a questionnaire for overall family's energy balance-related behaviors. METHODS: Questionnaires were developed to be used in 6 countries under standardized harmonization procedures and included questions regarding not only food intake and physical activity, but also questions of their determinants. A reliability study was conducted in 191 pairs of parents and their children (N = 191). Parents completed the questionnaires on two occasions, within a 1-2 week interval. Reliability was tested by the intra-class correlation coefficients (ICC) of test-retest. RESULTS: Most of the questions in all questionnaires had excellent reliability, assessed as an ICC of > 0.810. Mean ICCs for food-frequency and eating behaviors questionnaires were 0.838 and 0.787, and for physical activity and sedentary behaviors questionnaires were 0.734 and 0.793, in adults and children respectively. Mean ICC for overall family's energy balance-related behaviors and their determinants was 0.659. CONCLUSION: The developed questionnaires showed acceptable reliability and may be valuable tools in the assessment of children's and parents' behaviors related to diet, physical activity, sedentary behavior and overall energy balance in school- and community-based interventions.

Considerations for Integration of Perioperative Electronic Health Records Across Institutions for Research and Quality Improvement: The Approach Taken by the Multicenter Perioperative Outcomes Group.

Use of the electronic health record (EHR) has become a routine part of perioperative care in the United States. Secondary use of EHR data includes research, quality, and educational initiatives. Fundamental to secondary use is a framework to ensure fidelity, transparency, and completeness of the source data. In developing this framework, competing priorities must be considered as to which data sources are used and how data are organized and incorporated into a useable format. In assembling perioperative data from diverse institutions across the United States and Europe, the Multicenter Perioperative Outcomes Group (MPOG) has developed methods to support such a framework. This special article outlines how MPOG has approached considerations of data structure, validation, and accessibility to support multicenter integration of perioperative EHRs. In this multicenter practice registry, MPOG has developed processes to extract data from the perioperative EHR; transform data into a standardized format; and validate, deidentify, and transfer data to a secure central Coordinating Center database. Participating institutions may obtain access to this central database, governed by quality and research committees, to inform clinical practice and contribute to the scientific and clinical communities. Through a rigorous and standardized approach to ensure data integrity, MPOG enables data to be usable for quality improvement and advancing scientific knowledge. As of March 2019, our collaboration of 46 hospitals has accrued 10.7 million anesthesia records with associated perioperative EHR data across heterogeneous vendors. Facilitated by MPOG, each site retains access to a local repository containing all site-specific perioperative data, distinct from source EHRs and readily available for local research, quality, and educational initiatives. Through committee approval processes, investigators at participating sites may additionally access multicenter data for similar initiatives. Emerging from this work are 4 considerations that our group has prioritized to improve data quality: (1) data should be available at the local level before Coordinating Center transfer; (2) data should be rigorously validated against standardized metrics before use; (3) data should be curated into computable phenotypes that are easily accessible; and (4) data should be collected for both research and quality improvement purposes because these complementary goals bolster the strength of each endeavor.

International Clinical Research and Justice in the Belmont Report.

The Belmont Report was written by a US Commission charged by the US Congress to advise on research supported by the US government. Its focus was understandably domestic. In the 40 years since its publication, clinical research has become increasingly international. Many clinical trials have sites in multiple countries, and many of the host countries are relatively impoverished. Such research raises some distinctive ethical issues. This paper outlines some of the key ethical challenges that have been raised by clinical research conducted in low- and middle-income countries (LMICs) and sponsored by high-income country (HIC) institutions. It then considers whether the Belmont Report has the resources to address these problems and argues that it does not. The article closes by noting some parallels between this international research and domestic US research, which suggest that the US might benefit from the discussions abroad.

Uncovering multi-site identifiability based on resting-state functional connectomes.

Multi-site studies are becoming important to increase statistical power, enhance generalizability, and to improve the likelihood of pooling relevant subgroups together-activities which are otherwise limited by the availability of subjects or funds at a single site. Even with harmonized imaging sequences, site-dependent variability can mask the advantages of these multi-site studies. The aim of this study was to assess multi-site reproducibility in resting-state functional connectivity "fingerprints", and to improve identifiability of functional connectomes. The individual fingerprinting of functional connectivity profiles is promising due to its potential as a robust neuroimaging biomarker with which to draw single-subject inferences. We evaluated, on two independent multi-site datasets, individual fingerprints in test-retest visit pairs within and across two sites and present a generalized framework based on principal component analysis to improve identifiability. Those principal components that maximized differential identifiability of a training dataset were used as an orthogonal connectivity basis to reconstruct the individual functional connectomes of training and validation sets. The optimally reconstructed functional connectomes showed a substantial improvement in individual fingerprinting of the subjects within and across the two sites and test-retest visit pairs relative to the original data. A notable increase in ICC values for functional edges and resting-state networks were also observed for reconstructed functional connectomes. Improvements in identifiability were not found to be affected by global signal regression. Post-hoc analyses assessed the effect of the number of fMRI volumes on identifiability and showed that multi-site differential identifiability was for all cases maximized after optimal reconstruction. Finally, the generalizability of the optimal set of orthogonal basis of each dataset was evaluated through a leave-one-out procedure. Overall, results demonstrate that the data-driven framework presented in this study systematically improves identifiability in resting-state functional connectomes in multi-site studies.

Challenges and Opportunities for Using Big Health Care Data to Advance Medical Science and Public Health.

Methodological advancements in epidemiology, biostatistics, and data science have strengthened the research world's ability to use data captured from electronic health records (EHRs) to address pressing medical questions, but gaps remain. We describe methods investments that are needed to curate EHR data toward research quality and to integrate complementary data sources when EHR data alone are insufficient for research goals. We highlight new methods and directions for improving the integrity of medical evidence generated from pragmatic trials, observational studies, and predictive modeling. We also discuss needed methods contributions to further ease data sharing across multisite EHR data networks. Throughout, we identify opportunities for training and for bolstering collaboration among subject matter experts, methodologists, practicing clinicians, and health system leaders to help ensure that methods problems are identified and resulting advances are translated into mainstream research practice more quickly.

Detection of atypical data in multicenter clinical trials using unsupervised statistical monitoring.

BACKGROUND/AIMS: A risk-based approach to clinical research may include a central statistical assessment of data quality. We investigated the operating characteristics of unsupervised statistical monitoring aimed at detecting atypical data in multicenter experiments. The approach is premised on the assumption that, save for random fluctuations and natural variations, data coming from all centers should be comparable and statistically consistent. Unsupervised statistical monitoring consists of performing as many statistical tests as possible on all trial data, in order to detect centers whose data are inconsistent with data from other centers. METHODS: We conducted simulations using data from a large multicenter trial conducted in Japan for patients with advanced gastric cancer. The actual trial data were contaminated in computer simulations for varying percentages of centers, percentages of patients modified within each center and numbers and types of modified variables. The unsupervised statistical monitoring software was run by a blinded team on the contaminated data sets, with the purpose of detecting the centers with contaminated data. The operating characteristics (sensitivity, specificity and Youden's J-index) were calculated for three detection methods: one using the p-values of individual statistical tests after adjustment for multiplicity, one using a summary of all p-values for a given center, called the Data Inconsistency Score, and one using both of these methods. RESULTS: The operating characteristics of the three methods were satisfactory in situations of data contamination likely to occur in practice, specifically when a single or a few centers were contaminated. As expected, the sensitivity increased for increasing proportions of patients and increasing numbers of variables contaminated. The three methods showed a specificity better than 93% in all scenarios of contamination. The method based on the Data Inconsistency Score and individual p-values adjusted for multiplicity generally had slightly higher sensitivity at the expense of a slightly lower specificity. CONCLUSIONS: The use of brute force (a computer-intensive approach that generates large numbers of statistical tests) is an effective way to check data quality in multicenter clinical trials. It can provide a cost-effective complement to other data-management and monitoring techniques.

Transitioning to the National Institutes of Health single institutional review board model: Piloting the use of the Streamlined, Multi-site, Accelerated Resources for Trials IRB Reliance.

BACKGROUND/AIMS: Obtaining ethical approval from multiple institutional review boards is a long-standing challenge to multi-site clinical trials and often leads to significant delays in study activation and enrollment. As of 25 January 2018, the National Institutes of Health began requiring use of a single institutional review board for US multi-site trials. To learn more and further inform the research and regulatory communities around aspects of transitioning to single institutional review board review, this study evaluated the efficiency, resource use, and user perceptions of a nascent institutional review board reliance model (Streamlined, Multi-site, Accelerated Resources for Trials IRB Reliance). METHODS: This research was embedded within the Influenza Vaccine to Effectively Stop Cardio Thoracic Events and Decompensated Heart Failure trial-a multi-site trial of two influenza vaccine formulations. In the first year of the trial, a sample of sites agreed to use the developing Streamlined, Multi-site, Accelerated Resources for Trials IRB Reliance model and participated in its evaluation. In keeping with a least burdensome approach, short surveys were developed and obtained from each reporting entity (relying sites, non-relying site, lead site, and reviewing institutional review board). Data regarding time to institutional review board approval and site activation, costs, and user perceptions of reliant review were self-reported and collected via the survey form. Quantitative and qualitative analyses were performed, with costs analyzed as actual versus estimated due to the lack of established baseline cost data. RESULTS: A total of 13 sites ceded review and received institutional review board approval. Mean time to approval was substantially faster in sites that ceded review using the Streamlined, Multi-site, Accelerated Resources for Trials IRB Reliance model versus the site that did not cede review (81 vs 121 days). The mean time to approval was also faster than published averages for academic medical centers (81 vs 103 days). Time to first enrollment was faster for ceding sites versus the non-ceding site, and also faster than published averages (126 vs 149 and 169 days, respectively). Costs were higher than estimates for local institutional review board review and approval. Nearly half (47%) the stakeholders reported being very satisfied or satisfied with the reliance experience, although many noted the challenge related to institutional culture change. CONCLUSION: Implementation of a single institutional review board represents a shift in practice and culture for many institutions. Evaluation of the reliance arrangements for this study highlights both the potential of, and challenges for, institutions as they transition to single institutional review board review. Although efficiencies were observed for study start-up, we anticipate a learning curve as institutions and research teams implement necessary process and resource changes to adapt to single institutional review board oversight. Findings may inform research teams but are, however, limited by the relatively small number of sites and lack of a control group.

Overcoming Challenges in Multisite Trials.

BACKGROUND: Conducting multisite studies has many challenges, including determining the geographic distance between sites, ensuring effective communication, monitoring intervention and data integrity, handling institutional policy variations, seeking institutional review board approval with unique site needs or complex subcontracts, and planning for additional costs. OBJECTIVES: This article discusses common challenges in conducting multisite studies and identifies strategies to overcome these challenges using real-world examples from the literature, the authors' research studies, and their personal experiences. METHODS: A summary of articles on multisite trials conducted within the past 10 years was explored to uncover common challenges in conducting multisite trials. To enrich the context, exemplars from authors' works are included. Based on literature and experience, strategies to combat challenges are summarized. RESULTS: Unique issues related to multisite studies include site selection, use of epicenters/coordinating centers, hiring/managing staff, fidelity monitoring, institutional review board approval, statistical considerations, and approaches to authorship. CONCLUSION: Addressing challenges a priori can improve scientific rigor, reproducibility, and evidence from multisite studies. Given the benefits to scientific rigor, reproducibility, and design, findings from multisite studies are more likely to provide evidence to transform clinical practice and influence policy.