The Trial within Cohorts (TwiCs) study design in oncology: experience and methodological reflections.

A Trial within Cohorts (TwiCs) study design is a trial design that uses the infrastructure of an observational cohort study to initiate a randomized trial. Upon cohort enrollment, the participants provide consent for being randomized in future studies without being informed. Once a new treatment is available, eligible cohort participants are randomly assigned to the treatment or standard of care. Patients randomized to the treatment arm are offered the new treatment, which they can choose to refuse. Patients who refuse will receive standard of care instead. Patients randomized to the standard of care arm receive no information about the trial and continue receiving standard of care as part of the cohort study. Standard cohort measures are used for outcome comparisons. The TwiCs study design aims to overcome some issues encountered in standard Randomized Controlled Trials (RCTs). An example of an issue in standard RCTs is the slow patient accrual. A TwiCs study aims to improve this by selecting patients using a cohort and only offering the intervention to patients in the intervention arm. In oncology, the TwiCs study design has gained increasing interest during the last decade. Despite its potential advantages over RCTs, the TwiCs study design has several methodological challenges that need careful consideration when planning a TwiCs study. In this article, we focus on these challenges and reflect on them using experiences from TwiCs studies initiated in oncology. Important methodological challenges that are discussed are the timing of randomization, the issue of non-compliance (refusal) after randomization in the intervention arm, and the definition of the intention-to-treat effect in a TwiCs study and how this effect is related to its counterpart in standard RCTs.

The STEPWISE study: study protocol for a smartphone-based exercise solution for people with Parkinson's Disease (randomized controlled trial).

BACKGROUND: Exercise has various health benefits for people with Parkinson's disease (PD). However, implementing exercise into daily life and long-term adherence remain challenging. To increase a sustainable engagement with physical activity of people with PD, interventions that are motivating, accessible, and scalable are needed. We primarily aim to investigate whether a smartphone app (STEPWISE app) can increase physical activity (i.e., step count) in people with PD over one year. Our second aim is to investigate the potential effects of the intervention on physical fitness, and motor- and non-motor function. Our third aim is to explore whether there is a dose-response relationship between volume of physical activity and our secondary endpoints. METHODS: STEPWISE is a double-blind, randomized controlled trial. We aim to include 452 Dutch people with PD who can walk independently (Hoehn & Yahr stages 1-3) and who do not take more than 7,000 steps per day prior to inclusion. Physical activity levels are measured as step counts on the participant's own smartphone and scaled as percentage of each participant's baseline. Participants are randomly assigned to an active control group with an increase of 5-20% (active controls) or any of the three intervention arms with increases of 25-100% (intermediate dose), 50-200% (large dose), or 100-400% (very large dose). The primary endpoint is change in step count as measured by the STEPWISE smartphone app from baseline to 52 weeks. For our primary aim, we will evaluate the between-group difference in average daily step count change from baseline to 52 weeks. For our second aim, measures of physical fitness, and motor- and non-motor function are included. For our third aim, we will associate 52-week changes in step count with 52-week changes in secondary outcomes. DISCUSSION: This trial evaluates the potential of a smartphone-based intervention to increase activity levels in people with PD. We envision that motivational apps will increase adherence to physical activity recommendations and could permit conduct of remote clinical trials of exercise for people with PD or those at risk of PD. TRIAL REGISTRATION: ClinicalTrials.gov; NCT04848077; 19/04/2021. CLINICALTRIALS: gov/ct2/show/NCT04848077.

TUBectomy with delayed oophorectomy as an alternative to risk-reducing salpingo-oophorectomy in high-risk women to assess the safety of prevention: the TUBA-WISP II study protocol.

BACKGROUND: Risk-reducing salpingectomy with delayed oophorectomy has gained interest for individuals at high risk for tubo-ovarian cancer as there is compelling evidence that especially high-grade serous carcinoma originates in the fallopian tubes. Two studies have demonstrated a positive effect of salpingectomy on menopause-related quality of life and sexual health compared with standard risk-reducing salpingo-oophorectomy. PRIMARY OBJECTIVE: To investigate whether salpingectomy with delayed oophorectomy is non-inferior to the current standard salpingo-oophorectomy for the prevention of tubo-ovarian cancer among individuals at high inherited risk. STUDY HYPOTHESIS: We hypothesize that postponement of oophorectomy after salpingectomy, to the age of 40-45 (BRCA1) or 45-50 (BRCA2) years, compared with the current standard salpingo-oophorectomy at age 35-40 (BRCA1) or 40-45 (BRCA2) years, is non-inferior in regard to tubo-ovarian cancer risk. TRIAL DESIGN: In this international prospective preference trial, participants will choose between the novel salpingectomy with delayed oophorectomy and the current standard salpingo-oophorectomy. Salpingectomy can be performed after the completion of childbearing and between the age of 25 and 40 (BRCA1), 25 and 45 (BRCA2), or 25 and 50 (BRIP1, RAD51C, and RAD51D pathogenic variant carriers) years. Subsequent oophorectomy is recommended at a maximum delay of 5 years beyond the upper limit of the current guideline age for salpingo-oophorectomy. The current National Comprehensive Cancer Network (NCCN) guideline age, which is also the recommended age for salpingo-oophorectomy within the study, is 35-40 years for BRCA1, 40-45 years for BRCA2, and 45-50 years for BRIP1, RAD51C, and RAD51D pathogenic variant carriers. MAJOR INCLUSION/EXCLUSION CRITERIA: Premenopausal individuals with a documented class IV or V germline pathogenic variant in the BRCA1, BRCA2, BRIP1, RAD51C, or RAD51D gene who have completed childbearing are eligible for participation. Participants may have a personal history of a non-ovarian malignancy. PRIMARY ENDPOINT: The primary outcome is the cumulative tubo-ovarian cancer incidence at the target age: 46 years for BRCA1 and 51 years for BRCA2 pathogenic variant carriers. SAMPLE SIZE: The sample size to ensure sufficient power to test non-inferiority of salpingectomy with delayed oophorectomy compared with salpingo-oophorectomy requires 1500 BRCA1 and 1500 BRCA2 pathogenic variant carriers. ESTIMATED DATES FOR COMPLETING ACCRUAL AND PRESENTING RESULTS: Participant recruitment is expected to be completed at the end of 2026 (total recruitment period of 5 years). The primary outcome is expected to be available in 2036 (minimal follow-up period of 10 years). TRIAL REGISTRATION NUMBER: NCT04294927.

Multiple N-of-1 trials to investigate hypoxia therapy in Parkinson's disease: study rationale and protocol.

BACKGROUND: Parkinson's disease (PD) is a neurodegenerative disease, for which no disease-modifying therapies exist. Preclinical and clinical evidence suggest that hypoxia-based therapy might have short- and long-term benefits in PD. We present the contours of the first study to assess the safety, feasibility and physiological and symptomatic impact of hypoxia-based therapy in individuals with PD. METHODS/DESIGN: In 20 individuals with PD, we will investigate the safety, tolerability and short-term symptomatic efficacy of continuous and intermittent hypoxia using individual, double-blind, randomized placebo-controlled N-of-1 trials. This design allows for dose finding and for including more individualized outcomes, as each individual serves as its own control. A wide range of exploratory outcomes is deployed, including the Movement Disorders Society Unified Parkinson's Disease Rating scale (MDS-UPDRS) part III, Timed Up & Go Test, Mini Balance Evaluation Systems (MiniBES) test and wrist accelerometry. Also, self-reported impression of overall symptoms, motor and non-motor symptoms and urge to take dopaminergic medication will be assessed on a 10-point Likert scale. As part of a hypothesis-generating part of the study, we also deploy several exploratory outcomes to probe possible underlying mechanisms of action, including cortisol, erythropoietin and platelet-derived growth factor ß. Efficacy will be assessed primarily by a Bayesian analysis. DISCUSSION: This evaluation of hypoxia therapy could provide insight in novel pathways that may be pursued for PD treatment. This trial also serves as a proof of concept for deploying an N-of-1 design and for including individualized outcomes in PD research, as a basis for personalized treatment approaches. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT05214287 (registered January 28, 2022).

Estimation of treatment effects in short-term depression studies. An evaluation based on the ICH E9(R1) estimands framework.

Estimands aim to incorporate intercurrent events in design, data collection and estimation of treatment effects in clinical trials. Our aim was to understand what estimands may correspond to efficacy analyses commonly employed in clinical trials conducted before publication of ICH E9(R1). We re-analysed six clinical trials evaluating a new anti-depression treatment. We selected the following analysis methods-ANCOVA on complete cases, following last observation carried forward (LOCF) imputation and following multiple imputation; mixed-models for repeated measurements without imputation (MMRM), MMRM following LOCF imputation and following jump-to-reference imputation; and pattern-mixture mixed models. We included a principal stratum analysis based on the predicted subset of the study population who would not discontinue due to adverse events or lack of efficacy. We translated each analysis into the implicitly targeted estimand, and formulated corresponding clinical questions. We could map six estimands to analysis methods. The same analysis method could be mapped to more than one estimand. The major difference between estimands was the strategy for intercurrent events, with other attributes mostly the same across mapped estimands. The quantitative differences in MADRS10 population-level summaries between the estimands were 4-8 points. Not all six estimands had a clinically meaningful interpretation. Only a few analyses would target the same estimand, hence only few could be used as sensitivity analyses. The fact that an analysis could estimate different estimands emphasises the importance of prospectively defining the estimands targeting the primary objective of a trial. The fact that an estimand can be targeted by different analyses emphasises the importance of prespecifying precisely the estimator for the targeted estimand.

Estimation and expected sample size in Simon's two-stage designs that stop as early as possible.

In early phase clinical studies in oncology, Simon's two-stage designs are widely used. The trial design could be made more efficient by stopping early in the second stage when the required number of responses is reached, or when it has become clear that this target can no longer be met (a form of non-stochastic curtailment). Early stopping, however, will affect proper estimation of the response rate. We propose a uniformly minimum-variance unbiased estimator (UMVUE) for the response rate in this setting. The estimator is proven to be UMVUE using the Rao-Blackwell theorem. We evaluate the estimator's properties in terms of bias and mean squared error, both analytically and via simulations. We derive confidence intervals based on sample space orderings, and assess the coverage. For various design options, we evaluate the reduction in expected sample size as a function of the true response rate. Our method provides a solution for estimating response rates in case of a non-stochastic curtailment Simon's two-stage design.

Combined assessment of early and late-phase outcomes in orphan drug development.

In drug development programs, proof-of-concept Phase II clinical trials typically have a biomarker as a primary outcome, or an outcome that can be observed with relatively short follow-up. Subsequently, the Phase III clinical trials aim to demonstrate the treatment effect based on a clinical outcome that often needs a longer follow-up to be assessed. Early-phase outcomes or biomarkers are typically associated with late-phase outcomes and they are often included in Phase III trials. The decision to proceed to Phase III development is based on analysis of the early-Phase II outcome data. In rare diseases, it is likely that only one Phase II trial and one Phase III trial are available. In such cases and before drug marketing authorization requests, positive results of the early-phase outcome of Phase II trials are then likely seen as supporting (or even replicating) positive Phase III results on the late-phase outcome, without a formal retrospective combined assessment and without accounting for between-study differences. We used double-regression modeling applied to the Phase II and Phase III results to numerically mimic this informal retrospective assessment. We provide an analytical solution for the bias and mean square error of the overall effect that leads to a corrected double-regression. We further propose a flexible Bayesian double-regression approach that minimizes the bias by accounting for between-study differences via discounting the Phase II early-phase outcome when they are not in line with the Phase III biomarker outcome results. We illustrate all methods with an orphan drug example for Fabry disease.

Flexible modelling of risk factors on the incidence of pneumonia in young children in South Africa using piece-wise exponential additive mixed modelling.

INTRODUCTION: Recurrent episodes of pneumonia are frequently modeled using extensions of the Cox proportional hazards model with the underlying assumption of time-constant relative risks measured by the hazard ratio. We aim to relax this assumption in a study on the effect of factors on the evolution of pneumonia incidence over time based on data from a South African birth cohort study, the Drakenstein child health study. METHODS: We describe and apply two models: a time-constant and a time-varying relative effects model in a piece-wise exponential additive mixed model's framework for recurrent events. A more complex model that fits in the same framework is applied to study the continuously measured seasonal effects. RESULTS: We find that several risk factors (male sex, preterm birth, low birthweight, lower socioeconomic status, lower maternal education and maternal cigarette smoking) have strong relative effects that are persistent across time. When time-varying effects are allowed in the model, HIV exposure status (HIV exposed & uninfected versus HIV unexposed) shows a strong relative effect for younger children, but this effect weakens as children grow older, with a null effect reached from about 15 months. Weight-for-length at birth shows a time increasing relative effect. We also find that children born in the summer have a much higher risk of pneumonia in the 3-to-8-month age period compared with children born in winter. CONCLUSION: This work highlights the usefulness of flexible modelling tools in recurrent events models. It avoids stringent assumptions and allows estimation and visualization of absolute and relative risks over time of key factors associated with incidence of pneumonia in young children, providing new perspectives on the role of risk factors such HIV exposure.

A Road Map for Remote Digital Health Technology for Motor Neuron Disease.

Despite recent and potent technological advances, the real-world implementation of remote digital health technology in the care and monitoring of patients with motor neuron disease has not yet been realized. Digital health technology may increase the accessibility to and personalization of care, whereas remote biosensors could optimize the collection of vital clinical parameters, irrespective of patients' ability to visit the clinic. To facilitate the wide-scale adoption of digital health care technology and to align current initiatives, we outline a road map that will identify clinically relevant digital parameters; mediate the development of benefit-to-burden criteria for innovative technology; and direct the validation, harmonization, and adoption of digital health care technology in real-world settings. We define two key end products of the road map: (1) a set of reliable digital parameters to capture data collected under free-living conditions that reflect patient-centric measures and facilitate clinical decision making and (2) an integrated, open-source system that provides personalized feedback to patients, health care providers, clinical researchers, and caregivers and is linked to a flexible and adaptable platform that integrates patient data in real time. Given the ever-changing care needs of patients and the relentless progression rate of motor neuron disease, the adoption of digital health care technology will significantly benefit the delivery of care and accelerate the development of effective treatments.

Prior distributions for variance parameters in a sparse-event meta-analysis of a few small trials.

In rare diseases, typically only a small number of patients are available for a randomized clinical trial. Nevertheless, it is not uncommon that more than one study is performed to evaluate a (new) treatment. Scarcity of available evidence makes it particularly valuable to pool the data in a meta-analysis. When the primary outcome is binary, the small sample sizes increase the chance of observing zero events. The frequentist random-effects model is known to induce bias and to result in improper interval estimation of the overall treatment effect in a meta-analysis with zero events. Bayesian hierarchical modeling could be a promising alternative. Bayesian models are known for being sensitive to the choice of prior distributions for between-study variance (heterogeneity) in sparse settings. In a rare disease setting, only limited data will be available to base the prior on, therefore, robustness of estimation is desirable. We performed an extensive and diverse simulation study, aiming to provide practitioners with advice on the choice of a sufficiently robust prior distribution shape for the heterogeneity parameter. Our results show that priors that place some concentrated mass on small τ values but do not restrict the density for example, the Uniform(-10, 10) heterogeneity prior on the log(τ2 ) scale, show robust 95% coverage combined with less overestimation of the overall treatment effect, across varying degrees of heterogeneity. We illustrate the results with meta-analyzes of a few small trials.

Recall and Outcome of Screen-detected Microcalcifications during 2 Decades of Mammography Screening in the Netherlands National Breast Screening Program.

Background Trends in the detection of suspicious microcalcifications at mammography screening and the yield of these lesions after recall are unknown. Purpose To determine trends in recall and outcome of screen-detected microcalcifications during 20 years of mammography screening. Materials and Methods The authors performed a retrospective analysis of a consecutive series of 817 656 screening examinations (January 1997 to January 2017) in a national breast screening program. In 2009-2010 (transition period), screen-film mammography (SFM) was gradually replaced by full-field digital mammography (FFDM). The recalls of suspicious microcalcifications from all radiology reports and pathologic outcome of recalled women with 2-year follow-up were analyzed. Screening outcome in the era of SFM (1997-2008), the transition period (2009-2010), and the era of FFDM (2011-2016) were compared. Trends over time and variations between the SFM and FFDM periods were expressed by using proportions with 95% confidence intervals (CIs). In cases where the analysis based on the CI confirmed clear periods (eg, before and after introduction of FFDM), pre- and postchange outcomes were compared by using χ2 tests. Results A total of 18 592 women (median age, 59 years; interquartile range, 14 years) were recalled at mammography screening, 3556 of whom had suspicious microcalcifications. The recall rate for microcalcifications increased from 0.1% in 1997-1998 to 0.5% in 2015-2016 (P < .001). This was temporally associated with the change from SFM to FFDM. The recalls yielding ductal carcinoma in situ (DCIS) increased from 0.3 per 1000 screening examinations with SFM to 1.1 per 1000 screening examinations with FFDM (P < .001), resulting in a decrease in the positive predictive value for recall for suspicious microcalcifications from 51% to 33% (P < .001). More than half of all DCIS lesions were high grade (52.6%; 393 of 747). The distribution of DCIS grades was stable during the 20-year screening period (P = .36). Conclusion The recall rate for suspicious microcalcifications at mammographic screening increased during the past 2 decades, whereas the ductal carcinoma in situ detection rate increased less rapidly, resulting in a lower positive predictive value for recall. © RSNA, 2020.

Beyond "Intent-to-treat" and "Per protocol": Improving assessment of treatment effects in clinical trials through the specification of an estimand.

There is a key problem in randomised clinical trials as outcomes can be distorted due to informative post-randomisation events. This is inadequately addressed by the use of traditional intention-to-treat or per protocol analysis sets and often either ignored or wrongly labelled as missing data. As a consequence, the treatment effects of interest in a clinical trial are not well defined and their estimates might be misinterpreted. The estimand framework should help all those planning, conducting and analysing clinical trials as well as those interpreting the results to better define, estimate and understand the treatment effects of interest. This framework is described in the addendum to ICH E9 and addresses precisely this problem. It is relevant for regulatory drug trials and academic-run trials, as well as for trials of nonpharmacological interventions.

Efficacy gap between phase II and subsequent phase III studies in oncology.

AIMS: There is a trend for more flexibility in timing of evidence generation in relation to marketing authorization, including the option to complete phase III trials after authorization or not at all. This paper investigated the relation between phase II and III clinical trial efficacy in oncology. METHODS: All oncology drugs approved by the European Medicines Agency (2007-2016) were included. Phase II and phase III trials were matched based on indication and treatment and patient characteristics. Reported objective response rates (ORR), median progression-free survival (PFS) and median overall survival (OS) were analysed through weighted mixed-effects regression with previous treatment, treatment regimen, blinding, randomization, marketing authorization type and cancer type as covariates. RESULTS: A total of 81 phase II-III matches were identified including 252 trials. Mean (standard deviation) weighted difference (phase III minus II) was -4.2% (17.4) for ORR, 2.1 (6.7) months for PFS and -0.3 (5.1) months for OS, indicating very small average differences between phases. Differences varied substantially between individual indications: from -46.6% to 47.3% for ORR, from -5.3 to 35.9 months for PFS and from -13.3 to 10.8 months for OS. All covariates except blinding were associated with differences in effect sizes for at least 1 outcome. CONCLUSIONS: The lack of marked average differences between phases may encourage decision-makers to regard the quality of design and total body of evidence instead of differentiating between phases of clinical development. The large variability emphasizes that replication of study findings remains essential to confirm efficacy of oncology drugs and discern variables associated with demonstrated effects.

Pragmatic rules for comparability of biological medicinal products.

Comparability is a key concept in the evaluation of both manufacturing changes and biosimilars. It constitutes a pragmatic and flexible approach which recognises that biologicals are inherently variable and that minor variations in quality attributes are often clinically irrelevant. In this discussion paper, we argue that comparability exercises rely on a number of pragmatic criteria. These criteria have been remarkably robust for 20 years of comparability exercises; however, the increased scrutiny of biosimilar applications provides an impetus for both codification and improvement of criteria for establishing comparability. Such a more rigorous, methodologically sound, approach towards comparability seems both feasible and beneficial.

Critical design considerations for time-to-event endpoints in amyotrophic lateral sclerosis clinical trials.

BACKGROUND: Funding and resources for low prevalent neurodegenerative disorders such as amyotrophic lateral sclerosis (ALS) are limited, and optimising their use is vital for efficient drug development. In this study, we review the design assumptions for pivotal ALS clinical trials with time-to-event endpoints and provide optimised settings for future trials. METHODS: We extracted design settings from 13 completed placebo-controlled trials. Optimal assumptions were estimated using parametric survival models in individual participant data (n=4991). Designs were compared in terms of sample size, trial duration, drug use and costs. RESULTS: Previous trials overestimated the hazard rate by 18.9% (95% CI 3.4% to 34.5%, p=0.021). The median expected HR was 0.56 (range 0.33-0.66). Additionally, we found evidence for an increasing mean hazard rate over time (Weibull shape parameter of 2.03, 95% CI 1.93 to 2.15, p<0.001), which affects the design and planning of future clinical trials. Incorporating accrual time and assuming an increasing hazard rate at the design stage reduced sample size by 33.2% (95% CI 27.9 to 39.4), trial duration by 17.4% (95% CI 11.6 to 23.3), drug use by 14.3% (95% CI 9.6 to 19.0) and follow-up costs by 21.2% (95% CI 15.6 to 26.8). CONCLUSIONS: Implementing distributional knowledge and incorporating accrual at the design stage could achieve large gains in the efficiency of ALS clinical trials with time-to-event endpoints. We provide an open-source platform that helps investigators to make more accurate sample size calculations and optimise the use of their available resources.

Level of evidence for promising subgroup findings: The case of trends and multiple subgroups.

Subgroup analyses are an essential part of fully understanding the complete results from confirmatory clinical trials. However, they come with substantial methodological challenges. In case no statistically significant overall treatment effect is found in a clinical trial, this does not necessarily indicate that no patients will benefit from treatment. Subgroup analyses could be conducted to investigate whether a treatment might still be beneficial for particular subgroups of patients. Assessment of the level of evidence associated with such subgroup findings is primordial as it may form the basis for performing a new clinical trial or even drawing the conclusion that a specific patient group could benefit from a new therapy. Previous research addressed the overall type I error and the power associated with a single subgroup finding for continuous outcomes and suitable replication strategies. The current study aims at investigating two scenarios as part of a nonconfirmatory strategy in a trial with dichotomous outcomes: (a) when a covariate of interest is represented by ordered subgroups, eg, in case of biomarkers, and thus, a trend can be studied that may reflect an underlying mechanism, and (b) when multiple covariates, and thus multiple subgroups, are investigated at the same time. Based on simulation studies, this paper assesses the credibility of subgroup findings in overall nonsignificant trials and provides practical recommendations for evaluating the strength of evidence of subgroup findings in these settings.

Dynamic borrowing through empirical power priors that control type I error.

In order for historical data to be considered for inclusion in the design and analysis of clinical trials, prospective rules are essential. Incorporation of historical data may be of particular interest in the case of small populations where available data is scarce and heterogeneity is not as well understood, and thus conventional methods for evidence synthesis might fall short. The concept of power priors can be particularly useful for borrowing evidence from a single historical study. Power priors employ a parameter γ ∈ [ 0 , 1 ] that quantifies the heterogeneity between the historical study and the new study. However, the possibility of borrowing data from a historical trial will usually be associated with an inflation of the type I error. We suggest a new, simple method of estimating the power parameter suitable for the case when only one historical dataset is available. The method is based on predictive distributions and parameterized in such a way that the type I error can be controlled by calibrating to the degree of similarity between the new and historical data. The method is demonstrated for normal responses in a one or two group setting. Generalization to other models is straightforward.

Outlier classification performance of risk adjustment methods when profiling multiple providers.

BACKGROUND: When profiling multiple health care providers, adjustment for case-mix is essential to accurately classify the quality of providers. Unfortunately, misclassification of provider performance is not uncommon and can have grave implications. Propensity score (PS) methods have been proposed as viable alternatives to conventional multivariable regression. The objective was to assess the outlier classification performance of risk adjustment methods when profiling multiple providers. METHODS: In a simulation study based on empirical data, the classification performance of logistic regression (fixed and random effects), PS adjustment, and three PS weighting methods was evaluated when varying parameters such as the number of providers, the average incidence of the outcome, and the percentage of outliers. Traditional classification accuracy measures were considered, including sensitivity and specificity. RESULTS: Fixed effects logistic regression consistently had the highest sensitivity and negative predictive value, yet a low specificity and positive predictive value. Of the random effects methods, PS adjustment and random effects logistic regression performed equally well or better than all the remaining PS methods for all classification accuracy measures across the studied scenarios. CONCLUSIONS: Of the evaluated PS methods, only PS adjustment can be considered a viable alternative to random effects logistic regression when profiling multiple providers in different scenarios.

Meta-analysis approach as a gene selection method in class prediction: does it improve model performance? A case study in acute myeloid leukemia.

BACKGROUND: Aggregating gene expression data across experiments via meta-analysis is expected to increase the precision of the effect estimates and to increase the statistical power to detect a certain fold change. This study evaluates the potential benefit of using a meta-analysis approach as a gene selection method prior to predictive modeling in gene expression data. RESULTS: Six raw datasets from different gene expression experiments in acute myeloid leukemia (AML) and 11 different classification methods were used to build classification models to classify samples as either AML or healthy control. First, the classification models were trained on gene expression data from single experiments using conventional supervised variable selection and externally validated with the other five gene expression datasets (referred to as the individual-classification approach). Next, gene selection was performed through meta-analysis on four datasets, and predictive models were trained with the selected genes on the fifth dataset and validated on the sixth dataset. For some datasets, gene selection through meta-analysis helped classification models to achieve higher performance as compared to predictive modeling based on a single dataset; but for others, there was no major improvement. Synthetic datasets were generated from nine simulation scenarios. The effect of sample size, fold change and pairwise correlation between differentially expressed (DE) genes on the difference between MA- and individual-classification model was evaluated. The fold change and pairwise correlation significantly contributed to the difference in performance between the two methods. The gene selection via meta-analysis approach was more effective when it was conducted using a set of data with low fold change and high pairwise correlation on the DE genes. CONCLUSION: Gene selection through meta-analysis on previously published studies potentially improves the performance of a predictive model on a given gene expression data.

Selecting a classification function for class prediction with gene expression data.

MOTIVATION: Class predicting with gene expression is widely used to generate diagnostic and/or prognostic models. The literature reveals that classification functions perform differently across gene expression datasets. The question, which classification function should be used for a given dataset remains to be answered. In this study, a predictive model for choosing an optimal function for class prediction on a given dataset was devised. RESULTS: To achieve this, gene expression data were simulated for different values of gene-pairs correlations, sample size, genes' variances, deferentially expressed genes and fold changes. For each simulated dataset, ten classifiers were built and evaluated using ten classification functions. The resulting accuracies from 1152 different simulation scenarios by ten classification functions were then modeled using a linear mixed effects regression on the studied data characteristics, yielding a model that predicts the accuracy of the functions on a given data. An application of our model on eight real-life datasets showed positive correlations (0.33-0.82) between the predicted and expected accuracies. CONCLUSION: The here presented predictive model might serve as a guide to choose an optimal classification function among the 10 studied functions, for any given gene expression data. AVAILABILITY AND IMPLEMENTATION: The R source code for the analysis and an R-package 'SPreFuGED' are available at Bioinformatics online. CONTACT: v.l.jong@umcutecht.nl SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.