A survey of experts to identify methods to detect problematic studies: stage 1 of the INveStigating ProblEmatic Clinical Trials in Systematic Reviews project.

BACKGROUND AND OBJECTIVE: Randomized controlled trials (RCTs) inform health-care decisions. Unfortunately, some published RCTs contain false data, and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesize all RCTs which have been conducted on a given topic. This means that any of these 'problematic studies' are likely to be included, but there are no agreed methods for identifying them. The INveStigating ProblEmatic Clinical Trials in Systematic Reviews (INSPECT-SR) project is developing a tool to identify problematic RCTs in systematic reviews of health care-related interventions. The tool will guide the user through a series of 'checks' to determine a study's authenticity. The first objective in the development process is to assemble a comprehensive list of checks to consider for inclusion. METHODS: We assembled an initial list of checks for assessing the authenticity of research studies, with no restriction to RCTs, and categorized these into five domains: Inspecting results in the paper; Inspecting the research team; Inspecting conduct, governance, and transparency; Inspecting text and publication details; Inspecting the individual participant data. We implemented this list as an online survey, and invited people with expertise and experience of assessing potentially problematic studies to participate through professional networks and online forums. Participants were invited to provide feedback on the checks on the list, and were asked to describe any additional checks they knew of, which were not featured in the list. RESULTS: Extensive feedback on an initial list of 102 checks was provided by 71 participants based in 16 countries across five continents. Fourteen new checks were proposed across the five domains, and suggestions were made to reword checks on the initial list. An updated list of checks was constructed, comprising 116 checks. Many participants expressed a lack of familiarity with statistical checks, and emphasized the importance of feasibility of the tool. CONCLUSION: A comprehensive list of trustworthiness checks has been produced. The checks will be evaluated to determine which should be included in the INSPECT-SR tool. PLAIN LANGUAGE SUMMARY: Systematic reviews draw upon evidence from randomized controlled trials (RCTs) to find out whether treatments are safe and effective. The conclusions from systematic reviews are often very influential, and inform both health-care policy and individual treatment decisions. However, it is now clear that the results of many published RCTs are not genuine. In some cases, the entire study may have been fabricated. It is not usual for the veracity of RCTs to be questioned during the process of compiling a systematic review. As a consequence, these "problematic studies" go unnoticed, and are allowed to contribute to the conclusions of influential systematic reviews, thereby influencing patient care. This prompts the question of how these problematic studies could be identified. In this study, we created an extensive list of checks that could be performed to try to identify these studies. We started by assembling a list of checks identified in previous research, and conducting a survey of experts to ask whether they were aware of any additional methods, and to give feedback on the list. As a result, a list of 116 potential "trustworthiness checks" was created. In subsequent research, we will evaluate these checks to see which should be included in a tool, INveStigating ProblEmatic Clinical Trials in Systematic Reviews, which can be used to detect problematic studies.

Concerns about data integrity across 263 papers by one author.

OBJECTIVE: Comprehensive investigation of published work by authors suspected of academic misconduct can reveal further concerns. We aimed to test for data integrity concerns in papers published by an author with eight retracted articles. STUDY DESIGN: We investigated the integrity of all papers reporting on prospective clinical studies by this author. We assessed the feasibility of study methods, baseline characteristics, and outcomes. We plotted the author's clinical research activity over time. We conducted pairwise comparisons of text, tables, and figures to identify duplicate publications, and checked for consistency between conference abstracts, interim analyses, trial registrations, and final papers. Where indicated, we recalculated p-values from the reported summary statistics. RESULTS: We identified 263 papers claiming to have enrolled 74,667 participants between January 2009 and July 2022, 190 (72%) of which reported on studies that recruited from the Assiut Women's Health Hospital in Assiut, Egypt. The number of active studies per month was greatest between 2016 and 2019, with 88 ongoing studies in May 2017. We found evidence of data integrity concerns in 130 (49%) papers, 43 (33%) of which contained concerns sufficient to suggest that they could not be based on data reliably collected from human participants. CONCLUSION: Our investigation finds evidence of widespread integrity concerns in the collected work of one author. We recommend that the involved journals collaborate in a formal investigation.

Data integrity of 14 randomised controlled trials.

BACKGROUND: A review of the literature on iron treatments for iron-deficient anaemia in pregnancy indicated duplication of baseline and outcome tables in two separate randomised controlled trials (RCTs) that share only a single author. AIM: To assess the integrity of randomised clinical trials from Dr A.M. Darwish, Assiut University, Egypt. DESIGN: Assessment of Research Integrity. METHODS: We tabulated the characteristics of studies, compared baseline and outcome tables between articles and looked for implausible findings. We used the distribution of baseline p-values to assess whether the summary statistics of baseline characteristics were consistent with properly conducted randomisation. RESULTS: We identified 14 RCTs (1,405 participants) published between October 2004 and September 2019. Two pairs of studies showed considerable similarities in baseline characteristics, while another pair of studies was plagiarized. The analysis of baseline p-values indicated a low probability that all the studies featured randomised treatment allocation. CONCLUSION: Our analysis of the RCTs of Dr Darwish suggests possible integrity problems. We recommend a critical investigation of the studies that have not been retracted. Until that has been completed, these studies should not be used to inform clinical practice.

A survey of experts to identify methods to detect problematic studies: Stage 1 of the INSPECT-SR Project.

Background: Randomised controlled trials (RCTs) inform healthcare decisions. Unfortunately, some published RCTs contain false data, and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesise all RCTs which have been conducted on a given topic. This means that any of these 'problematic studies' are likely to be included, but there are no agreed methods for identifying them. The INSPECT-SR project is developing a tool to identify problematic RCTs in systematic reviews of healthcare-related interventions. The tool will guide the user through a series of 'checks' to determine a study's authenticity. The first objective in the development process is to assemble a comprehensive list of checks to consider for inclusion. Methods: We assembled an initial list of checks for assessing the authenticity of research studies, with no restriction to RCTs, and categorised these into five domains: Inspecting results in the paper; Inspecting the research team; Inspecting conduct, governance, and transparency; Inspecting text and publication details; Inspecting the individual participant data. We implemented this list as an online survey, and invited people with expertise and experience of assessing potentially problematic studies to participate through professional networks and online forums. Participants were invited to provide feedback on the checks on the list, and were asked to describe any additional checks they knew of, which were not featured in the list. Results: Extensive feedback on an initial list of 102 checks was provided by 71 participants based in 16 countries across five continents. Fourteen new checks were proposed across the five domains, and suggestions were made to reword checks on the initial list. An updated list of checks was constructed, comprising 116 checks. Many participants expressed a lack of familiarity with statistical checks, and emphasized the importance of feasibility of the tool. Conclusions: A comprehensive list of trustworthiness checks has been produced. The checks will be evaluated to determine which should be included in the INSPECT-SR tool.

Educational Outcomes for Children at 7 to 9 Years of Age After Birth at 39 vs 40 to 42 Weeks' Gestation.

Importance: Birth at 39 weeks' gestation is common and thought to be safe for mother and neonate. However, findings of long-term outcomes for children born at this gestational age have been conflicting. Objective: To evaluate the association of birth at 39 weeks' gestation with childhood numeracy and literacy scores at ages 7 to 9 years compared with birth at 40 to 42 weeks' gestation. Design, Setting, and Participants: In this Australian statewide, population-based cohort study using a causal inference framework based on target trial emulation, perinatal data on births between January 1, 2005, and December 31, 2011, were linked to educational outcomes at 7 to 9 years of age. Statistical analyses were performed from December 2022 to June 2023. Exposure: Birth at 39 weeks' gestation compared with birth at 40 to 42 weeks' gestation. Main Outcomes and Measures: Numeracy and literacy outcomes were assessed at 7 to 9 years of age using Australian National Assessment Program-Literacy and Numeracy data and defined by overall z score across 5 domains (grammar and punctuation, reading, writing, spelling, and numeracy). Multiple imputation and doubly robust inverse probability weighted regression adjustment were used to estimate population average causal effects. Results: The study population included 155 575 children. Of these children, 49 456 (31.8%; 24 952 boys [50.5%]) were born at 39 weeks' gestation and were compared with 106 119 (68.2%; 52 083 boys [49.1%]) born at 40 to 42 weeks' gestation. Birth at 39 weeks' gestation was not associated with altered educational outcomes for children aged 7 to 9 years compared with their peers born at 40 to 42 weeks' gestation (mean [SE] z score, 0.0008 [0.0019] vs -0.0031 [0.0038]; adjusted risk difference, -0.004 [95% CI, -0.015 to 0.007]). Each educational domain was investigated, and no significant difference was found in grammar and punctuation (risk difference [RD], -0.006 [95% CI, -0.016 to 0.005]), numeracy (RD, -0.009 [95% CI, -0.020 to 0.001]), spelling (RD, 0.001 [95% CI, -0.011 to 0.0013]), reading (RD, -0.008 [95% CI, -0.019 to 0.003]), or writing (RD, 0.006 [95% CI, -0.005 to 0.016]) scores for children born at 39 weeks' gestation compared with those born at 40 to 42 weeks' gestation. Birth at 39 weeks' gestation also did not increase the risk of scoring below national minimum standards in any of the 5 tested domains. Conclusions and Relevance: Using data from a statewide linkage study to emulate the results of a target randomized clinical trial, this study suggests that there is no evidence of an association of birth at 39 weeks' gestation with numeracy and literacy outcomes for children aged 7 to 9 years.

Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions.

Introduction: Randomised controlled trials (RCTs) inform healthcare decisions. It is now apparent that some published RCTs contain false data and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesise all RCTs that have been conducted on a given topic. While it is usual to assess methodological features of the RCTs in the process of undertaking a systematic review, it is not usual to consider whether the RCTs contain false data. Studies containing false data therefore go unnoticed and contribute to systematic review conclusions. The INSPECT-SR project will develop a tool to assess the trustworthiness of RCTs in systematic reviews of healthcare related interventions. Methods and analysis: The INSPECT-SR tool will be developed using expert consensus in combination with empirical evidence, over five stages: 1) a survey of experts to assemble a comprehensive list of checks for detecting problematic RCTs, 2) an evaluation of the feasibility and impact of applying the checks to systematic reviews, 3) a Delphi survey to determine which of the checks are supported by expert consensus, culminating in 4) a consensus meeting to select checks to be included in a draft tool and to determine its format, 5) prospective testing of the draft tool in the production of new health systematic reviews, to allow refinement based on user feedback. We anticipate that the INSPECT-SR tool will help researchers to identify problematic studies, and will help patients by protecting them from the influence of false data on their healthcare.

Benefits and harms of breast cancer screening: Cohort study of breast cancer mortality and overdiagnosis.

BACKGROUND: Quantifying the benefits and harms of breast cancer screening accurately is important for planning and evaluating screening programs and for enabling women to make informed decisions about participation. However, few cohort studies have attempted to estimate benefit and harm simultaneously. AIMS: We aimed to quantify the impact of mammographic screening on breast cancer mortality and overdiagnosis using a cohort of women invited to attend Australia's national screening program, BreastScreen. METHODS: In a cohort of 41,330 women without prior breast cancer diagnosis, screening, or diagnostic procedures invited to attend BreastScreen Western Australia in 1994-1995, we estimated the cumulative risk of breast cancer mortality and breast cancer incidence (invasive and ductal carcinoma in situ) from age 50 to 85 years for attenders and non-attenders. Data were obtained by linking population-based state and national health registries. Breast cancer mortality risks were estimated from a survival analysis that accounted for competing risk of death from other causes. Breast cancer risk for unscreened women was estimated by survival analysis, while accounting for competing causes of death. For screened women, breast cancer risk was the sum of risk of being diagnosed at first screen, estimated using logistic regression, and risk of diagnosis following a negative first screen estimated from a survival analysis. RESULTS: For every 1,000 women 50 years old at first invitation to attend BreastScreen, there were 20 (95% CI 12-30) fewer breast cancer deaths and 25 (95% CI 15-35) more breast cancers diagnosed for women who attended than for non-attendees by age 85. Of the breast cancers diagnosed in screened women, 21% (95% CI 13%-27%) could be attributed to screening. DISCUSSION: The estimated ratio of benefit to harm was consistent with, but slightly less favourable to screening than most other estimates from cohort studies. CONCLUSION: Women who participate in organised screening for breast cancer in Australia have substantially lower breast cancer mortality, while some screen-detected cancers may be overdiagnosed.

Checklist to assess Trustworthiness in RAndomised Controlled Trials (TRACT checklist): concept proposal and pilot.

OBJECTIVES: To propose a checklist that can be used to assess trustworthiness of randomized controlled trials (RCTs). DESIGN: A screening tool was developed using the four-stage approach proposed by Moher et al. This included defining the scope, reviewing the evidence base, suggesting a list of items from piloting, and holding a consensus meeting. The initial checklist was set-up by a core group who had been involved in the assessment of problematic RCTs for several years. We piloted this in a consensus panel of several stakeholders, including health professionals, reviewers, journal editors, policymakers, researchers, and evidence-synthesis specialists. Each member was asked to score three articles with the checklist and the results were then discussed in consensus meetings. OUTCOME: The Trustworthiness in RAndomised Clinical Trials (TRACT) checklist includes 19 items organised into seven domains that are applicable to every RCT: 1) Governance, 2) Author Group, 3) Plausibility of Intervention Usage, 4) Timeframe, 5) Drop-out Rates, 6) Baseline Characteristics, and 7) Outcomes. Each item can be answered as either no concerns, some concerns/no information, or major concerns. If a study is assessed and found to have a majority of items rated at a major concern level, then editors, reviewers or evidence synthesizers should consider a more thorough investigation, including assessment of original individual participant data. CONCLUSIONS: The TRACT checklist is the first checklist developed specifically to detect trustworthiness issues in RCTs. It might help editors, publishers and researchers to screen for such issues in submitted or published RCTs in a transparent and replicable manner.

School-age outcomes among IVF-conceived children: A population-wide cohort study.

BACKGROUND: In vitro fertilisation (IVF) is a common mode of conception. Understanding the long-term implications for these children is important. The aim of this study was to determine the causal effect of IVF conception on primary school-age childhood developmental and educational outcomes, compared with outcomes following spontaneous conception. METHODS AND FINDINGS: Causal inference methods were used to analyse observational data in a way that emulates a target randomised clinical trial. The study cohort comprised statewide linked maternal and childhood administrative data. Participants included singleton infants conceived spontaneously or via IVF, born in Victoria, Australia between 2005 and 2014 and who had school-age developmental and educational outcomes assessed. The exposure examined was conception via IVF, with spontaneous conception the control condition. Two outcome measures were assessed. The first, childhood developmental vulnerability at school entry (age 4 to 6), was assessed using the Australian Early Developmental Census (AEDC) (n = 173,200) and defined as scoring <10th percentile in ≥2/5 developmental domains (physical health and wellbeing, social competence, emotional maturity, language and cognitive skills, communication skills, and general knowledge). The second, educational outcome at age 7 to 9, was assessed using National Assessment Program-Literacy and Numeracy (NAPLAN) data (n = 342,311) and defined by overall z-score across 5 domains (grammar and punctuation, reading, writing, spelling, and numeracy). Inverse probability weighting with regression adjustment was used to estimate population average causal effects. The study included 412,713 children across the 2 outcome cohorts. Linked records were available for 4,697 IVF-conceived cases and 168,503 controls for AEDC, and 8,976 cases and 333,335 controls for NAPLAN. There was no causal effect of IVF-conception on the risk of developmental vulnerability at school-entry compared with spontaneously conceived children (AEDC metrics), with an adjusted risk difference of -0.3% (95% CI -3.7% to 3.1%) and an adjusted risk ratio of 0.97 (95% CI 0.77 to 1.25). At age 7 to 9 years, there was no causal effect of IVF-conception on the NAPLAN overall z-score, with an adjusted mean difference of 0.030 (95% CI -0.018 to 0.077) between IVF- and spontaneously conceived children. The models were adjusted for sex at birth, age at assessment, language background other than English, socioeconomic status, maternal age, parity, and education. Study limitations included the use of observational data, the potential for unmeasured confounding, the presence of missing data, and the necessary restriction of the cohort to children attending school. CONCLUSIONS: In this analysis, under the given causal assumptions, the school-age developmental and educational outcomes for children conceived by IVF are equivalent to those of spontaneously conceived children. These findings provide important reassurance for current and prospective parents and for clinicians.

Emulating a target trial of intensive nurse home visiting in the policy-relevant population using linked administrative data.

BACKGROUND: Populations willing to participate in randomized trials may not correspond well to policy-relevant target populations. Evidence of effectiveness that is complementary to randomized trials may be obtained by combining the 'target trial' causal inference framework with whole-of-population linked administrative data. METHODS: We demonstrate this approach in an evaluation of the South Australian Family Home Visiting Program, a nurse home visiting programme targeting socially disadvantaged families. Using de-identified data from 2004-10 in the ethics-approved Better Evidence Better Outcomes Linked Data (BEBOLD) platform, we characterized the policy-relevant population and emulated a trial evaluating effects on child developmental vulnerability at 5 years (n = 4160) and academic achievement at 9 years (n = 6370). Linkage to seven health, welfare and education data sources allowed adjustment for 29 confounders using Targeted Maximum Likelihood Estimation (TMLE) with SuperLearner. Sensitivity analyses assessed robustness to analytical choices. RESULTS: We demonstrated how the target trial framework may be used with linked administrative data to generate evidence for an intervention as it is delivered in practice in the community in the policy-relevant target population, and considering effects on outcomes years down the track. The target trial lens also aided in understanding and limiting the increased measurement, confounding and selection bias risks arising with such data. Substantively, we did not find robust evidence of a meaningful beneficial intervention effect. CONCLUSIONS: This approach could be a valuable avenue for generating high-quality, policy-relevant evidence that is complementary to trials, particularly when the target populations are multiply disadvantaged and less likely to participate in trials.

Association Between Earlier Introduction of Peanut and Prevalence of Peanut Allergy in Infants in Australia.

Importance: Randomized clinical trials showed that earlier peanut introduction can prevent peanut allergy in select high-risk populations. This led to changes in infant feeding guidelines in 2016 to recommend early peanut introduction for all infants to reduce the risk of peanut allergy. Objective: To measure the change in population prevalence of peanut allergy in infants after the introduction of these new guidelines and evaluate the association between early peanut introduction and peanut allergy. Design: Two population-based cross-sectional samples of infants aged 12 months were recruited 10 years apart using the same sampling frame and methods to allow comparison of changes over time. Infants were recruited from immunization centers around Melbourne, Australia. Infants attending their 12-month immunization visit were eligible to participate (eligible age range, 11-15 months), regardless of history of peanut exposure or allergy history. Exposures: Questionnaires collected data on demographics, food allergy risk factors, peanut introduction, and reactions. Main Outcome and Measures: All infants underwent skin prick tests to peanut and those with positive results underwent oral food challenges. Prevalence estimates were standardized to account for changes in population demographics over time. Results: This study included 7209 infants (1933 in 2018-2019 and 5276 in 2007-2011). Of the participants in the older vs more recent cohort, 51.8% vs 50.8% were male; median (IQR) ages were 12.5 (12.2-13.0) months vs 12.4 (12.2-12.9) months. There was an increase in infants of East Asian ancestry over time (16.5% in 2018-2019 vs 10.5% in 2007-2011), which is a food allergy risk factor. After standardizing for infant ancestry and other demographics changes, peanut allergy prevalence was 2.6% (95% CI, 1.8%-3.4%) in 2018-2019, compared with 3.1% in 2007-2011 (difference, -0.5% [95% CI, -1.4% to 0.4%]; P = .26). Earlier age of peanut introduction was significantly associated with a lower risk of peanut allergy among infants of Australian ancestry in 2018-2019 (age 12 months compared with age 6 months or younger: adjusted odds ratio, 0.08 [05% CI, 0.02-0.36]; age 12 months compared with 7 to less than 10 months: adjusted odds ratio, 0.09 [95% CI, 0.02-0.53]), but not significant among infants of East Asian ancestry (P for interaction = .002). Conclusions and Relevance: In cross-sectional analyses, introduction of a guideline recommending early peanut introduction in Australia was not associated with a statistically significant lower or higher prevalence of peanut allergy across the population.

The natural history of peanut and egg allergy in children up to age 6 years in the HealthNuts population-based longitudinal study.

BACKGROUND: Prospectively collected data on the natural history of food allergy are lacking. OBJECTIVE: We examined the natural history of egg and peanut allergy in children from age 1 to 6 years and assessed whether a skin prick test (SPT) result or other clinical factors at diagnosis are associated with the persistence or resolution of food allergy in early childhood. METHODS: The HealthNuts cohort consists of 5276 children who were recruited at age 1 year and have been followed prospectively. Children with food allergy at age 1 year (peanut [n = 156] or raw egg [n = 471] allergy ) and children who developed new sensitizations or food reactions after age 1 year were assessed for food sensitization and allergy (confirmed by oral food challenge when indicated) at the 6-year follow-up. RESULTS: New-onset food allergy developed by age 6 years was more common for peanut (0.7% [95% CI = 0.5%-1.1%]) than egg (0.09% [95% CI = 0.03%-0.3%]). Egg allergy resolved more commonly (89% [95% CI = 85%-92%]) than peanut allergy (29% [95% CI = 22%-38%]) by age 6 years. The overall weighted prevalence of peanut allergy at age 6 years was 3.1% (95% CI = 2.6-3.7%) and that of egg allergy was 1.2% (95% = CI 0.9%-1.6%). The factors at age 1 year associated with persistence of peanut allergy were peanut SPT result of 8 mm or larger (odds ratio [OR] = 2.35 [95% CI 1.08-5.12]), sensitization to tree nuts (adjusted OR [aOR] = 2.51 [95% CI = 1.00-6.35]), and early-onset severe eczema (aOR = 3.23, [95% CI 1.17-8.88]). Factors at age 1 associated with persistence of egg allergy at age 6 were egg SPT result of 4 mm or larger (OR = 2.98 [95% CI 1.35-6.36]), other (peanut and/or sesame) food sensitizations (aOR = 2.80 [95% CI = 1.11-7.03]), baked egg allergy (aOR = 7.41 [95% CI = 2.16-25.3]), and early-onset severe eczema (aOR = 3.77 [95% CI = 1.35-10.52]). CONCLUSION: Most egg allergy and nearly one-third of peanut allergy resolves naturally by age 6 years. The prevalence of peanut allergy at age 6 years was similar to that observed at age 1 year, largely owing to new-onset food peanut allergy after age 1 year. Infants with early-onset eczema, larger SPT wheals, or multiple food sensitizations and/or allergies were less likely to acquire tolerance to either peanut or egg.

Violation of research integrity principles occurs more often than we think.

The science community generally believes that the violation of research integrity is rare. Built upon this belief, the scientific system makes little effort to examine the trustworthiness of research. Research misconduct refers to an intentional violation of research integrity principles, which has an extensive and far-reaching impact on the trustworthiness and reputation of science. Emerging evidence has suggested that research misconduct is far more common than we normally perceive. Far more problematic papers should be retracted than are being retracted because of poor actions when confronting research misconduct. Research misconduct is usually driven by incentives in the form of pursuing publications for researchers' career needs and is further facilitated by poor research governance. The current strategy that tackles potential research misconduct focuses on protecting the reputation of authors and their institutions but neglects the interests of patients, clinicians and honest researchers. Removing improper incentives, training researchers and imposing better governance are vital to reducing research misconduct. Awareness of the possibility of misconduct and formalized procedures that scrutinize study trustworthiness are important during peer review and in systematic reviews.

An investigation of seven other publications by the first author of a retracted paper due to doubts about data integrity.

BACKGROUND: In 2019, a randomized controlled trial (RCT) authored by Dr. Ismail was retracted due to concerns about data integrity. Since there are no policies in place to investigate other publications of authors of retracted studies, we investigated Dr. Ismail's other trials. METHODS: We searched for RCTs authored by Dr. Ismail. We made pairwise comparisons of values in baseline and outcome tables between trials. We assessed whether the distributions of baseline characteristics were compatible with properly conducted randomization, using Monte Carlo simulations and the Kolmogorov-Smirnov test. We read the publications carefully for unusual features. RESULTS: Dr. Ismail was author in eight published and one unpublished trial. In three of his first author studies we found multiple identical values in the baseline and/or outcome tables from different trials. At least some of the trials were unlikely to have followed a proper randomization process. There were a number of other unusual features in the papers we reviewed. CONCLUSIONS: It is probable that other trials published by Dr. Ismail contain questionable data. We call for a thorough investigation of the original trial data and related official documents. Our exercise suggests that the practice to assess research integrity should include all publications of authors with retracted fabricated articles.

To share or not to share data: how valid are trials evaluating first-line ovulation induction for polycystic ovary syndrome?

BACKGROUND: In our recent individual participant data (IPD) meta-analysis evaluating the effectiveness of first-line ovulation induction for polycystic ovary syndrome (PCOS), IPD were only available from 20 studies of 53 randomized controlled trials (RCTs). We noticed that the summary effect sizes of meta-analyses of RCTs without IPD sharing were different from those of RCTs with IPD sharing. Granting access to IPD for secondary analysis has implications for promoting fair and transparent conduct of RCTs. It is, however, still common for authors to choose to withhold IPD, limiting the impact of and confidence in the results of RCTs and systematic reviews based on aggregate data. OBJECTIVE AND RATIONALE: We performed a meta-epidemiologic study to elucidate if RCTs without IPD sharing have lower quality and more methodological issues than those with IPD sharing in an IPD meta-analysis evaluating first-line ovulation induction for PCOS. SEARCH METHODS: We included RCTs identified for the IPD meta-analysis. We dichotomized RCTs according to whether they provided IPD (shared group) or not (non-shared group) in the IPD meta-analysis. We restricted RCTs to full-text published trials written in English.We assessed and compared RCTs in the shared and non-shared groups on the following criteria: Risk of Bias (RoB 2.0), GRADE approach, adequacy of trial registration; description of statistical methods and reproducibility of univariable statistical analysis; excessive similarity or difference in baseline characteristics that is not compatible with chance; and other miscellaneous methodological issues. OUTCOMES: In total, 45 trials (8697 women) were included in this study. IPD were available from 17 RCTs and 28 trials were categorized as the non-shared IPD group. Pooled risk rates obtained from the shared and non-shared groups were different. Overall low risk of bias was associated with 13/17 (76%) of shared RCTs versus 7/28 (25%) of non-shared RCTs. For RCTs that started recruitment after 1 July 2005, adequate trial registration was found in 3/9 (33%) of shared IPD RCTs versus 0/16 (0%) in non-shared RCTs. In total, 7/17 (41%) of shared RCTs and 19/28 (68%) of non-shared RCTs had issues with the statistical methods described. The median (range) of inconsistency rate per study, between reported and reproduced analyses for baseline variables, was 0% (0-92%) (6 RCTs applicable) in the shared group and 54% (0-100%) (13 RCTs applicable) in the non-shared group. The median (range) of inconsistency rate of univariable statistical results for the outcome(s) per study was 0% (0-63%) (14 RCTs applicable) in the shared group and 44% (0-100%) (24 RCTs applicable) in the non-shared group. The distributions of simulation-generated P-values from comparisons of baseline continuous variables between intervention and control arms suggested that RCTs in the shared group are likely to be consistent with properly conducted randomization (P = 0.163), whereas this was not the case for the RCTs in the non-shared group (P = 4.535 × 10-8). WIDER IMPLICATIONS: IPD meta-analysis on evaluating first-line ovulation induction for PCOS preserves validity and generates more accurate estimates of risk than meta-analyses using aggregate data, which enables more transparent assessments of benefits and risks. The availability of IPD and the willingness to share these data may be a good indicator of quality, methodological soundness and integrity of RCTs when they are being considered for inclusion in systematic reviews and meta-analyses.

Data integrity of 35 randomised controlled trials in women' health.

While updating a systematic review on the topic of ovulation of induction, we observed unusual similarities in a number of randomised controlled trials (RCTs) published by two authors from the same institute in the same disease spectrum in a short period of time. We therefore undertook a focused analysis of the data integrity of all RCTs published by the two authors. We made pairwise comparisons to find identical or similar values in baseline characteristics and outcome tables between trials. We also assessed whether baseline characteristics were compatible with chance, using Monte Carlo simulations and Kolmogorov-Smirnov test. For 35 trials published between September 2006 and January 2016, we found a large number of similarities in both the baseline characteristics and outcomes of 26. Analysis of the baseline characteristics of the trials indicated that their distribution was unlikely to be the result of proper randomisation. The procedures demonstrated in this paper may help to assess data integrity in future attempts to verify the authenticity of published RCTs.