Meta-epidemiology and the MetaBLIND study: Response to Tack.

In 2020 we published MetaBLIND, a large meta-epidemiological study on the impact of masking on effect estimates in randomized clinical trials (RCTs). While masking is an established methodological practice in RCTs it is not clear to what extent results of non-masked RCTs are biased. Surprisingly, we found no evidence of an impact of masking on effect estimates, on average. Michiel Tack commented on the MetaBLIND study, and here we respond. The issues he raised were examples of standard themes when interpreting meta-epidemiological studies, which we have discussed at some length elsewhere, and did not warrant change of our conclusion. We maintain that the MetaBLIND results do not provide a sufficient basis for recommending abandoning masking as a methodological safeguard.

Impact of blinding on estimated treatment effects in randomised clinical trials: meta-epidemiological study.

OBJECTIVES: To study the impact of blinding on estimated treatment effects, and their variation between trials; differentiating between blinding of patients, healthcare providers, and observers; detection bias and performance bias; and types of outcome (the MetaBLIND study). DESIGN: Meta-epidemiological study. DATA SOURCE: Cochrane Database of Systematic Reviews (2013-14). ELIGIBILITY CRITERIA FOR SELECTING STUDIES: Meta-analyses with both blinded and non-blinded trials on any topic. REVIEW METHODS: Blinding status was retrieved from trial publications and authors, and results retrieved automatically from the Cochrane Database of Systematic Reviews. Bayesian hierarchical models estimated the average ratio of odds ratios (ROR), and estimated the increases in heterogeneity between trials, for non-blinded trials (or of unclear status) versus blinded trials. Secondary analyses adjusted for adequacy of concealment of allocation, attrition, and trial size, and explored the association between outcome subjectivity (high, moderate, low) and average bias. An ROR lower than 1 indicated exaggerated effect estimates in trials without blinding. RESULTS: The study included 142 meta-analyses (1153 trials). The ROR for lack of blinding of patients was 0.91 (95% credible interval 0.61 to 1.34) in 18 meta-analyses with patient reported outcomes, and 0.98 (0.69 to 1.39) in 14 meta-analyses with outcomes reported by blinded observers. The ROR for lack of blinding of healthcare providers was 1.01 (0.84 to 1.19) in 29 meta-analyses with healthcare provider decision outcomes (eg, readmissions), and 0.97 (0.64 to 1.45) in 13 meta-analyses with outcomes reported by blinded patients or observers. The ROR for lack of blinding of observers was 1.01 (0.86 to 1.18) in 46 meta-analyses with subjective observer reported outcomes, with no clear impact of degree of subjectivity. Information was insufficient to determine whether lack of blinding was associated with increased heterogeneity between trials. The ROR for trials not reported as double blind versus those that were double blind was 1.02 (0.90 to 1.13) in 74 meta-analyses. CONCLUSION: No evidence was found for an average difference in estimated treatment effect between trials with and without blinded patients, healthcare providers, or outcome assessors. These results could reflect that blinding is less important than often believed or meta-epidemiological study limitations, such as residual confounding or imprecision. At this stage, replication of this study is suggested and blinding should remain a methodological safeguard in trials.

Ten questions to consider when interpreting results of a meta-epidemiological study-the MetaBLIND study as a case.

Randomized clinical trials underpin evidence-based clinical practice, but flaws in their conduct may lead to biased estimates of intervention effects and hence invalid treatment recommendations. The main approach to the empirical study of bias is to collate a number of meta-analyses and, within each, compare the results of trials with and without a methodological characteristic such as blinding of participants and health professionals. Estimated within-meta-analysis differences are combined across meta-analyses, leading to an estimate of mean bias. Such "meta-epidemiological" studies are published in increasing numbers and have the potential to inform trial design, assessment of risk of bias, and reporting guidelines. However, their interpretation is complicated by issues of confounding, imprecision, and applicability. We developed a guide for interpreting meta-epidemiological studies, illustrated using MetaBLIND, a large study on the impact of blinding. Applying generally accepted principles of research methodology to meta-epidemiology, we framed 10 questions covering the main issues to consider when interpreting results of such studies, including risk of systematic error, risk of random error, issues related to heterogeneity, and theoretical plausibility. We suggest that readers of a meta-epidemiological study reflect comprehensively on the research question posed in the study, whether an experimental intervention was unequivocally identified for all included trials, the risk of misclassification of the trial characteristic, and the risk of confounding, i.e the adequacy of any adjustment for the likely confounders. We hope that our guide to interpretation of results of meta-epidemiological studies is helpful for readers of such studies.

Unreported formal assessment of unblinding occurred in 4 of 10 randomized clinical trials, unreported loss of blinding in 1 of 10 trials.

OBJECTIVES: Randomized clinical trials often involve blinding as a methodological procedure to avoid bias. Unfortunately, blinding procedures may be unsuccessful, but the risk of unblinding is rarely reported in trial publications. Our primary aim was to assess the occurrence of unreported assessment of the risk of unblinding in randomized clinical trials and to describe the assessment procedures involved. Our secondary aim was to assess the occurrence of unreported suspected or overt unblinding and the mechanisms of unblinding. STUDY DESIGN AND SETTING: A Web-based questionnaire survey of authors to trial publications which did not report risk of unblinding. Respondents were corresponding authors to a random sample of PubMed indexed articles on blinded randomized clinical trials published in 2010. We initially sampled 300 publications of which 24 reported on risk of unblinding. RESULTS: Of the 276 contacted trial authors, 129 (47%) responded. Assessment of the risk of unblinding was conducted in 56 trials (43%), often based on a pretrial evaluation involving a group of healthy assessors trying to identify differences between experimental and control interventions. When we included informal assessments of the risk of unblinding, the number of trials assessing the risk of unblinding increased to 75 (58%). Suspected or overt unblinding occurred in 14 trials (11%), mostly based on perceptible differences between experimental and control interventions. CONCLUSION: Approximately 4 of 10 trials assessed risk of unblinding without reporting such assessments in the trial publication, and approximately 1 in 10 trials identified cases of overt or suspected unblinding, also without reporting them. Unblinding is not an exceptional event in randomized clinical trials; it occurs regularly but is rarely reported.

Subjective and objective outcomes in randomized clinical trials: definitions differed in methods publications and were often absent from trial reports.

OBJECTIVES: The degree of bias in randomized clinical trials varies depending on whether the outcome is subjective or objective. Assessment of the risk of bias in a clinical trial will therefore often involve categorization of the type of outcome. Our primary aim was to examine how the concepts "subjective outcome" and "objective outcome" are defined in methodological publications and clinical trial reports. To put this examination into perspective, we also provide an overview of how outcomes are classified more broadly. STUDY DESIGN AND SETTING: A systematic review of methodological publications providing a classification of clinical trial outcomes and a descriptive study of how outcomes were classified in 200 PubMed indexed clinical trial reports published in 2012. RESULTS: We identified 90 methodological publications with some form of a classification of outcomes. Three distinct definitions were provided for subjective outcome: (1) dependent on assessor judgment, (2) patient-reported outcome, or (3) private phenomena (ie, phenomena only assessable by the patient). Of the 200 clinical trial reports, 12 used the term "subjective" and/or "objective" about outcomes, but no clinical trial reports explicitly defined the terms. CONCLUSION: The terms "subjective" and "objective" are ambiguous when used to describe outcomes in randomized clinical trials. We suggest that the terms should be defined explicitly when used in connection with the assessment of risk of bias in a clinical trial, in metaepidemiological research, and generally in the reporting of clinical trials. We also suggest that adding an explicit clarification of the terms in future versions of the Cochrane Handbook might further strengthen its important role in guiding review authors.

The risk of unblinding was infrequently and incompletely reported in 300 randomized clinical trial publications.

OBJECTIVES: To assess the proportion of clinical trials explicitly reporting the risk of unblinding, to evaluate the completeness of reporting on unblinding risk, and to describe the reported procedures involved in assessing unblinding. STUDY DESIGN AND SETTING: We sampled at random 300 blinded randomized clinical trials indexed in PubMed in 2010. Two authors read the trial publications and extracted data independently. RESULTS: Twenty-four trial publications, or 8% (95% confidence interval [CI], 5, 12%), explicitly reported the risk of unblinding, of which 16 publications, or 5% (95% CI, 3, 8%), reported compromised blinding; and 8 publications, or 3% (95% CI, 1, 5%), intact blinding. The reporting on risk of unblinding in the 24 trial publications was generally incomplete. The median proportion of assessments per trial affected by unblinding was 3% (range 1-30%). The most common mechanism for unblinding was perceptible physical properties of the treatments, for example, a difference in the taste and odor of a typhoid vaccine compared with its placebo. CONCLUSION: Published articles on randomized clinical trials infrequently reported risk of unblinding. This may reflect a tendency for avoiding reporting actual or suspected unblinding or a genuine low risk of unblinding.

Gender differences in the association between westernization and metabolic risk among Greenland Inuit.

BACKGROUND: The Inuit have gone through an accelerated process of modernization especially since 1950. Primarily because of the dietary transition, westernisation is expected to influence the Inuit's metabolic risk in a negative way with respect to cardiovascular risk. The aim was to analyze metabolic risk factors among Inuit in Greenland and Denmark and their relation to westernization. METHODS: 1173 adult Inuit participated in a health survey in Greenland and Denmark. The examination included a 75 g OGTT. BMI, waist-to-hip ratio, and blood pressure were measured. P-glucose, s-insulin, lipids and urine-albumin/creatinine ratio were analysed. Westernization was estimated by place of residence and language. RESULTS: The prevalence of the metabolic syndrome was 20.3% among men and 19.5% among women (p = 0.73). The association between the metabolic syndrome and westernization was different for men and women. For men there was an increase in prevalence of the metabolic syndrome with westernization within Greenland, but the variation was less pronounced than the difference between the migrants and the Inuit in Greenland. Age, family history of diabetes, and non-smoking were directly associated with the metabolic syndrome, whereas high physical activity was negatively associated with the metabolic syndrome. For women there was a significant negative association between westernization and the metabolic syndrome among the three population groups in Greenland, whereas the prevalence was not significantly lower among female migrants compared with Inuit women in Greenland. Age, family history of diabetes, non-smoking, and low education were associated with the metabolic syndrome. CONCLUSIONS: The effect of westernization on metabolic risk was different for men and women. For men physical inactivity due to a decrease in subsistence hunting and fishing seems to increase the metabolic risk; for women higher education is associated with a more favorable risk profile.

Diabetes among Inuit migrants in Denmark.

OBJECTIVES: The study aimed to estimate the prevalence of diabetes and impaired glucose intolerance (IGT) among Inuit migrants living in Denmark, and to compare with findings from Greenland. Further, we analyzed determinants for diabetes and impaired glucose metabolism. STUDY DESIGN: Cross-sectional, population-based epidemiological study. METHODS: This cross-sectional study included randomly selected Inuit migrants in Denmark aged 34 years and above. Diabetes and IGT were diagnosed using the oral glucose tolerance test. Body mass index (BMI) and waist circumference were measured, and blood samples were taken from each subject. Socio-demographic characteristics were investigated using a questionnaire. For comparison, data from the Greenland Population Study were used (n = 917). RESULTS: Of 506 eligible subjects, 256 (51%) participated. Twenty-six subjects had diabetes (10.2%) and twenty-eight had IGT (10.9%). Of those with diabetes, 64% had not been previously diagnosed. The prevalences of diabetes and IGT were not significantly different from those among Inuit in Greenland. Significant predictors of diabetes and impaired glucose metabolism (IGM) were found to be age, waist circumference and physical inactivity. The association between waist circumference and diabetes was significantly stronger among Inuit migrants in Denmark than among Inuit in Greenland. CONCLUSIONS: The prevalence of diabetes is high among the Inuit migrants in Denmark. However, unlike that reported in most studies, the prevalence was not significantly higher in the migrant population compared with the population of origin.