Comparison of Patient Health Questionnaire-9, Edinburgh Postnatal Depression Scale and Hospital Anxiety and Depression - Depression subscale scores by administration mode: An individual participant data differential item functioning meta-analysis.

Administration mode of patient-reported outcome measures (PROMs) may influence responses. We assessed if Patient Health Questionnaire-9 (PHQ-9), Edinburgh Postnatal Depression Scale (EPDS) and Hospital Anxiety and Depression Scale - Depression subscale (HADS-D) item responses and scores were associated with administration mode. We compared (1) self-administration versus interview-administration; within self-administration (2) research or medical setting versus private; and (3) pen-and-paper versus electronic; and within interview-administration (4) in-person versus phone. We analysed individual participant data meta-analysis datasets with item-level data for the PHQ-9 (N = 34,529), EPDS (N = 16,813), and HADS-D (N = 16,768). We used multiple indicator multiple cause models to assess differential item functioning (DIF) by administration mode. We found statistically significant DIF for most items on all measures due to large samples, but influence on total scores was negligible. In 10 comparisons conducted across the PHQ-9, EPDS, and HADS-D, Pearson's correlations and intraclass correlation coefficients between latent depression symptom scores from models that did or did not account for DIF were between 0.995 and 1.000. Total PHQ-9, EPDS, and HADS-D scores did not differ materially across administration modes. Researcher and clinicians who evaluate depression symptoms with these questionnaires can select administration methods based on patient preferences, feasibility, or cost.

Analyses of academician cohorts generate biased pandemic excess death estimates.

OBJECTIVE: Death data from cohorts of academicians have been used to estimate pandemic excess deaths. We aimed to evaluate the validity of this approach. STUDY DESIGN AND SETTING: Data were analyzed from living and deceased member lists from Mainland China, UK and Greece academies; and Nobel laureates (and US subset thereof). Samples of early elected academicians were probed for unrecorded deaths; datasets overtly missing deaths were excluded from further analyses. Actuarial risks were compared against the general population in the same country in respective age strata. Relative incidence risk increases in death in active pandemic periods were compared to population-wide pandemic excess death estimates for the same country. RESULTS: Royal Society and Academy of Athens datasets overtly missed deaths. Pre-pandemic death rates were 4-12-fold lower in the Chinese Academy of Engineering (CAE) versus respective age strata of the Mainland China population. A +158% relative increase in death risk was seen in CAE data during the first 12-months of wide viral spread. Both increases (+34% in British Academy) and decreases (-27% in US Nobel laureates) in death rates occurred in pandemic (2020-22) versus pre-pandemic (2017-2019) years; point estimates were far from known excess deaths in the respective countries (+6% and +14%, respectively). Published excess death estimates for urban-dwelling Mainland China selectively analyzed CAE that had double the pandemic death rates than another Chinese academy (Chinese Academy of Sciences). CONCLUSION: Missingness, lack of representativeness, large uncertainty, and selective analysis reporting make data from academy rosters unreliable for estimating general population excess deaths.

A novel framework to assess haematology and oncology registration trials: The THEOREMM project.

BACKGROUND: Methodological limitations affect a significant number of oncology and haematology trials, raising concerns about the applicability of their results. For example, a suboptimal control arm or limited access to best care upon progression may skew the trial results toward a benefit in the experimental arm. Beyond the fact that such limitations do not prevent drugs reaching the market, other assessment tools, such as those developed by professional societies-ESMO-MCBS and ASCO Value Framework-do not integrate these important shortcomings. METHODS: We propose creating a novel framework with the scope of assessing registration cancer clinical trials in haematology and oncology (randomized or single arm)-that is trials leading to a marketing authorization. The main steps of the methods are (1) assembling a scientific board; (2) defining the scope, goal and methods through pre-specified, pre-registered and protocolized methodology; (3) preregistration of the protocol; (4) conducting a scoping review of limitations and biases affecting oncology trials and assessing existing scores or methods; (5) developing a list of features to be included and assessed within the framework; (6) assessing each feature through a questionnaire sent to highly cited haematologists and oncologists involved in clinical trials; and (7) finalizing the first version of framework. RESULTS: Not applicable. CONCLUSIONS: Our proposal emerged in response to the lack of consideration for key limitations in current trial assessments. The goal is to create a framework specifically designed to assess single trials leading to marketing authorization in the field of oncology and haematogy.

Individual Patient Data Meta-Analysis Estimates the Minimal Detectable Change of the Geriatric Depression Scale-15.

OBJECTIVE: To use individual participant data meta-analysis (IPDMA) to estimate the minimal detectable change (MDC) of the Geriatric Depression Scale-15 (GDS-15) and to examine whether MDC may differ based on participant characteristics and study-level variables. STUDY DESIGN AND SETTING: This was a secondary analysis of data from an IPDMA on the depression screening accuracy of the GDS. Datasets from studies published in any language were eligible for the present study if they included GDS-15 scores for participants aged 60 or older. MDC of the GDS-15 was estimated via random-effects meta-analysis using 2.77 (MDC95) and 1.41 (MDC67) standard errors of measurement (SEM). Subgroup analyses were used to evaluate differences in MDC by participant age and sex. Meta-regression was conducted to assess for differences based on study-level variables, including mean age, proportion male, proportion with major depression, and recruitment setting. RESULTS: 5,876 participants (mean age 76 years, 40% male, 11% with major depression) from 21 studies were included. The MDC95 was 3.81 points (95% confidence interval [CI] 3.59, 4.04), and MDC67 was 1.95 (95% CI 1.83, 2.03). The difference in MDC95 was 0.26 points (95% CI 0.04, 0.48) between ≥ 80-year-olds and < 80-year-olds; MDC95 was similar for females and males (0.05, 95% CI -0.12, 0.22). The MDC95 increased by 0.29 points (95% CI 0.17, 0.41) per 10% increase in proportion of participants with major depression; mean age had a small association (0.04 points, 95% CI 0.00 to 0.09) with MDC95, but sex and recruitment setting were not significantly associated. CONCLUSIONS: The MDC95 was 3.81 points and MDC67 was 1.95 points. MDC95 increased with the proportion of participants with major depression. Results can be used to evaluate individual changes in depression symptoms and as a threshold for assessing minimal clinical important difference estimates.

Assessment of transparency indicators in space medicine.

Space medicine is a vital discipline with often time-intensive and costly projects and constrained opportunities for studying various elements such as space missions, astronauts, and simulated environments. Moreover, private interests gain increasing influence in this discipline. In scientific disciplines with these features, transparent and rigorous methods are essential. Here, we undertook an evaluation of transparency indicators in publications within the field of space medicine. A meta-epidemiological assessment of PubMed Central Open Access (PMC OA) eligible articles within the field of space medicine was performed for prevalence of code sharing, data sharing, pre-registration, conflicts of interest, and funding. Text mining was performed with the rtransparent text mining algorithms with manual validation of 200 random articles to obtain corrected estimates. Across 1215 included articles, 39 (3%) shared code, 258 (21%) shared data, 10 (1%) were registered, 110 (90%) contained a conflict-of-interest statement, and 1141 (93%) included a funding statement. After manual validation, the corrected estimates for code sharing, data sharing, and registration were 5%, 27%, and 1%, respectively. Data sharing was 32% when limited to original articles and highest in space/parabolic flights (46%). Overall, across space medicine we observed modest rates of data sharing, rare sharing of code and almost non-existent protocol registration. Enhancing transparency in space medicine research is imperative for safeguarding its scientific rigor and reproducibility.

Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review.

This scoping review of randomised controlled trials on artificial intelligence (AI) in clinical practice reveals an expanding interest in AI across clinical specialties and locations. The USA and China are leading in the number of trials, with a focus on deep learning systems for medical imaging, particularly in gastroenterology and radiology. A majority of trials (70 [81%] of 86) report positive primary endpoints, primarily related to diagnostic yield or performance; however, the predominance of single-centre trials, little demographic reporting, and varying reports of operational efficiency raise concerns about the generalisability and practicality of these results. Despite the promising outcomes, considering the likelihood of publication bias and the need for more comprehensive research including multicentre trials, diverse outcome measures, and improved reporting standards is crucial. Future AI trials should prioritise patient-relevant outcomes to fully understand AI's true effects and limitations in health care.

Consolidated guidance for behavioral intervention pilot and feasibility studies.

BACKGROUND: In the behavioral sciences, conducting pilot and/or feasibility studies (PFS) is a key step that provides essential information used to inform the design, conduct, and implementation of a larger-scale trial. There are more than 160 published guidelines, reporting checklists, frameworks, and recommendations related to PFS. All of these publications offer some form of guidance on PFS, but many focus on one or a few topics. This makes it difficult for researchers wanting to gain a broader understanding of all the relevant and important aspects of PFS and requires them to seek out multiple sources of information, which increases the risk of missing key considerations to incorporate into their PFS. The purpose of this study was to develop a consolidated set of considerations for the design, conduct, implementation, and reporting of PFS for interventions conducted in the behavioral sciences. METHODS: To develop this consolidation, we undertook a review of the published guidance on PFS in combination with expert consensus (via a Delphi study) from the authors who wrote such guidance to inform the identified considerations. A total of 161 PFS-related guidelines, checklists, frameworks, and recommendations were identified via a review of recently published behavioral intervention PFS and backward/forward citation tracking of a well-known PFS literature (e.g., CONSORT Ext. for PFS). Authors of all 161 PFS publications were invited to complete a three-round Delphi survey, which was used to guide the creation of a consolidated list of considerations to guide the design, conduct, and reporting of PFS conducted by researchers in the behavioral sciences. RESULTS: A total of 496 authors were invited to take part in the three-round Delphi survey (round 1, N = 46; round 2, N = 24; round 3, N = 22). A set of twenty considerations, broadly categorized into six themes (intervention design, study design, conduct of trial, implementation of intervention, statistical analysis, and reporting) were generated from a review of the 161 PFS-related publications as well as a synthesis of feedback from the three-round Delphi process. These 20 considerations are presented alongside a supporting narrative for each consideration as well as a crosswalk of all 161 publications aligned with each consideration for further reading. CONCLUSION: We leveraged expert opinion from researchers who have published PFS-related guidelines, checklists, frameworks, and recommendations on a wide range of topics and distilled this knowledge into a valuable and universal resource for researchers conducting PFS. Researchers may use these considerations alongside the previously published literature to guide decisions about all aspects of PFS, with the hope of creating and disseminating interventions with broad public health impact.

Best practices for data management and sharing in experimental biomedical research.

Effective data management is crucial for scientific integrity and reproducibility, a cornerstone of scientific progress. Well-organized and well-documented data enable validation and building on results. Data management encompasses activities including organization, documentation, storage, sharing, and preservation. Robust data management establishes credibility, fostering trust within the scientific community and benefiting researchers' careers. In experimental biomedicine, comprehensive data management is vital due to the typically intricate protocols, extensive metadata, and large datasets. Low-throughput experiments, in particular, require careful management to address variations and errors in protocols and raw data quality. Transparent and accountable research practices rely on accurate documentation of procedures, data collection, and analysis methods. Proper data management ensures long-term preservation and accessibility of valuable datasets. Well-managed data can be revisited, contributing to cumulative knowledge and potential new discoveries. Publicly funded research has an added responsibility for transparency, resource allocation, and avoiding redundancy. Meeting funding agency expectations increasingly requires rigorous methodologies, adherence to standards, comprehensive documentation, and widespread sharing of data, code, and other auxiliary resources. This review provides critical insights into raw and processed data, metadata, high-throughput versus low-throughput datasets, a common language for documentation, experimental and reporting guidelines, efficient data management systems, sharing practices, and relevant repositories. We systematically present available resources and optimal practices for wide use by experimental biomedical researchers.

Is society caught up in a Death Spiral? Modeling societal demise and its reversal.

Just like an army of ants caught in an ant mill, individuals, groups and even whole societies are sometimes caught up in a Death Spiral, a vicious cycle of self-reinforcing dysfunctional behavior characterized by continuous flawed decision making, myopic single-minded focus on one (set of) solution(s), denial, distrust, micromanagement, dogmatic thinking and learned helplessness. We propose the term Death Spiral Effect to describe this difficult-to-break downward spiral of societal decline. Specifically, in the current theory-building review we aim to: (a) more clearly define and describe the Death Spiral Effect; (b) model the downward spiral of societal decline as well as an upward spiral; (c) describe how and why individuals, groups and even society at large might be caught up in a Death Spiral; and (d) offer a positive way forward in terms of evidence-based solutions to escape the Death Spiral Effect. Management theory hints on the occurrence of this phenomenon and offers turn-around leadership as solution. On a societal level strengthening of democracy may be important. Prior research indicates that historically, two key factors trigger this type of societal decline: rising inequalities creating an upper layer of elites and a lower layer of masses; and dwindling (access to) resources. Historical key markers of societal decline are a steep increase in inequalities, government overreach, over-integration (interdependencies in networks) and a rapidly decreasing trust in institutions and resulting collapse of legitimacy. Important issues that we aim to shed light on are the behavioral underpinnings of decline, as well as the question if and how societal decline can be reversed. We explore the extension of these theories from the company/organization level to the society level, and make use of insights from both micro-, meso-, and macro-level theories (e.g., Complex Adaptive Systems and collapsology, the study of the risks of collapse of industrial civilization) to explain this process of societal demise. Our review furthermore draws on theories such as Social Safety Theory, Conservation of Resources Theory, and management theories that describe the decline and fall of groups, companies and societies, as well as offer ways to reverse this trend.

The Subjective Interpretation of the Medical Evidence.

This Viewpoint discusses whether commonly occuring, sometimes divergent interpretations of new evidence's validity and usefulness should be explained when experts and guideline committees provide recommendations for screening or treatment.

Are the Risk of Generalizability Biases Generalizable? A Meta-Epidemiological Study.

Background: Preliminary studies (e.g., pilot/feasibility studies) can result in misleading evidence that an intervention is ready to be evaluated in a large-scale trial when it is not. Risk of Generalizability Biases (RGBs, a set of external validity biases) represent study features that influence estimates of effectiveness, often inflating estimates in preliminary studies which are not replicated in larger-scale trials. While RGBs have been empirically established in interventions targeting obesity, the extent to which RGBs generalize to other health areas is unknown. Understanding the relevance of RGBs across health behavior intervention research can inform organized efforts to reduce their prevalence. Purpose: The purpose of our study was to examine whether RGBs generalize outside of obesity-related interventions. Methods: A systematic review identified health behavior interventions across four behaviors unrelated to obesity that follow a similar intervention development framework of preliminary studies informing larger-scale trials (i.e., tobacco use disorder, alcohol use disorder, interpersonal violence, and behaviors related to increased sexually transmitted infections). To be included, published interventions had to be tested in a preliminary study followed by testing in a larger trial (the two studies thus comprising a study pair). We extracted health-related outcomes and coded the presence/absence of RGBs. We used meta-regression models to estimate the impact of RGBs on the change in standardized mean difference (ΔSMD) between the preliminary study and larger trial. Results: We identified sixty-nine study pairs, of which forty-seven were eligible for inclusion in the analysis (k = 156 effects), with RGBs identified for each behavior. For pairs where the RGB was present in the preliminary study but removed in the larger trial the treatment effect decreased by an average of ΔSMD=-0.38 (range - 0.69 to -0.21). This provides evidence of larger drop in effectiveness for studies containing RGBs relative to study pairs with no RGBs present (treatment effect decreased by an average of ΔSMD =-0.24, range - 0.19 to -0.27). Conclusion: RGBs may be associated with higher effect estimates across diverse areas of health intervention research. These findings suggest commonalities shared across health behavior intervention fields may facilitate introduction of RGBs within preliminary studies, rather than RGBs being isolated to a single health behavior field.

Smoking Cessation, or How to Avert Half a Billion Premature Deaths - Now.

An estimated 1.1 billion people currently smoke cigarettes,1 and 50 to 70% likely will die from tobacco-related causes.2 This translates to 550 to 770 million expected tobacco deaths among those who currently smoke. Many additional deaths will accrue in successive generations if the status quo continues. Of interest is the reversibility of the excess mortality risk of smoking. The meta-analysis by Cho et al.3 of four large national cohorts of nearly 1.5 million adults followed on average 14.8 years yielded 23.0 million person-years of observational data with over 120,000 deaths identified through linked death registries.

Impact of trial attrition rates on treatment effect estimates in chronic inflammatory diseases: A meta-epidemiological study.

The objective of this meta-epidemiological study was to explore the impact of attrition rates on treatment effect estimates in randomised trials of chronic inflammatory diseases (CID) treated with biological and targeted synthetic disease-modifying drugs. We sampled trials from Cochrane reviews. Attrition rates and primary endpoint results were retrieved from trial publications; Odds ratios (ORs) were calculated from the odds of withdrawing in the experimental intervention compared to the control comparison groups (i.e., differential attrition), as well as the odds of achieving a clinical response (i.e., the trial outcome). Trials were combined using random effects restricted maximum likelihood meta-regression models and associations between estimates of treatment effects and attrition rates were analysed. From 37 meta-analyses, 179 trials were included, and 163 were analysed (301 randomised comparisons; n = 62,220 patients). Overall, the odds of withdrawal were lower in the experimental compared to control groups (random effects summary OR = 0.45, 95% CI, 0.41-0.50). The corresponding overall treatment effects were large (random effects summary OR = 4.43, 95% CI 3.92-4.99) with considerable heterogeneity across interventions and clinical specialties (I2 = 85.7%). The ORs estimating treatment effect showed larger treatment benefits when the differential attrition was more prominent with more attrition in the control group (OR = 0.73, 95% CI 0.55-0.96). Higher attrition rates from the control arm are associated with larger estimated benefits of treatments with biological or targeted synthetic disease-modifying drugs in CID trials; differential attrition may affect estimates of treatment benefit in randomised trials.

An empirical comparison of statistical methods for multiple cut-off diagnostic test accuracy meta-analysis of the Edinburgh postnatal depression scale (EPDS) depression screening tool using published results vs individual participant data.

BACKGROUND: Selective reporting of results from only well-performing cut-offs leads to biased estimates of accuracy in primary studies of questionnaire-based screening tools and in meta-analyses that synthesize results. Individual participant data meta-analysis (IPDMA) of sensitivity and specificity at each cut-off via bivariate random-effects models (BREMs) can overcome this problem. However, IPDMA is laborious and depends on the ability to successfully obtain primary datasets, and BREMs ignore the correlation between cut-offs within primary studies. METHODS: We compared the performance of three recent multiple cut-off models developed by Steinhauser et al., Jones et al., and Hoyer and Kuss, that account for missing cut-offs when meta-analyzing diagnostic accuracy studies with multiple cut-offs, to BREMs fitted at each cut-off. We used data from 22 studies of the accuracy of the Edinburgh Postnatal Depression Scale (EPDS; 4475 participants, 758 major depression cases). We fitted each of the three multiple cut-off models and BREMs to a dataset with results from only published cut-offs from each study (published data) and an IPD dataset with results for all cut-offs (full IPD data). We estimated pooled sensitivity and specificity with 95% confidence intervals (CIs) for each cut-off and the area under the curve. RESULTS: Compared to the BREMs fitted to the full IPD data, the Steinhauser et al., Jones et al., and Hoyer and Kuss models fitted to the published data produced similar receiver operating characteristic curves; though, the Hoyer and Kuss model had lower area under the curve, mainly due to estimating slightly lower sensitivity at lower cut-offs. When fitting the three multiple cut-off models to the full IPD data, a similar pattern of results was observed. Importantly, all models had similar 95% CIs for sensitivity and specificity, and the CI width increased with cut-off levels for sensitivity and decreased with an increasing cut-off for specificity, even the BREMs which treat each cut-off separately. CONCLUSIONS: Multiple cut-off models appear to be the favorable methods when only published data are available. While collecting IPD is expensive and time consuming, IPD can facilitate subgroup analyses that cannot be conducted with published data only.

Large language models for science and medicine.

Large language models (LLMs) are a type of machine learning model that learn statistical patterns over text, such as predicting the next words in a sequence of text. Both general purpose and task-specific LLMs have demonstrated potential across diverse applications. Science and medicine have many data types that are highly suitable for LLMs, such as scientific texts (publications, patents and textbooks), electronic medical records, large databases of DNA and protein sequences and chemical compounds. Carefully validated systems that can understand and reason across all these modalities may maximize benefits. Despite the inevitable limitations and caveats of any new technology and some uncertainties specific to LLMs, LLMs have the potential to be transformative in science and medicine.

Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics.

Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine.

Estimating the extent of selective reporting: An application to economics.

Using a sample of 70,399 published p-values from 192 meta-analyses, we empirically estimate the counterfactual distribution of p-values in the absence of any biases. Comparing observed p-values with counterfactually expected p-values allows us to estimate how many p-values are published as being statistically significant when they should have been published as non-significant. We estimate the extent of selectively reported p-values to range between 57.7% and 71.9% of the significant p-values. The counterfactual p-value distribution also allows us to assess shifts of p-values along the entire distribution of published p-values, revealing that particularly very small p-values (p < 0.001) are unexpectedly abundant in the published literature. Subsample analysis suggests that the extent of selective reporting is reduced in research fields that use experimental designs, analyze microeconomics research questions, and have at least some adequately powered studies.

Personalized and longitudinal electronic informed consent in clinical trials: How to move the needle?

Changes in the clinical trials landscape have been driven by advancements in digital technology. The use of electronic informed consent to inform research participants and to obtain their consent electronically has the potential to improve participant-researcher interactions over time, facilitate clinical trial participation, and increase efficiency in clinical trial conduct. A personalized electronic informed consent platform that enables long-term interactions with the research team could function as a tool to empower participant engagement in clinical trials. However, significant challenges persist impeding successful and widespread implementation. This Perspective provides insights into the opportunities and challenges for the implementation of electronic informed consent in clinical trials. It sets out key recommendations to promote the implementation of this innovative approach to the informed consent process, including the creation of uniform electronic informed consent platforms at regional and national level.

Methods proposed for monitoring the implementation of evidence-based research: a cross-sectional study.

OBJECTIVES: Evidence-based research (EBR) is the systematic and transparent use of prior research to inform a new study so that it answers questions that matter in a valid, efficient, and accessible manner. This study surveyed experts about existing (e.g., citation analysis) and new methods for monitoring EBR and collected ideas about implementing these methods. STUDY DESIGN AND SETTING: We conducted a cross-sectional study via an online survey between November 2022 and March 2023. Participants were experts from the fields of evidence synthesis and research methodology in health research. Open-ended questions were coded by recurring themes; descriptive statistics were used for quantitative questions. RESULTS: Twenty-eight expert participants suggested that citation analysis should be supplemented with content evaluation (not just what is cited but also in which context), content expert involvement, and assessment of the quality of cited systematic reviews. They also suggested that citation analysis could be facilitated with automation tools. They emphasized that EBR monitoring should be conducted by ethics committees and funding bodies before the research starts. Challenges identified for EBR implementation monitoring were resource constraints and clarity on responsibility for EBR monitoring. CONCLUSION: Ideas proposed in this study for monitoring the implementation of EBR can be used to refine methods and define responsibility but should be further explored in terms of feasibility and acceptability. Different methods may be needed to determine if the use of EBR is improving over time.