Using Bayesian statistics in confirmatory clinical trials in the regulatory setting: a tutorial review.

Bayesian statistics plays a pivotal role in advancing medical science by enabling healthcare companies, regulators, and stakeholders to assess the safety and efficacy of new treatments, interventions, and medical procedures. The Bayesian framework offers a unique advantage over the classical framework, especially when incorporating prior information into a new trial with quality external data, such as historical data or another source of co-data. In recent years, there has been a significant increase in regulatory submissions using Bayesian statistics due to its flexibility and ability to provide valuable insights for decision-making, addressing the modern complexity of clinical trials where frequentist trials are inadequate. For regulatory submissions, companies often need to consider the frequentist operating characteristics of the Bayesian analysis strategy, regardless of the design complexity. In particular, the focus is on the frequentist type I error rate and power for all realistic alternatives. This tutorial review aims to provide a comprehensive overview of the use of Bayesian statistics in sample size determination, control of type I error rate, multiplicity adjustments, external data borrowing, etc., in the regulatory environment of clinical trials. Fundamental concepts of Bayesian sample size determination and illustrative examples are provided to serve as a valuable resource for researchers, clinicians, and statisticians seeking to develop more complex and innovative designs.

The reporting quality and spin of randomized controlled trials of endometriosis pain: Methodological study based on CONSORT extension on abstracts.

OBJECTIVE: To assess the reporting quality of published RCT abstracts regarding patients with endometriosis pelvic pain and investigate the prevalence and characteristics of spin in these abstracts. METHODS: PubMed and Scopus were searched for RCT abstracts addressing endometriosis pelvic pain published from January 1st, 2010 to December 1st, 2023.The reporting quality of RCT abstracts was assessed using the CONSORT statement for abstracts. Additionally, spin was evaluated in the results and conclusions section of the abstracts, defined as the misleading reporting of study findings to emphasize the perceived benefits of an intervention or to confound readers from statistically non-significant results. Assessing factors affecting the reporting quality and spin existence, linear and logistic regression was used, respectively. RESULTS: A total of 47 RCT abstracts were included. Out of 16 checklist items, only three items including objective, intervention and conclusions were sufficiently reported in the most abstracts (more than 95%), and none of the abstracts presented precise data as required by the CONSORT-A guidelines. In the reporting quality of material and method section, trial design, type of randomization, the generation of random allocation sequences, the allocation concealment and blinding were most items identified that were suboptimal. The total score for the quality varied between 5 and 15 (mean: 9.59, SD: 3.03, median: 9, IQR: 5). Word count (beta = 0.015, p-value = 0.005) and publishing in open-accessed journals (beta = 2.023, p-value = 0.023) were the significant factors that affecting the reporting quality. Evaluating spin within each included paper, we found that 18 (51.43%) papers had statistically non-significant results. From these studies, 12 (66.66%) had spin in both results and conclusion sections. Furthermore, the spin intensity increased during 2010-2023 and 38.29% of abstracts had spin in both results and conclusion sections. CONCLUSION: Overall poor adherence to CONSORT-A was observed, with spin detected in several RCTs featuring non-significant primary endpoints in obstetrics and gynecology literature.

Best Practices and Recommendations for Research Using Virtual Real-Time Data Collection: Protocol for Virtual Data Collection Studies.

BACKGROUND: The COVID-19 pandemic and the subsequent need for social distancing required the immediate pivoting of research modalities. Research that had previously been conducted in person had to pivot to remote data collection. Researchers had to develop data collection protocols that could be conducted remotely with limited or no evidence to guide the process. Therefore, the use of web-based platforms to conduct real-time research visits surged despite the lack of evidence backing these novel approaches. OBJECTIVE: This paper aims to review the remote or virtual research protocols that have been used in the past 10 years, gather existing best practices, and propose recommendations for continuing to use virtual real-time methods when appropriate. METHODS: Articles (n=22) published from 2013 to June 2023 were reviewed and analyzed to understand how researchers conducted virtual research that implemented real-time protocols. "Real-time" was defined as data collection with a participant through a live medium where a participant and research staff could talk to each other back and forth in the moment. We excluded studies for the following reasons: (1) studies that collected participant or patient measures for the sole purpose of engaging in a clinical encounter; (2) studies that solely conducted qualitative interview data collection; (3) studies that conducted virtual data collection such as surveys or self-report measures that had no interaction with research staff; (4) studies that described research interventions but did not involve the collection of data through a web-based platform; (5) studies that were reviews or not original research; (6) studies that described research protocols and did not include actual data collection; and (7) studies that did not collect data in real time, focused on telehealth or telemedicine, and were exclusively intended for medical and not research purposes. RESULTS: Findings from studies conducted both before and during the COVID-19 pandemic suggest that many types of data can be collected virtually in real time. Results and best practice recommendations from the current protocol review will be used in the design and implementation of a substudy to provide more evidence for virtual real-time data collection over the next year. CONCLUSIONS: Our findings suggest that virtual real-time visits are doable across a range of participant populations and can answer a range of research questions. Recommended best practices for virtual real-time data collection include (1) providing adequate equipment for real-time data collection, (2) creating protocols and materials for research staff to facilitate or guide participants through data collection, (3) piloting data collection, (4) iteratively accepting feedback, and (5) providing instructions in multiple forms. The implementation of these best practices and recommendations for future research are further discussed in the paper. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/53790.

Retrospective and prospective study designs.

The labels "retrospective" and "prospective" strongly connote study quality, often favouring prospective studies. However, three definitions of these terms exist, each suggesting distinct methodological limitations. In this review, we summarize and evaluate these definitions. Caution is warranted when labeling a study "retrospective": This label should only be used when implying a risk of recall bias, which can only occur in retrospective data collection. Generally, assessing random and systematic errors is necessary to appraise study quality rather than relying on ambiguous labels.

Considerations for using potential surrogate endpoints in cancer screening trials.

The requirement of large-scale expensive cancer screening trials spanning decades creates considerable barriers to the development, commercialisation, and implementation of novel screening tests. One way to address these problems is to use surrogate endpoints for the ultimate endpoint of interest, cancer mortality, at an earlier timepoint. This Review aims to highlight the issues underlying the choice and use of surrogate endpoints for cancer screening trials, to propose criteria for when and how we might use such endpoints, and to suggest possible candidates. We present the current landscape and challenges, and discuss lessons and shortcomings from the therapeutic trial setting. It is hugely challenging to validate a surrogate endpoint, even with carefully designed clinical studies. Nevertheless, we consider whether there are candidates that might satisfy the requirements defined by research and regulatory bodies.

Using Qualitative Questionnaires in Medical Education Research.

Most students in Health Profession Education courses are new to the world of qualitative research. Faced with the challenge of designing a research project, they are often drawn towards using the questionnaire as a data collection method, commonly assuming that utilising open-ended questions alone constitutes qualitative research design. Designing questionnaires that meet the standards of rigour is challenging, and this common assumption reflects inexperience with and misunderstandings of qualitative ontology, as well as the lack of methodological literature on designing and developing qualitative questionnaires. This paper is written with research supervisors as well as students in mind, as it is aimed to help elucidate the decision-making process and the justification for using a qualitative questionnaire. Drawing upon examples of research conducted by our students, and the wider literature, we demonstrate how qualitative questionnaires can produce rich and meaningful findings when they (1) prioritise qualitative research values, and (2) follow a rigorous design process when the questionnaire is developed. We conclude by offering a simple framework for developing rigorous qualitative questionnaires to those who may consider using this approach.

A practical guide to EEG hyperscanning in joint action research: from motivation to implementation.

Developments in cognitive neuroscience have led to the emergence of hyperscanning, the simultaneous measurement of brain activity from multiple people. Hyperscanning is useful for investigating social cognition, including joint action, because of its ability to capture neural processes that occur within and between people as they coordinate actions toward a shared goal. Here, we provide a practical guide for researchers considering using hyperscanning to study joint action and seeking to avoid frequently raised concerns from hyperscanning skeptics. We focus specifically on Electroencephalography (EEG) hyperscanning, which is widely available and optimally suited for capturing fine-grained temporal dynamics of action coordination. Our guidelines cover questions that are likely to arise when planning a hyperscanning project, ranging from whether hyperscanning is appropriate for answering one's research questions to considerations for study design, dependent variable selection, data analysis and visualization. By following clear guidelines that facilitate careful consideration of the theoretical implications of research design choices and other methodological decisions, joint action researchers can mitigate interpretability issues and maximize the benefits of hyperscanning paradigms.

How to Use and Report on p-values.

The use of the p-value in quantitative research, particularly its threshold of "P < 0.05" for determining "statistical significance," has long been a cornerstone of statistical analysis in research. However, this standard has been increasingly scrutinized for its potential to mislead findings, especially when the practical significance, the number of comparisons, or the suitability of statistical tests are not properly considered. In response to controversy around use of p-values, the American Statistical Association published a statement in 2016 that challenged the research community to abandon the term "statistically significant". This stance has been echoed by leading scientific journals to urge a significant reduction or complete elimination in the reliance on p-values when reporting results. To provide guidance to researchers in health professions education, this paper provides a succinct overview of the ongoing debate regarding the use of p-values and the definition of p-values. It reflects on the controversy by highlighting the common pitfalls associated with p-value interpretation and usage, such as misinterpretation, overemphasis, and false dichotomization between "significant" and "non-significant" results. This paper also outlines specific recommendations for the effective use of p-values in statistical reporting including the importance of reporting effect sizes, confidence intervals, the null hypothesis, and conducting sensitivity analyses for appropriate interpretation. These considerations aim to guide researchers toward a more nuanced and informative use of p-values.

REporting quality of PilOt randomised controlled trials in surgery (REPORTS): a methodological survey protocol.

INTRODUCTION: The aim of this methodological review is to evaluate the completeness of reporting of surgical pilot and feasibility randomised trials as per the Consolidated Standards of Reporting Trials (CONSORT) extension to randomised pilot and feasibility trials. Moreover, we aim to assess for the presence of spin reporting and inconsistency between abstract and main text reporting in surgical pilot and feasibility randomised trials. METHODS AND ANALYSIS: A comprehensive, electronic search strategy will be used to identify studies indexed in Medline, Embase and Cochrane Central Register of Controlled Trials (CENTRAL) databases. Studies will be included if they are pilot or feasibility randomised trials of surgical interventions. The primary outcome will be overall CONSORT statement extension to randomised pilot and feasibility trials checklist completeness. This will be defined as trials reporting each of the 40 items in the CONSORT statement extension to randomised pilot and feasibility trials checklist. Secondary outcomes will include the reporting of individual studies as per the CONSORT extension to randomised pilot and feasibility trials, the use of spin reporting strategies, trial factors associated with reporting quality and spin strategy use, and consistency between abstract and main text reporting. Poisson and logistic regressions will be performed to explore the association between trial factors and completeness of reporting as measured by the number of reported CONSORT items. ETHICS AND DISSEMINATION: This is a methodological survey that has been registered a priori on the International Prospective Register for Systematic Reviews (PROSPERO) (CRD42023475512). Local ethics approval is not required. We plan to disseminate study results through peer-reviewed publication and conference presentations.

The Conceptualization and Measurement of Research Impact in Primary Health Care: Protocol for a Rapid Scoping Review.

BACKGROUND: The generation of research evidence and knowledge in primary health care (PHC) is crucial for informing the development and implementation of interventions and innovations and driving health policy, health service improvements, and potential societal changes. PHC research has broad effects on patients, practices, services, population health, community, and policy formulation. The in-depth exploration of the definition and measures of research impact within PHC is essential for broadening our understanding of research impact in the discipline and how it compares to other health services research. OBJECTIVE: The objectives of the study are (1) to understand the conceptualizations and measures of research impact within the realm of PHC and (2) to identify methodological frameworks for evaluation and research impact and the benefits and challenges of using these approaches. The forthcoming review seeks to guide future research endeavors and enhance methodologies used in assessing research impact within PHC. METHODS: The protocol outlines the rapid review and environmental scan approach that will be used to explore research impact in PHC and will be guided by established frameworks such as the Canadian Academy of Health Sciences Impact Framework and the Canadian Health Services and Policy Research Alliance. The rapid review follows scoping review guidelines (PRISMA-ScR; Preferred Reporting Items for Systematic Review and Meta-Analysis Extension for Scoping Reviews). The environmental scan will be done by consulting with professional organizations, academic institutions, information science, and PHC experts. The search strategy will involve multiple databases, citation and forward citation searching, and manual searches of gray literature databases, think tank websites, and relevant catalogs. We will include gray and scientific literature focusing explicitly on research impact in PHC from high-income countries using the World Bank classification. Publications published in English from 1978 will be considered. The collected papers will undergo a 2-stage independent review process based on predetermined inclusion criteria. The research team will extract data from selected studies based on the research questions and the CRISP (Consensus Reporting Items for Studies in Primary Care) protocol statement. The team will discuss the extracted data, enabling the identification and categorization of key themes regarding research impact conceptualization and measurement in PHC. The narrative synthesis will evolve iteratively based on the identified literature. RESULTS: The results of this study are expected at the end of 2024. CONCLUSIONS: The forthcoming review will explore the conceptualization and measurement of research impact in PHC. The synthesis will offer crucial insights that will guide subsequent research, emphasizing the need for a standardized approach that incorporates diverse perspectives to comprehensively gauge the true impact of PHC research. Furthermore, trends and gaps in current methodologies will set the stage for future studies aimed at enhancing our understanding and measurement of research impact in PHC. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/55860.

Accuracy of Event Rate and Effect Size Estimation in Major Cardiovascular Trials: A Systematic Review.

Importance: For the design of a randomized clinical trial (RCT), estimation of the expected event rate and effect size of an intervention is needed to calculate the sample size. Overestimation may lead to an underpowered trial. Objective: To evaluate the accuracy of published estimates of event rate and effect size in contemporary cardiovascular RCTs. Evidence Review: A systematic search was conducted in MEDLINE for multicenter cardiovascular RCTs associated with MeSH (Medical Subject Headings) terms for cardiovascular diseases published in the New England Journal of Medicine, JAMA, or the Lancet between January 1, 2010, and December 31, 2019. Identified trials underwent abstract review; eligible trials then underwent full review, and those with insufficiently reported data were excluded. Data were extracted from the original publication or the study protocol, and a random-effects model was used for data pooling. This review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guideline. The primary outcome was the accuracy of event rate and effect size estimation. Accuracy was determined by comparing the observed event rate in the control group and the effect size with their hypothesized values. Linear regression was used to determine the association between estimation accuracy and trial characteristics. Findings: Of the 873 RCTs identified, 374 underwent full review and 30 were subsequently excluded, resulting in 344 trials for analysis. The median observed event rate was 9.0% (IQR, 4.3% to 21.4%), which was significantly lower than the estimated event rate of 11.0% (IQR, 6.0% to 25.0%) with a median deviation of -12.3% (95% CI, -16.4% to -5.6%; P < .001). More than half of the trials (196 [61.1%]) overestimated the expected event rate. Accuracy of event rate estimation was associated with a higher likelihood of refuting the null hypothesis (0.13 [95% CI, 0.01 to 0.25]; P = .03). The median observed effect size in superiority trials was 0.91 (IQR, 0.74 to 0.99), which was significantly lower than the estimated effect size of 0.72 (IQR, 0.60 to 0.80), indicating a median overestimation of 23.1% (95% CI, 17.9% to 28.3%). A total of 216 trials (82.1%) overestimated the effect size. Conclusions and Relevance: In this systematic review of contemporary cardiovascular RCTs, event rates of the primary end point and effect sizes of an intervention were frequently overestimated. This overestimation may have contributed to the inability to adequately test the trial hypothesis.

Artificial intelligence and illusions of understanding in scientific research.

Scientists are enthusiastically imagining ways in which artificial intelligence (AI) tools might improve research. Why are AI tools so attractive and what are the risks of implementing them across the research pipeline? Here we develop a taxonomy of scientists' visions for AI, observing that their appeal comes from promises to improve productivity and objectivity by overcoming human shortcomings. But proposed AI solutions can also exploit our cognitive limitations, making us vulnerable to illusions of understanding in which we believe we understand more about the world than we actually do. Such illusions obscure the scientific community's ability to see the formation of scientific monocultures, in which some types of methods, questions and viewpoints come to dominate alternative approaches, making science less innovative and more vulnerable to errors. The proliferation of AI tools in science risks introducing a phase of scientific enquiry in which we produce more but understand less. By analysing the appeal of these tools, we provide a framework for advancing discussions of responsible knowledge production in the age of AI.

Accelerating Medicines Partnership® Schizophrenia (AMP® SCZ): Rationale and Study Design of the Largest Global Prospective Cohort Study of Clinical High Risk for Psychosis.

This article describes the rationale, aims, and methodology of the Accelerating Medicines Partnership® Schizophrenia (AMP® SCZ). This is the largest international collaboration to date that will develop algorithms to predict trajectories and outcomes of individuals at clinical high risk (CHR) for psychosis and to advance the development and use of novel pharmacological interventions for CHR individuals. We present a description of the participating research networks and the data processing analysis and coordination center, their processes for data harmonization across 43 sites from 13 participating countries (recruitment across North America, Australia, Europe, Asia, and South America), data flow and quality assessment processes, data analyses, and the transfer of data to the National Institute of Mental Health (NIMH) Data Archive (NDA) for use by the research community. In an expected sample of approximately 2000 CHR individuals and 640 matched healthy controls, AMP SCZ will collect clinical, environmental, and cognitive data along with multimodal biomarkers, including neuroimaging, electrophysiology, fluid biospecimens, speech and facial expression samples, novel measures derived from digital health technologies including smartphone-based daily surveys, and passive sensing as well as actigraphy. The study will investigate a range of clinical outcomes over a 2-year period, including transition to psychosis, remission or persistence of CHR status, attenuated positive symptoms, persistent negative symptoms, mood and anxiety symptoms, and psychosocial functioning. The global reach of AMP SCZ and its harmonized innovative methods promise to catalyze the development of new treatments to address critical unmet clinical and public health needs in CHR individuals.

Practical Guidelines for Standardised Resolution of Important Protocol Deviations in Clinical Trials Conducted in Sub-Saharan Africa.

A clinical trial is any research on human subjects that involves an investigational medicinal product or device. Investigational medicinal products include unlicensed drugs or drugs used outside the product license (e.g. for a new indication) (ICH-GCP). As per the internationally accepted ICH-GCP guidelines, clinical trials should be conducted strictly per the approved protocol. However, during the lifecycle of a trial, protocol deviations may occur. Under ICH efficacy guidelines, protocol deviations are divided into non-important (minor) or important (major), and the latter can jeopardise the participant's rights, safety or the quality of data generated by the study. Existing guidelines on protocol deviation management do not detail or standardise actions to be taken for participants, investigational products, data or samples as part of a holistic management of important protocol deviations. Herein, we propose guidelines to address the current literature gap and promote the standardisation of actions to address important protocol deviations in clinical trials. The advised actions should complement the existing local institutional review board and national regulatory authority requirements.