A survey of experts to identify methods to detect problematic studies: Stage 1 of the INSPECT-SR Project.

Background: Randomised controlled trials (RCTs) inform healthcare decisions. Unfortunately, some published RCTs contain false data, and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesise all RCTs which have been conducted on a given topic. This means that any of these 'problematic studies' are likely to be included, but there are no agreed methods for identifying them. The INSPECT-SR project is developing a tool to identify problematic RCTs in systematic reviews of healthcare-related interventions. The tool will guide the user through a series of 'checks' to determine a study's authenticity. The first objective in the development process is to assemble a comprehensive list of checks to consider for inclusion. Methods: We assembled an initial list of checks for assessing the authenticity of research studies, with no restriction to RCTs, and categorised these into five domains: Inspecting results in the paper; Inspecting the research team; Inspecting conduct, governance, and transparency; Inspecting text and publication details; Inspecting the individual participant data. We implemented this list as an online survey, and invited people with expertise and experience of assessing potentially problematic studies to participate through professional networks and online forums. Participants were invited to provide feedback on the checks on the list, and were asked to describe any additional checks they knew of, which were not featured in the list. Results: Extensive feedback on an initial list of 102 checks was provided by 71 participants based in 16 countries across five continents. Fourteen new checks were proposed across the five domains, and suggestions were made to reword checks on the initial list. An updated list of checks was constructed, comprising 116 checks. Many participants expressed a lack of familiarity with statistical checks, and emphasized the importance of feasibility of the tool. Conclusions: A comprehensive list of trustworthiness checks has been produced. The checks will be evaluated to determine which should be included in the INSPECT-SR tool.

Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions.

INTRODUCTION: Randomised controlled trials (RCTs) inform healthcare decisions. It is now apparent that some published RCTs contain false data and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesise all RCTs that have been conducted on a given topic. While it is usual to assess methodological features of the RCTs in the process of undertaking a systematic review, it is not usual to consider whether the RCTs contain false data. Studies containing false data therefore go unnoticed and contribute to systematic review conclusions. The INveStigating ProblEmatic Clinical Trials in Systematic Reviews (INSPECT-SR) project will develop a tool to assess the trustworthiness of RCTs in systematic reviews of healthcare-related interventions. METHODS AND ANALYSIS: The INSPECT-SR tool will be developed using expert consensus in combination with empirical evidence, over five stages: (1) a survey of experts to assemble a comprehensive list of checks for detecting problematic RCTs, (2) an evaluation of the feasibility and impact of applying the checks to systematic reviews, (3) a Delphi survey to determine which of the checks are supported by expert consensus, culminating in, (4) a consensus meeting to select checks to be included in a draft tool and to determine its format and (5) prospective testing of the draft tool in the production of new health systematic reviews, to allow refinement based on user feedback. We anticipate that the INSPECT-SR tool will help researchers to identify problematic studies and will help patients by protecting them from the influence of false data on their healthcare. ETHICS AND DISSEMINATION: The University of Manchester ethics decision tool was used, and this returned the result that ethical approval was not required for this project (30 September 2022), which incorporates secondary research and surveys of professionals about subjects relating to their expertise. Informed consent will be obtained from all survey participants. All results will be published as open-access articles. The final tool will be made freely available.

Qualitative and Quantitative Evaluation of an Innovative Primary and Secondary Diabetes Clinic in Western Sydney.

Introduction: Western Sydney Diabetes (WSD) established an innovative diabetes service in May 2020, using virtual and in-person care, linking primary care with the diabetes specialist team. This study evaluated the service's feasibility using qualitative and quantitative methods. Method: Evaluation included: 1) thematic analysis of interviews and workshops with patients and health professionals (n = 28); 2) quantitative analysis of records of patients admitted July 2020-June 2021 (n = 110). Results: Key themes related to 1) benefits: convenient location, access to integrated care, advantages of virtual care; 2) challenges: hard for patients to ask questions, technology issues; 3) confidence: shared care decision making, multidisciplinary team; and 4) future directions: additional multidisciplinary services, expanded insulin stabilisation service, promotion.Improvements between baseline and 3 months included 1.3% reduction in HbA1c (p < 0.05). Sulfonylurea dropped by 25% between initial appointment and follow-up, and GLP1RA/SGLT2i use increasing by 30% (p < 0.05). The clinic covered costs using Medicare billings and Nationally Weighted Activity Units. Discussion: The findings suggest this integrated care model was feasible and perceived as beneficial by both patients and providers. The clinic offers a promising model of practice that could be developed further to roll out in other regions for rural delivery of care.

Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions.

Introduction: Randomised controlled trials (RCTs) inform healthcare decisions. It is now apparent that some published RCTs contain false data and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesise all RCTs that have been conducted on a given topic. While it is usual to assess methodological features of the RCTs in the process of undertaking a systematic review, it is not usual to consider whether the RCTs contain false data. Studies containing false data therefore go unnoticed and contribute to systematic review conclusions. The INSPECT-SR project will develop a tool to assess the trustworthiness of RCTs in systematic reviews of healthcare related interventions. Methods and analysis: The INSPECT-SR tool will be developed using expert consensus in combination with empirical evidence, over five stages: 1) a survey of experts to assemble a comprehensive list of checks for detecting problematic RCTs, 2) an evaluation of the feasibility and impact of applying the checks to systematic reviews, 3) a Delphi survey to determine which of the checks are supported by expert consensus, culminating in 4) a consensus meeting to select checks to be included in a draft tool and to determine its format, 5) prospective testing of the draft tool in the production of new health systematic reviews, to allow refinement based on user feedback. We anticipate that the INSPECT-SR tool will help researchers to identify problematic studies, and will help patients by protecting them from the influence of false data on their healthcare.

Diabetic Retinopathy Screening at the Point of Care (DR SPOC): detecting undiagnosed and vision-threatening retinopathy by integrating portable technologies within existing services.

INTRODUCTION: The aim of this study was to determine the prevalence of diabetic retinopathy (DR) in a low socioeconomic region of a high-income country, as well as determine the diagnostic utility of point-of-care screening for high-risk populations in tertiary care settings. RESEARCH DESIGN AND METHODS: This was a cross-sectional study of patients with diabetes attending foot ulcer or integrated care diabetes clinics at two Western Sydney hospitals (n=273). DR was assessed using portable, two-field, non-mydriatic fundus photography and combined electroretinogram/ pupillometry (ERG). With mydriatic photographs used as the reference standard, sensitivity and specificity of the devices were determined. Prevalence of DR and vision-threatening diabetic retinopathy (VTDR) were reported, with multivariate logistic regression used to identify predictors of DR. RESULTS: Among 273 patients, 39.6% had any DR, while 15.8% had VTDR, of whom 59.3% and 62.8% were previously undiagnosed, respectively. Non-mydriatic photography demonstrated 20.2% sensitivity and 99.5% specificity for any DR, with a 56.7% screening failure rate. Meanwhile, mydriatic photography produced high-quality images with a 7.6% failure rate. ERG demonstrated 72.5% sensitivity and 70.1% specificity, with a 15.0% failure rate. The RETeval ERG was noted to have an optimal DR cut-off score at 22. Multivariate logistic regression identified an eGFR of ≤29 mL/min/1.73 m2, HbA1c of ≥7.0%, pupil size of <4 mm diameter, diabetes duration of 5-24 years and RETeval score of ≥22 as strong predictors of DR. CONCLUSION: There is a high prevalence of vision-threatening and undiagnosed DR among patients attending high-risk tertiary clinics in Western Sydney. Point-of-care DR screening using portable, mydriatic photography demonstrates potential as a model of care which is easily accessible, targeted for high-risk populations and substantially enhances DR detection.

Raising concerns on questionable ethics approvals - a case study of 456 trials from the Institut Hospitalo-Universitaire Méditerranée Infection.

BACKGROUND: The practice of clinical research is strictly regulated by law. During submission and review processes, compliance of such research with the laws enforced in the country where it was conducted is not always correctly filled in by the authors or verified by the editors. Here, we report a case of a single institution for which one may find hundreds of publications with seemingly relevant ethical concerns, along with 10 months of follow-up through contacts with the editors of these articles. We thus argue for a stricter control of ethical authorization by scientific editors and we call on publishers to cooperate to this end. METHODS: We present an investigation of the ethics and legal aspects of 456 studies published by the IHU-MI (Institut Hospitalo-Universitaire Méditerranée Infection) in Marseille, France. RESULTS: We identified a wide range of issues with the stated research authorization and ethics of the published studies with respect to the Institutional Review Board and the approval presented. Among the studies investigated, 248 were conducted with the same ethics approval number, even though the subjects, samples, and countries of investigation were different. Thirty-nine (39) did not even contain a reference to the ethics approval number while they present research on human beings. We thus contacted the journals that published these articles and provide their responses to our concerns. It should be noted that, since our investigation and reporting to journals, PLOS has issued expressions of concerns for several publications we analyze here. CONCLUSION: This case presents an investigation of the veracity of ethical approval, and more than 10 months of follow-up by independent researchers. We call for stricter control and cooperation in handling of these cases, including editorial requirement to upload ethical approval documents, guidelines from COPE to address such ethical concerns, and transparent editorial policies and timelines to answer such concerns. All supplementary materials are available.

Dynamics of SARS-CoV-2 seroassay sensitivity: a systematic review and modelling study.

BackgroundSerological surveys have been the gold standard to estimate numbers of SARS-CoV-2 infections, the dynamics of the epidemic, and disease severity. Serological assays have decaying sensitivity with time that can bias their results, but there is a lack of guidelines to account for this phenomenon for SARS-CoV-2.AimOur goal was to assess the sensitivity decay of seroassays for detecting SARS-CoV-2 infections, the dependence of this decay on assay characteristics, and to provide a simple method to correct for this phenomenon.MethodsWe performed a systematic review and meta-analysis of SARS-CoV-2 serology studies. We included studies testing previously diagnosed, unvaccinated individuals, and excluded studies of cohorts highly unrepresentative of the general population (e.g. hospitalised patients).ResultsOf the 488 screened studies, 76 studies reporting on 50 different seroassays were included in the analysis. Sensitivity decay depended strongly on the antigen and the analytic technique used by the assay, with average sensitivities ranging between 26% and 98% at 6 months after infection, depending on assay characteristics. We found that a third of the included assays departed considerably from manufacturer specifications after 6 months.ConclusionsSeroassay sensitivity decay depends on assay characteristics, and for some types of assays, it can make manufacturer specifications highly unreliable. We provide a tool to correct for this phenomenon and to assess the risk of decay for a given assay. Our analysis can guide the design and interpretation of serosurveys for SARS-CoV-2 and other pathogens and quantify systematic biases in the existing serology literature.

Diabetes during the COVID-19 pandemic: are people getting access to the right level of care?

INTRODUCTION: Avoidance of health services, in particular hospital and community services, is problematic for people with diabetes. Evidence has demonstrated that such missed attendances are associated with worse health, faster declines in functioning, and higher rates of mortality long-term. This paper investigated the impact of the pandemic on healthcare access across community and hospital care, including Virtual Care (VC) using several large datasets of General Practice (GP) and hospital services in western Sydney. METHODS: A retrospective cohort study using a time-series database of 173,805 HbA1c tests done at Blacktown and Mt Druitt hospitals and 1.8 million recorded consultations at GP clinics in the region was undertaken. RESULTS: The average rate of diabetes in Emergency Department fell from 17.8% pre-pandemic to 11% after January 2020 (p < 0.001). This rate varied substantially over time, and correlated well with large outbreaks of COVID-19 in the state. Conversely, attendances of people with diabetes to GP clinics, especially using VC services, increased substantially over the pandemic period. DISCUSSION/CONCLUSION: During the pandemic there was a substantial avoidance of hospital care by patients with diabetes. However, this may have been replaced by VC offered in the community for those with less severe diseases.

Evaluating the Effectiveness of Joint Specialist Case Conferences in Improving Diabetes Control in Patients With Schizophrenia on Clozapine.

ABSTRACT: Clozapine treatment for schizophrenia is typically long-term and is associated with a high rate of diabetes. Mental health and diabetes specialist teams at a local hospital in Australia have undertaken a series of joint specialist case conferences (JSCCs) where the diabetes team works with the psychiatry team to improve diabetes management. In this retrospective cohort study conducted between 2013 and 2018, we found that glycemic control in clozapine clinics linked with JSCCs was improved significantly compared with that in the non-JSCC clinics. In the non-JSCC clozapine clinics (control), the poor glycemic control rates stayed at a similar level: 23% in 2013 and 24% in 2018. In contrast, whereas the control patients' poor glycemic rate in JSCC clozapine clinics in 2013 was 24%, it decreased markedly in 2018 to 13%. This study indicates that JSCCs can improve diabetes outcomes in a group of patients with severe mental illness.

Causal and Associational Language in Observational Health Research: A Systematic Evaluation.

We estimated the degree to which language used in the high-profile medical/public health/epidemiology literature implied causality using language linking exposures to outcomes and action recommendations; examined disconnects between language and recommendations; identified the most common linking phrases; and estimated how strongly linking phrases imply causality. We searched for and screened 1,170 articles from 18 high-profile journals (65 per journal) published from 2010-2019. Based on written framing and systematic guidance, 3 reviewers rated the degree of causality implied in abstracts and full text for exposure/outcome linking language and action recommendations. Reviewers rated the causal implication of exposure/outcome linking language as none (no causal implication) in 13.8%, weak in 34.2%, moderate in 33.2%, and strong in 18.7% of abstracts. The implied causality of action recommendations was higher than the implied causality of linking sentences for 44.5% or commensurate for 40.3% of articles. The most common linking word in abstracts was "associate" (45.7%). Reviewers' ratings of linking word roots were highly heterogeneous; over half of reviewers rated "association" as having at least some causal implication. This research undercuts the assumption that avoiding "causal" words leads to clarity of interpretation in medical research.

Plausibility of Claimed Covid-19 Vaccine Efficacies by Age: A Simulation Study.

BACKGROUND: Multiple vaccines against Covid-19 have passed through phase-3 trials; however, concerns have been raised about alleged excessive similarity of efficacy across age groups for the Sputnik V vaccine. STUDY QUESTION: How likely are the observed efficacies for all age subgroups to fall within the range of by-age efficacies claimed for the AstraZeneca, Janssen, Moderna, Pfizer, and Sputnik V vaccines, assuming that there is no effect of age on efficacy? STUDY DESIGN: We performed a simulation study using R of 1000 and then 50,000 simulated trials for each vaccine, with random allocation to each arm but fixed enrollment numbers by age group. We used study-wide efficacy and infection rate for all age groups. We recorded the observed vaccine efficacies in each age group and summated how many simulations had all observed efficacies fall within the range of efficacies described in the relevant article. RESULTS: In the 1000-trial simulation for the AstraZeneca vaccine, in 23.8% of simulated trials, the observed efficacies of all age subgroups fell within the efficacy bounds for age subgroups in the published article. The J + J simulation showed 44.7%, Moderna 51.1%, Pfizer 30.5%, and 0.0% of the Sputnik simulated trials had all age subgroups fall within the limits of the efficacy estimates described by the published article. In 50,000 simulated trials of the Sputnik vaccine, 0.026% had all age subgroups fall within the limits of the efficacy estimates described by the published article, whereas 99.974% did not. CONCLUSIONS: The distribution of alleged vaccine efficacies of the Sputnik vaccine by age in the phase-III trial is very unlikely to occur in genuine experimental data, even if the number of patients recruited, vaccine efficacy, and overall infection rate are true and there is no underlying difference in vaccine efficacy by age.

COVID-19 Vaccine Misperceptions in a Community Sample of Adults Aged 18-49 Years in Australia.

Central to a successful population vaccination program is high uptake of vaccines. However, COVID-19 vaccine uptake may be impeded by beliefs based on misinformation. We sought to understand the prevalence and nature of misbeliefs about COVID-19 vaccines, and identify associated factors, shortly after commencement of Australia's national vaccine rollout. A cross-sectional survey was administered to unvaccinated young adults (n = 2050) in Australia aged 18-49 years (mean age 33 years), 13 July-21 August 2021. This sample was previously under-represented in COVID-19 research but shown to have less willingness to vaccinate. Two thirds of participants agreed with at least one misbelief item. Misperceptions about COVID-19 vaccines were found to be significantly associated with lower health literacy, less knowledge about vaccines, lower perceived personal risk of COVID-19, greater endorsement of conspiracy beliefs, and lower confidence and trust in government and scientific institutions. Misbeliefs were more common in participants with less educational attainment, in younger age groups, and in males, as per previous research. Understanding determinants and barriers to vaccination uptake, such as knowledge and beliefs based on misinformation, can help to shape effective public health communication and inform debunking efforts at this critical time and in the future.

Assessing the burden of COVID-19 in developing countries: systematic review, meta-analysis and public policy implications.

INTRODUCTION: The infection fatality rate (IFR) of COVID-19 has been carefully measured and analysed in high-income countries, whereas there has been no systematic analysis of age-specific seroprevalence or IFR for developing countries. METHODS: We systematically reviewed the literature to identify all COVID-19 serology studies in developing countries that were conducted using representative samples collected by February 2021. For each of the antibody assays used in these serology studies, we identified data on assay characteristics, including the extent of seroreversion over time. We analysed the serology data using a Bayesian model that incorporates conventional sampling uncertainty as well as uncertainties about assay sensitivity and specificity. We then calculated IFRs using individual case reports or aggregated public health updates, including age-specific estimates whenever feasible. RESULTS: In most locations in developing countries, seroprevalence among older adults was similar to that of younger age cohorts, underscoring the limited capacity that these nations have to protect older age groups.Age-specific IFRs were roughly 2 times higher than in high-income countries. The median value of the population IFR was about 0.5%, similar to that of high-income countries, because disparities in healthcare access were roughly offset by differences in population age structure. CONCLUSION: The burden of COVID-19 is far higher in developing countries than in high-income countries, reflecting a combination of elevated transmission to middle-aged and older adults as well as limited access to adequate healthcare. These results underscore the critical need to ensure medical equity to populations in developing countries through provision of vaccine doses and effective medications.

Making Statistical Sense of the Molnupiravir MOVe-OUT Clinical Trial.

Oral therapies for the early treatment of COVID-19 may prevent disease progression and health system overcrowding. A new oral therapeutic named molnupiravir has been promoted as providing an approximately 50% reduction in death or the need for hospitalization. The clinical trial evaluating this drug was stopped early at the recommendation of the Data Safety and Monitoring Board after approximately 50% of the sample had been recruited. At the point of discontinuing the trial, approximately 90% of the planned sample had been recruited and had available follow-up data accessible. We discuss issues about the study conduct, analysis, and interpretation, including 1) the authors and sponsors presented the interim analysis as the primary analysis; 2) communication between sponsors and the Data Safety and Monitoring Board was insufficient; 3) the treatment effects reverse when examining only the post-interim analysis population, and are substantially attenuated when examining the full data; 4) the choice of primary analysis is incorrect; 5) analysis of lost-to-follow-up patients favors the study drug; and 6) other known molnupiravir trials were not presented in the primary study findings. As a result of methodological and statistical concerns, it seems that external trials, separate from those supported by the sponsoring company, are required to determine the utility of this drug.

Correction of scientific literature: Too little, too late!

The Coronavirus Disease 2019 (COVID-19) pandemic has highlighted the limitations of the current scientific publication system, in which serious post-publication concerns are often addressed too slowly to be effective. In this Perspective, we offer suggestions to improve academia's willingness and ability to correct errors in an appropriate time frame.