What do Australians affected by cancer think about oncology researchers sharing research data? A cross-sectional survey.

AIM: Previous research has shown patients and the public in Australia generally support medical researchers in making de-identified research data available to other scientists. However, this research has focussed on certain types of data and recipients. We surveyed Australians affected by cancer to characterize their attitudes toward the sharing of research data with multiple third parties, including the public. METHODS: A short, anonymous online survey of Australians with a previous diagnosis of cancer was advertised between October 27, 2022, and February 27, 2023. Quantitative responses were analyzed with descriptive statistics. Free-text responses were coded deductively and summarised using content analysis. RESULTS: In total, 551 respondents contributed data to the survey. There was strong support for cancer researchers sharing non-human and de-identified human research data with clinicians (90% and 95%, respectively) and non-profit researchers (both 94%). However, fewer participants supported sharing data with for-profit researchers (both 64%) or publicly (both 61%). When asked if they would hypothetically consent to researchers at their treatment location using and sharing their de-identified data publicly, only half agreed. In contrast, after being shown a visual representation of the de-identified survey data, 80% of respondents supported sharing it publicly. CONCLUSION: Australians affected by cancer support the sharing of research data, particularly with clinicians and non-profit researchers. Our results also imply that visualization of the data to be shared may enhance support for making it publicly available. These results should help alleviate any concerns about research participants' attitudes toward data sharing, as well as boost researchers' motivation for sharing.

Cancer researchers' experiences with and perceptions of research data sharing: Results of a cross-sectional survey.

BACKGROUND: Despite wide recognition of the benefits of sharing research data, public availability rates have not increased substantially in oncology or medicine more broadly over the last decade. METHODS: We surveyed 285 cancer researchers to determine their prior experience with sharing data and views on known drivers and inhibitors. RESULTS: We found that 45% of respondents had shared some data from their most recent empirical publication, with respondents who typically studied non-human research participants, or routinely worked with human genomic data, more likely to share than those who did not. A third of respondents added that they had previously shared data privately, with 74% indicating that doing so had also led to authorship opportunities or future collaborations for them. Journal and funder policies were reported to be the biggest general drivers toward sharing, whereas commercial interests, agreements with industrial sponsors and institutional policies were the biggest prohibitors. We show that researchers' decisions about whether to share data are also likely to be influenced by participants' desires. CONCLUSIONS: Our survey suggests that increased promotion and support by research institutions, alongside greater championing of data sharing by journals and funders, may motivate more researchers in oncology to share their data.

Prevalence and predictors of data and code sharing in the medical and health sciences: systematic review with meta-analysis of individual participant data.

OBJECTIVES: To synthesise research investigating data and code sharing in medicine and health to establish an accurate representation of the prevalence of sharing, how this frequency has changed over time, and what factors influence availability. DESIGN: Systematic review with meta-analysis of individual participant data. DATA SOURCES: Ovid Medline, Ovid Embase, and the preprint servers medRxiv, bioRxiv, and MetaArXiv were searched from inception to 1 July 2021. Forward citation searches were also performed on 30 August 2022. REVIEW METHODS: Meta-research studies that investigated data or code sharing across a sample of scientific articles presenting original medical and health research were identified. Two authors screened records, assessed the risk of bias, and extracted summary data from study reports when individual participant data could not be retrieved. Key outcomes of interest were the prevalence of statements that declared that data or code were publicly or privately available (declared availability) and the success rates of retrieving these products (actual availability). The associations between data and code availability and several factors (eg, journal policy, type of data, trial design, and human participants) were also examined. A two stage approach to meta-analysis of individual participant data was performed, with proportions and risk ratios pooled with the Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis. RESULTS: The review included 105 meta-research studies examining 2 121 580 articles across 31 specialties. Eligible studies examined a median of 195 primary articles (interquartile range 113-475), with a median publication year of 2015 (interquartile range 2012-2018). Only eight studies (8%) were classified as having a low risk of bias. Meta-analyses showed a prevalence of declared and actual public data availability of 8% (95% confidence interval 5% to 11%) and 2% (1% to 3%), respectively, between 2016 and 2021. For public code sharing, both the prevalence of declared and actual availability were estimated to be <0.5% since 2016. Meta-regressions indicated that only declared public data sharing prevalence estimates have increased over time. Compliance with mandatory data sharing policies ranged from 0% to 100% across journals and varied by type of data. In contrast, success in privately obtaining data and code from authors historically ranged between 0% and 37% and 0% and 23%, respectively. CONCLUSIONS: The review found that public code sharing was persistently low across medical research. Declarations of data sharing were also low, increasing over time, but did not always correspond to actual sharing of data. The effectiveness of mandatory data sharing policies varied substantially by journal and type of data, a finding that might be informative for policy makers when designing policies and allocating resources to audit compliance. SYSTEMATIC REVIEW REGISTRATION: Open Science Framework doi:10.17605/OSF.IO/7SX8U.

Predicting and reasoning about replicability using structured groups.

This paper explores judgements about the replicability of social and behavioural sciences research and what drives those judgements. Using a mixed methods approach, it draws on qualitative and quantitative data elicited from groups using a structured approach called the IDEA protocol ('investigate', 'discuss', 'estimate' and 'aggregate'). Five groups of five people with relevant domain expertise evaluated 25 research claims that were subject to at least one replication study. Participants assessed the probability that each of the 25 research claims would replicate (i.e. that a replication study would find a statistically significant result in the same direction as the original study) and described the reasoning behind those judgements. We quantitatively analysed possible correlates of predictive accuracy, including self-rated expertise and updating of judgements after feedback and discussion. We qualitatively analysed the reasoning data to explore the cues, heuristics and patterns of reasoning used by participants. Participants achieved 84% classification accuracy in predicting replicability. Those who engaged in a greater breadth of reasoning provided more accurate replicability judgements. Some reasons were more commonly invoked by more accurate participants, such as 'effect size' and 'reputation' (e.g. of the field of research). There was also some evidence of a relationship between statistical literacy and accuracy.

Predicting reliability through structured expert elicitation with the repliCATS (Collaborative Assessments for Trustworthy Science) process.

As replications of individual studies are resource intensive, techniques for predicting the replicability are required. We introduce the repliCATS (Collaborative Assessments for Trustworthy Science) process, a new method for eliciting expert predictions about the replicability of research. This process is a structured expert elicitation approach based on a modified Delphi technique applied to the evaluation of research claims in social and behavioural sciences. The utility of processes to predict replicability is their capacity to test scientific claims without the costs of full replication. Experimental data supports the validity of this process, with a validation study producing a classification accuracy of 84% and an Area Under the Curve of 0.94, meeting or exceeding the accuracy of other techniques used to predict replicability. The repliCATS process provides other benefits. It is highly scalable, able to be deployed for both rapid assessment of small numbers of claims, and assessment of high volumes of claims over an extended period through an online elicitation platform, having been used to assess 3000 research claims over an 18 month period. It is available to be implemented in a range of ways and we describe one such implementation. An important advantage of the repliCATS process is that it collects qualitative data that has the potential to provide insight in understanding the limits of generalizability of scientific claims. The primary limitation of the repliCATS process is its reliance on human-derived predictions with consequent costs in terms of participant fatigue although careful design can minimise these costs. The repliCATS process has potential applications in alternative peer review and in the allocation of effort for replication studies.

Changing patterns in reporting and sharing of review data in systematic reviews with meta-analysis of the effects of interventions: cross sectional meta-research study.

OBJECTIVES: To examine changes in completeness of reporting and frequency of sharing data, analytical code, and other review materials in systematic reviews over time; and factors associated with these changes. DESIGN: Cross sectional meta-research study. POPULATION: Random sample of 300 systematic reviews with meta-analysis of aggregate data on the effects of a health, social, behavioural, or educational intervention. Reviews were indexed in PubMed, Science Citation Index, Social Sciences Citation Index, Scopus, and Education Collection in November 2020. MAIN OUTCOME MEASURES: The extent of complete reporting and the frequency of sharing review materials in the systematic reviews indexed in 2020 were compared with 110 systematic reviews indexed in February 2014. Associations between completeness of reporting and various factors (eg, self-reported use of reporting guidelines, journal policies on data sharing) were examined by calculating risk ratios and 95% confidence intervals. RESULTS: Several items were reported suboptimally among 300 systematic reviews from 2020, such as a registration record for the review (n=113; 38%), a full search strategy for at least one database (n=214; 71%), methods used to assess risk of bias (n=185; 62%), methods used to prepare data for meta-analysis (n=101; 34%), and source of funding for the review (n=215; 72%). Only a few items not already reported at a high frequency in 2014 were reported more frequently in 2020. No evidence indicated that reviews using a reporting guideline were more completely reported than reviews not using a guideline. Reviews published in 2020 in journals that mandated either data sharing or inclusion of data availability statements were more likely to share their review materials (eg, data, code files) than reviews in journals without such mandates (16/87 (18%) v 4/213 (2%)). CONCLUSION: Incomplete reporting of several recommended items for systematic reviews persists, even in reviews that claim to have followed a reporting guideline. Journal policies on data sharing might encourage sharing of review materials.

How often do cancer researchers make their data and code available and what factors are associated with sharing?

BACKGROUND: Various stakeholders are calling for increased availability of data and code from cancer research. However, it is unclear how commonly these products are shared, and what factors are associated with sharing. Our objective was to evaluate how frequently oncology researchers make data and code available and explore factors associated with sharing. METHODS: A cross-sectional analysis of a random sample of 306 cancer-related articles indexed in PubMed in 2019 which studied research subjects with a cancer diagnosis was performed. All articles were independently screened for eligibility by two authors. Outcomes of interest included the prevalence of affirmative sharing declarations and the rate with which declarations connected to data complying with key FAIR principles (e.g. posted to a recognised repository, assigned an identifier, data license outlined, non-proprietary formatting). We also investigated associations between sharing rates and several journal characteristics (e.g. sharing policies, publication models), study characteristics (e.g. cancer rarity, study design), open science practices (e.g. pre-registration, pre-printing) and subsequent citation rates between 2020 and 2021. RESULTS: One in five studies declared data were publicly available (59/306, 19%, 95% CI: 15-24%). However, when data availability was investigated this percentage dropped to 16% (49/306, 95% CI: 12-20%), and then to less than 1% (1/306, 95% CI: 0-2%) when data were checked for compliance with key FAIR principles. While only 4% of articles that used inferential statistics reported code to be available (10/274, 95% CI: 2-6%), the odds of reporting code to be available were 5.6 times higher for researchers who shared data. Compliance with mandatory data and code sharing policies was observed in 48% (14/29) and 0% (0/6) of articles, respectively. However, 88% of articles (45/51) included data availability statements when required. Policies that encouraged data sharing did not appear to be any more effective than not having a policy at all. The only factors associated with higher rates of data sharing were studying rare cancers and using publicly available data to complement original research. CONCLUSIONS: Data and code sharing in oncology occurs infrequently, and at a lower rate than would be expected given the prevalence of mandatory sharing policies. There is also a large gap between those declaring data to be available, and those archiving data in a way that facilitates its reuse. We encourage journals to actively check compliance with sharing policies, and researchers consult community-accepted guidelines when archiving the products of their research.

Rehabilitation for Women Undergoing Breast Cancer Surgery: A Systematic Review and Meta-Analysis of the Effectiveness of Early, Unrestricted Exercise Programs on Upper Limb Function.

Upper limb impairments are common in women following surgery for breast cancer. Range of movement (ROM) exercises are commonly prescribed, but the optimal timing to begin these exercises is not clear. The aim of this systematic review and meta-analysis was to evaluate the effectiveness of early ROM exercises (Early ROM) compared to delayed ROM exercises (Delayed ROM) or usual care (UC) in reducing common complications in women following breast cancer surgery. Cochrane Central Register of Controlled Trials, Pubmed, EMBASE, CINAHL, and PEDro were searched from date of inception until the 15th of February 2021, to identify randomized controlled trials which compared Early ROM to either Delayed ROM or UC in women following surgery for breast cancer. Outcomes included shoulder range, wound outcomes and lymphedema incidence. Risk of bias was evaluated with the PEDro scale. Data analysis was conducted in R (version 3.6.0), with a priori sensitivity analyses conducted for studies with low risk of bias and published after the year 2000. Of the 703 articles retrieved, 20 trials (2442 participants) were eligible for inclusion. There were few differences between groups in ROM, except for flexion ROM when more recent evidence was considered. Total drainage time and hematoma incidence were significantly higher in the Early ROM group compared to Delayed ROM (WMD 1·2 days [95%CI 0·7,1·7], and RR 1·6 [95%CI 1·1,2·3], respectively). When considering more recent evidence, there were no differences between groups for these outcomes. There were no differences between groups in other wound outcomes. Lymphedema incidence was higher in the Early ROM group compared to Delayed ROM in the short-term only (RR 3·7 [95% CI 1·3;10·9]), and there was no difference when compared to UC. The quality of evidence using the GRADE approach was generally low to very low. This review found that when considering more contemporary evidence, the timing of exercise may influence ROM, but not wound outcomes. Further research is recommended to understand the effect on lymphedema incidence. No funding was sought for this review. A protocol for this systematic review was posted on the Open Science Framework prior to commencement (DOI: 10.17605/OSF.IO/Q5FHS).

Assessing the impact of the COVID-19 pandemic on breast cancer screening and diagnosis rates: A rapid review and meta-analysis.

OBJECTIVE: The ongoing COVID-19 pandemic has caused an indefinite delay to cancer screening programs worldwide. This study aims to explore the impact on breast cancer screening outcomes such as mammography and diagnosis rates. METHODS: We searched Ovid MEDLINE, Ovid Embase, medRxiv and bioRxiv between January 2020 to October 2021 to identify studies that reported on the rates of screening mammography and breast cancer diagnosis before and during the pandemic. The effects of 'lockdown' measures, age and ethnicity on outcomes were also examined. All studies were assessed for risk of bias using the Newcastle-Ottawa Scale (NOS). Rate ratios were calculated for all outcomes and pooled using standard inverse-variance random effects meta-analysis. RESULTS: We identified 994 articles, of which 7 registry-based and 24 non-registry-based retrospective cohort studies, including data on 4,860,786 and 629,823 patients respectively across 18 different countries, were identified. Overall, breast cancer screening and diagnosis rates dropped by an estimated 41-53% and 18-29% respectively between 2019 and 2020. No differences in mammogram screening rates depending on patient age or ethnicity were observed. However, countries that implemented lockdown measures were associated with a significantly greater reduction in mammogram and diagnosis rates between 2019 and 2020 in comparison to those that did not. CONCLUSION: The pandemic has caused a substantial reduction in the screening and diagnosis of breast cancer, with reductions more pronounced in countries under lockdown restrictions. It is early yet to know if delayed screening during the pandemic translates into higher breast cancer mortality.

Reimagining peer review as an expert elicitation process.

Journal peer review regulates the flow of ideas through an academic discipline and thus has the power to shape what a research community knows, actively investigates, and recommends to policymakers and the wider public. We might assume that editors can identify the 'best' experts and rely on them for peer review. But decades of research on both expert decision-making and peer review suggests they cannot. In the absence of a clear criterion for demarcating reliable, insightful, and accurate expert assessors of research quality, the best safeguard against unwanted biases and uneven power distributions is to introduce greater transparency and structure into the process. This paper argues that peer review would therefore benefit from applying a series of evidence-based recommendations from the empirical literature on structured expert elicitation. We highlight individual and group characteristics that contribute to higher quality judgements, and elements of elicitation protocols that reduce bias, promote constructive discussion, and enable opinions to be objectively and transparently aggregated.

Data and code availability statements in systematic reviews of interventions were often missing or inaccurate: a content analysis.

OBJECTIVES: To estimate the frequency of data and code availability statements in a random sample of systematic reviews with meta-analysis of aggregate data, summarize the content of the statements and investigate how often data and code files were shared. METHODS: We searched for systematic reviews with meta-analysis of aggregate data on the effects of a health, social, behavioral, or educational intervention that were indexed in PubMed, Education Collection via ProQuest, Scopus via Elsevier, or Social Sciences Citation Index and Science Citation Index Expanded via Web of Science during a 4-week period (between November 2, and December 2, 2020). Records were randomly sorted and screened independently by two authors until our target sample of 300 systematic reviews was reached. Two authors independently recorded whether a data or code availability statement (or both) appeared in each review and coded the content of the statements using an inductive approach. RESULTS: Of the 300 included systematic reviews with meta-analysis, 86 (29%) had a data availability statement, and seven (2%) had both a data and code availability statement. In 12/93 (13%) data availability statements, authors stated that data files were available for download from the journal website or a data repository, which we verified as being true. While 39/93 (42%) authors stated data were available upon request, 37/93 (40%) implied that sharing of data files was not necessary or applicable to them, most often because "all data appear in the article" or "no datasets were generated or analyzed". DISCUSSION: Data and code availability statements appear infrequently in systematic review manuscripts. Authors who do provide a data availability statement often incorrectly imply that data sharing is not applicable to systematic reviews. Our results suggest the need for various interventions to increase data and code sharing by systematic reviewers.

Rates and predictors of data and code sharing in the medical and health sciences: Protocol for a systematic review and individual participant data meta-analysis.

Numerous studies have demonstrated low but increasing rates of data and code sharing within medical and health research disciplines. However it remains unclear how commonly data and code are shared across all fields of medical and health research, as well as whether sharing rates are positively associated with implementation of progressive policies by publishers and funders, or growing expectations from the medical and health research community at large. Therefore this systematic review aims to synthesise the findings of medical and health science studies that have empirically investigated the prevalence of data or code sharing, or both. Objectives include the investigation of: (i) the prevalence of public sharing of research data and code alongside published articles (including preprints), (ii) the prevalence of private sharing of research data and code in response to reasonable requests, and (iii) factors associated with the sharing of either research output (e.g., the year published, the publisher's policy on sharing, the presence of a data or code availability statement). It is hoped that the results will provide some insight into how often research data and code are shared publicly and privately, how this has changed over time, and how effective some measures such as the institution of data sharing policies and data availability statements have been in motivating researchers to share their underlying data and code.

The REPRISE project: protocol for an evaluation of REProducibility and Replicability In Syntheses of Evidence.

BACKGROUND: Investigations of transparency, reproducibility and replicability in science have been directed largely at individual studies. It is just as critical to explore these issues in syntheses of studies, such as systematic reviews, given their influence on decision-making and future research. We aim to explore various aspects relating to the transparency, reproducibility and replicability of several components of systematic reviews with meta-analysis of the effects of health, social, behavioural and educational interventions. METHODS: The REPRISE (REProducibility and Replicability In Syntheses of Evidence) project consists of four studies. We will evaluate the completeness of reporting and sharing of review data, analytic code and other materials in a random sample of 300 systematic reviews of interventions published in 2020 (Study 1). We will survey authors of systematic reviews to explore their views on sharing review data, analytic code and other materials and their understanding of and opinions about replication of systematic reviews (Study 2). We will then evaluate the extent of variation in results when we (a) independently reproduce meta-analyses using the same computational steps and analytic code (if available) as used in the original review (Study 3), and (b) crowdsource teams of systematic reviewers to independently replicate a subset of methods (searches for studies, selection of studies for inclusion, collection of outcome data, and synthesis of results) in a sample of the original reviews; 30 reviews will be replicated by 1 team each and 2 reviews will be replicated by 15 teams (Study 4). DISCUSSION: The REPRISE project takes a systematic approach to determine how reliable systematic reviews of interventions are. We anticipate that results of the REPRISE project will inform strategies to improve the conduct and reporting of future systematic reviews.

Journal policies and editors' opinions on peer review.

Peer review practices differ substantially between journals and disciplines. This study presents the results of a survey of 322 editors of journals in ecology, economics, medicine, physics and psychology. We found that 49% of the journals surveyed checked all manuscripts for plagiarism, that 61% allowed authors to recommend both for and against specific reviewers, and that less than 6% used a form of open peer review. Most journals did not have an official policy on altering reports from reviewers, but 91% of editors identified at least one situation in which it was appropriate for an editor to alter a report. Editors were also asked for their views on five issues related to publication ethics. A majority expressed support for co-reviewing, reviewers requesting access to data, reviewers recommending citations to their work, editors publishing in their own journals, and replication studies. Our results provide a window into what is largely an opaque aspect of the scientific process. We hope the findings will inform the debate about the role and transparency of peer review in scholarly publishing.

Comparison between electromagnetic transponders and radiographic imaging for prostate localization: A pelvic phantom study with rotations and translations.

The aim of this study was to evaluate the differences in target localization between Calypso® , kV orthogonal imaging and cone-beam computed tomography (CBCT) for combined translations and rotations of an anthropomorphic pelvic phantom. The phantom was localized using all three systems in 50 different positions, with applied translational and rotational offsets randomly sampled from representative normal distributions of prostate motion. Lin's concordance correlation coefficient (ρc) and 95% confidence intervals were calculated to assess the agreement between the localization systems. Mean differences and difference vectors between the three systems were also calculated. Agreement between systems for lateral, vertical, and longitudinal translations was excellent, with ρc values of greater than 0.98 between all three systems in all axes. There was excellent agreement between the systems for rotations around the lateral axis (pitch) (ρc > 0.99), and around the vertical axis (yaw) (ρc > 0.97). However, somewhat poorer agreement for rotations around the longitudinal axis (roll) was observed, with the lowest correlation observed between Calypso and kV orthogonal imaging (ρc = 0.895). Mean differences between the phantom position reported by Calypso and the radiographic systems were less than 1 mm and 1° for all translations and rotations. The results for translations are consistent with the publications of previous authors. There is no comparable published data for rotations. While there is lower correlation between the three systems for roll than for the other angles, the mean differences in reported rotations are not clinically significant.