Publications poorly report the essential RadiOmics ParametERs (PROPER): A meta-research on quality of reporting.

PURPOSE: To investigate the quality of reporting in radiomics research, with a focus on the most basic technical parameters. METHODS: A PubMed literature search was conducted to identify original studies on radiomics (last search: January 2, 2023). Following a sample size calculation with an a priori power analysis, a random sample of the radiomic literature was collected. In addition to baseline characteristics, the key aspects of radiomic software, resampling, and discretization were evaluated. Agreement between raters was analyzed. Disagreements were resolved through consensus. RESULTS: A sample of 87 publications was evaluated. Most publications (89%; 77/87) were retrospective. They were conducted predominantly with private data (87%; 76/87) at a single institution (77%; 67/87) without external validation (90%; 78/87). 69% (60/87) of the papers reported the radiomic software used (p < 0.001), with nearly half (43%; 26/60) omitting the version. 37% (32/87) reported the resampling size (p = 0.018), while 22% (7/32) did not report using iso-voxel resampling. 34% (30/87) reported the discretization parameters (p < 0.01), but more than three-quarters (77%; 23/30) did not experiment with different discretization parameters. A wide range of discretization parameter values were reported. Most papers (79%; 69/87) failed to report all three essential items simultaneously (p < 0.001). CONCLUSION: Even the essential radiomic parameters that are usually displayed on the user interface of radiomic software tools were poorly reported in radiomics-related publications. This issue of transparency may require additional action from researchers, editors, and reviewers in the form of adopting more stringent reporting standards (e.g., checklists, guidelines).

NEgatiVE results in Radiomics research (NEVER): A meta-research study of publication bias in leading radiology journals.

PURPOSE: The purpose of this study was to conduct a meta-research of radiomics-related articles for the publication of negative results, with a focus on the leading clinical radiology journals due to their purportedly high editorial standards. METHODS: A literature search was performed in PubMed to identify original research studies on radiomics (last search date: August 16th, 2022). The search was restricted to studies published in Q1 clinical radiology journals indexed by Scopus and Web of Science. Following an a priori power analysis based on our null hypothesis, a random sampling of the published literature was conducted. Besides the six baseline study characteristics, a total of three items about publication bias were evaluated. Agreement between raters was analyzed. Disagreements were resolved through consensus. Statistical synthesis of the qualitative evaluations was presented. RESULTS: Following a priori power analysis, we included a random sample of 149 publications in this study. Most of the publications were retrospective (95%; 142/149), based on private data (91%; 136/149), centered on a single institution (75%; 111/149), and lacked external validation (81%; 121/149). Slightly fewer than half (44%; 66/149) made no comparison to non-radiomic approaches. Overall, only one study (1%; 1/149) reported negative results for radiomics, yielding a statistically significant binomial test (p < 0.0001). CONCLUSION: The top clinical radiology journals almost never publish negative results, having a strong bias toward publishing positive results. Almost half of the publications did not even compare their approach with a non-radiomic method.