ABSTRACT
BACKGROUND: Systematic literature screening is a key component in systematic reviews. However, this approach is resource intensive as generally two persons independently of each other (double screening) screen a vast number of search results. To develop approaches for increasing efficiency, we tested the use of text mining to prioritize search results as well as the involvement of only one person (single screening) in the study selection process. METHOD: Our study is based on health technology assessments (HTAs) of drug and non-drug interventions. Using a sample size calculation, we consecutively included 11 searches resulting in 33 study selection processes. Of the three screeners for each search, two used screening tools with prioritization (Rayyan, EPPI Reviewer) and one a tool without prioritization. For each prioritization tool, we investigated the proportion of citations classified as relevant at three cut-offs or STOP criteria (after screening 25%, 50% and 75% of the citation set). For each STOP criterion, we measured sensitivity (number of correctly identified relevant studies divided by the total number of relevant studies in the study pool). In addition, we determined the number of relevant studies identified per single screening round and investigated whether missed studies were relevant to the HTA conclusion. RESULTS: Overall, EPPI Reviewer performed better than Rayyan and identified the vast majority (88%, Rayyan 66%) of relevant citations after screening half of the citation set. As long as additional information sources were screened, it was sufficient to apply a single-screening approach to identify all studies relevant to the HTA conclusion. Although many relevant publications (n = 63) and studies (n = 29) were incorrectly excluded, ultimately only 5 studies could not be identified at all in 2 of the 11 searches (1x 1 study, 1x 4 studies). However, their omission did not change the overall conclusion in any HTA. CONCLUSIONS: EPPI Reviewer helped to identify relevant citations earlier in the screening process than Rayyan. Single screening would have been sufficient to identify all studies relevant to the HTA conclusion. However, this requires screening of further information sources. It also needs to be considered that the credibility of an HTA may be questioned if studies are missing, even if they are not relevant to the HTA conclusion.
Subject(s)
Data Mining , Information Sources , Humans , Systematic Reviews as Topic , Technology Assessment, BiomedicalABSTRACT
BACKGROUND: Vestibular schwannomas are benign tumours for which various treatments are available. We performed a systematic review of prospective controlled trials comparing the patient-relevant benefits and harms of single-fraction stereotactic radiosurgery (sfSRS) with microsurgical resection (MR) in patients with vestibular schwannoma. METHODS: We searched for randomized controlled trials (RCTs) and non-randomized prospective controlled trials in MEDLINE, Embase, the Cochrane Central Register of Controlled Trials, and study registries (last search: 09/2021) and also screened reference lists of relevant systematic reviews. Manufacturers were asked to provide unpublished data. Eligible studies investigated at least one patient-relevant outcome. We assessed the risk of bias (high or low) at the study and outcome level. If feasible, meta-analyses were performed. We graded the results into different categories (hint, indication, or proof of greater benefit or harm). RESULTS: We identified three non-randomized prospective controlled trials of generally low quality with evaluable data on 339 patients with unilateral vestibular schwannoma. There was an indication of greater benefit of sfSRS compared with MR for facial palsy (OR 0.06, 95% CI 0.02-0.21, p < 0.001, 2 studies), hearing function (no pooled estimate available, 2 studies), and length of hospital stay (no pooled estimate available, 2 studies). We found no clinically relevant differences for mortality, vertigo, headaches, tinnitus, balance function, work disability, adverse events, and health-related quality of life. CONCLUSIONS: Our systematic review indicates that sfSRS has greater benefits than MR in patients with unilateral vestibular schwannoma. However, it is unclear whether this conclusion still holds after 2 years, as long-term studies are lacking. It is also unclear whether the effects of sfSRS are similar in patients with bilateral vestibular schwannomas. Long-term prospective studies including patients with this condition would therefore be useful. SYSTEMATIC REVIEW REGISTRATION: The full (German language) protocol and report (Commission No. N20-03) are available on the institute's website: www.iqwig.de/en/projects/n20-03.html.