Software with artificial intelligence-derived algorithms for analysing CT brain scans in people with a suspected acute stroke: a systematic review and cost-effectiveness analysis.

Background: Artificial intelligence-derived software technologies have been developed that are intended to facilitate the review of computed tomography brain scans in patients with suspected stroke. Objectives: To evaluate the clinical and cost-effectiveness of using artificial intelligence-derived software to support review of computed tomography brain scans in acute stroke in the National Health Service setting. Methods: Twenty-five databases were searched to July 2021. The review process included measures to minimise error and bias. Results were summarised by research question, artificial intelligence-derived software technology and study type. The health economic analysis focused on the addition of artificial intelligence-derived software-assisted review of computed tomography angiography brain scans for guiding mechanical thrombectomy treatment decisions for people with an ischaemic stroke. The de novo model (developed in R Shiny, R Foundation for Statistical Computing, Vienna, Austria) consisted of a decision tree (short-term) and a state transition model (long-term) to calculate the mean expected costs and quality-adjusted life-years for people with ischaemic stroke and suspected large-vessel occlusion comparing artificial intelligence-derived software-assisted review to usual care. Results: A total of 22 studies (30 publications) were included in the review; 18/22 studies concerned artificial intelligence-derived software for the interpretation of computed tomography angiography to detect large-vessel occlusion. No study evaluated an artificial intelligence-derived software technology used as specified in the inclusion criteria for this assessment. For artificial intelligence-derived software technology alone, sensitivity and specificity estimates for proximal anterior circulation large-vessel occlusion were 95.4% (95% confidence interval 92.7% to 97.1%) and 79.4% (95% confidence interval 75.8% to 82.6%) for Rapid (iSchemaView, Menlo Park, CA, USA) computed tomography angiography, 91.2% (95% confidence interval 77.0% to 97.0%) and 85.0 (95% confidence interval 64.0% to 94.8%) for Viz LVO (Viz.ai, Inc., San Fransisco, VA, USA) large-vessel occlusion, 83.8% (95% confidence interval 77.3% to 88.7%) and 95.7% (95% confidence interval 91.0% to 98.0%) for Brainomix (Brainomix Ltd, Oxford, UK) e-computed tomography angiography and 98.1% (95% confidence interval 94.5% to 99.3%) and 98.2% (95% confidence interval 95.5% to 99.3%) for Avicenna CINA (Avicenna AI, La Ciotat, France) large-vessel occlusion, based on one study each. These studies were not considered appropriate to inform cost-effectiveness modelling but formed the basis by which the accuracy of artificial intelligence plus human reader could be elicited by expert opinion. Probabilistic analyses based on the expert elicitation to inform the sensitivity of the diagnostic pathway indicated that the addition of artificial intelligence to detect large-vessel occlusion is potentially more effective (quality-adjusted life-year gain of 0.003), more costly (increased costs of £8.61) and cost-effective for willingness-to-pay thresholds of £3380 per quality-adjusted life-year and higher. Limitations and conclusions: The available evidence is not suitable to determine the clinical effectiveness of using artificial intelligence-derived software to support the review of computed tomography brain scans in acute stroke. The economic analyses did not provide evidence to prefer the artificial intelligence-derived software strategy over current clinical practice. However, results indicated that if the addition of artificial intelligence-derived software-assisted review for guiding mechanical thrombectomy treatment decisions increased the sensitivity of the diagnostic pathway (i.e. reduced the proportion of undetected large-vessel occlusions), this may be considered cost-effective. Future work: Large, preferably multicentre, studies are needed (for all artificial intelligence-derived software technologies) that evaluate these technologies as they would be implemented in clinical practice. Study registration: This study is registered as PROSPERO CRD42021269609. Funding: This award was funded by the National Institute for Health and Care Research (NIHR) Evidence Synthesis programme (NIHR award ref: NIHR133836) and is published in full in Health Technology Assessment; Vol. 28, No. 11. See the NIHR Funding and Awards website for further award information.

Stroke is a serious life-threatening medical condition caused by a blood clot or haemorrhage in the brain. Quick and effective management, including a brain scan, of the patients with suspected stroke can make a big difference in their outcome. Artificial intelligence-derived computer programmes exist that are intended to help with the interpretation of computed tomography scans of the brain in stroke. We undertook a thorough review of the existing research into the effectiveness and value for money of using these programmes to help doctors and other specialists to interpret computed tomography brain scans. We found very little evidence to tell us how well artificial intelligence-derived computer programmes work in practice. Some studies have looked at artificial intelligence-derived computer programmes on their own (i.e. not taken together with a doctor's judgement, as they were designed to be used). Other studies have looked at what happens to patients who are treated for stroke when artificial intelligence-derived computer programmes are used; these studies provide no information about whether using artificial intelligence-derived computer programmes may have led to patients who could have benefitted from treatment being missed. It is unclear how well artificial intelligence-derived software-assisted review works when added to current clinical practice.

Oral Azacitidine for Maintenance Treatment of Acute Myeloid Leukaemia After Induction Therapy: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

The National Institute for Health and Care Excellence (NICE) invited the manufacturer (Celgene) of oral azacitidine (ONUREG), as part of the Single Technology Appraisal (STA) process, to submit evidence for the clinical effectiveness and cost-effectiveness of oral azacitidine for maintenance treatment of acute myeloid leukaemia (AML) after induction therapy compared with watch-and-wait plus best supportive care (BSC) and midostaurin. Kleijnen Systematic Reviews Ltd, in collaboration with Maastricht University Medical Centre+, was commissioned to act as the independent Evidence Review Group (ERG). This paper summarises the company submission (CS), presents the ERG's critical review on the clinical and cost-effectiveness evidence in the CS, highlights the key methodological considerations and describes the development of the NICE guidance by the Appraisal Committee. In the QUAZAR AML-001 trial, oral azacitidine significantly improved overall survival (OS) versus placebo: median OS gain of 9.9 months (24.7 months versus 14.8 months; hazard ratio (HR) 0.69 (95% CI 0.55-0.86), p < 0.001). The median time to relapse was also better for oral azacitidine, and the incidences of TEAEs were similar for the two arms. The company excluded two of the comparators listed in the scope, low-dose cytarabine and subcutaneous azacitidine, informed only by clinical expert opinion, leaving only best supportive care (BSC) and midostaurin for the FLT3-ITD and/or FLT3-TKD (FLT3 mutation)-positive subgroup. An ITC comparing oral azacitidine to midostaurin as maintenance therapy in the appropriate subgroup demonstrated that the OS and relapse-free survival (RFS) HRs were favourable for oral azacitidine when compared with midostaurin. However, in the only available trial of midostaurin as maintenance treatment in AML that was used for this ITC, subjects were not randomised at the maintenance phase, but at induction, which posed a substantial risk of bias. The revised and final probabilistic incremental cost-effectiveness ratio (ICER) presented by the company, including a commercial arrangement, was £32,480 per quality-adjusted life year (QALY) gained for oral azacitidine versus watch-and-wait plus BSC. Oral azacitidine was dominant versus midostaurin in the FLT-3 subgroup. The ERG's concerns included the approach of modelling haematopoietic stem cell transplantation (HSCT), the generalisability of the population and the number of cycles of consolidation therapy pre-treatment in the QUAZAR AML-001 trial to UK clinical practice, and uncertainty in the relapse utility. The revised and final ERG base case resulted in a similar probabilistic ICER of £33,830 per QALY gained versus watch-and-wait plus BSC, but with remaining uncertainty. Oral azacitidine remained dominant versus midostaurin in the FLT-3 subgroup. After the second NICE appraisal committee meeting, the NICE Appraisal Committee recommended oral azacitidine (according to the commercial arrangement), within its marketing authorisation, as an option for maintenance treatment for AML in adults who are in complete remission, or complete remission with incomplete blood count recovery, after induction therapy with or without consolidation treatment, and cannot have or do not want HSCT.

Abemaciclib in Combination with Endocrine Therapy for Adjuvant Treatment of Hormone Receptor-Positive, HER2-Negative, Node-Positive Early Breast Cancer: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

The National Institute for Health and Care Excellence (NICE) invited the manufacturer (Eli Lilly) of abemaciclib (Verzenios) to submit evidence for the clinical and cost effectiveness of this drug in combination with endocrine therapy (ET) for the treatment of adult patients with hormone receptor (HR)-positive, human epidermal growth factor receptor 2 (HER2)-negative, node-positive early breast cancer at high risk of recurrence, as part of the Institute's Single Technology Appraisal (STA) process. Kleijnen Systematic Reviews Ltd, in combination with Newcastle University, was commissioned to act as the independent Evidence Review Group (ERG). This paper summarised the Company Submission (CS), presents the ERG's critical review of the clinical and cost-effectiveness evidence in the CS, highlights the key methodological considerations, and describes the development of the NICE guidance by the Appraisal Committee. The ERG produced a critical review of the evidence for the clinical and cost-effectiveness evidence in the CS and also independently searched for relevant evidence and modified the manufacturer decision analytic model to examine the impact of altering some of the key assumptions. A systematic literature review identified the MonarchE trial, an ongoing, open-label, randomised, double blind trial involving 5637 people comparing abemaciclib in combination with ET versus ET alone. The trial included two cohorts that used different inclusion criteria to define high risk of recurrence. The ERG considered Cohort 1 as an adequate representation of this population and the AC concluded that Cohort 1 was generalisable to National Health Service clinical practice. Trial results showed improvements in invasive disease-free survival for the abemaciclib arm, which was considered an appropriate surrogate outcome. The ERG believed that the modelling structure presented in the de novo economic model by the company was appropriate but highlighted several areas of uncertainty that had the potential to have a significant impact on the resulting incremental cost-effectiveness ratio (ICER). Areas of uncertainty included the extrapolation of long-term survival curves, the duration of treatment effect and treatment waning, and the proportion of patients who receive other CDK4/6 treatments for metastatic disease after receiving abemaciclib. ICER estimates were £9164 per quality-adjusted life-year gained for the company's base-case and £17,810 for the ERG's base-case. NICE recommended abemaciclib with ET as an option for the adjuvant treatment of HR-positive, HER2-negative, node-positive early breast cancer at high risk of recurrence.

Pralsetinib for RET Fusion-Positive Advanced Non-small-Cell Lung Cancer: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

The National Institute for Health and Care Excellence (NICE) invited the manufacturer (Roche) of pralsetinib (Gavreto®), as part of the single technology appraisal (STA) process, to submit evidence for the clinical effectiveness and cost effectiveness of pralsetinib for the treatment of adult patients with rearranged during transfection (RET) fusion-positive advanced non-small-cell lung cancer (NSCLC) not previously treated with a RET inhibitor. Kleijnen Systematic Reviews Ltd, in collaboration with University Medical Center Groningen, was commissioned to act as the independent Evidence Review Group (ERG). This paper summarizes the company submission (CS), presents the ERG's critical review of the clinical and cost-effectiveness evidence in the CS, highlights the key methodological considerations, and describes the development of the NICE guidance by the Appraisal Committee. The CS reported data from the ARROW trial. ARROW is a single-arm, multicenter, non-randomized, open-label, multi-cohort study in patients with RET fusion-positive NSCLC and other advanced solid tumors. The CS included both untreated and pre-treated RET fusion-positive NSCLC patients, among other disease types. The comparators in the untreated population were pembrolizumab + pemetrexed + chemotherapy and pembrolizumab monotherapy. The comparators for the pre-treated population were docetaxel monotherapy, docetaxel + nintedanib, and platinum-based chemotherapy ± pemetrexed. As no comparators were included in ARROW, an indirect treatment comparison was conducted to estimate relative effectiveness. The ERG's concerns included the immaturity of data, small sample size, and lack of comparative safety evidence. The ERG considers the clinical evidence presented to be insufficiently robust to inform the economic model. Even when all the ERG preferred assumptions were implemented in the model, uncertainty remained on a number of issues, such as the appropriateness of the hazard ratios and the methods and data used to derive them, long-term efficacy of pralsetinib, and direct evidence for health-related quality of life (HRQoL). NICE did not recommend pralsetinib within its marketing authorization for treating RET fusion-positive advanced non-small-cell lung cancer (NSCLC) in adults who have not had a RET inhibitor before. The uncertainty of the clinical evidence and the estimates of cost effectiveness were too high to be considered a cost-effective use of NHS resources. Therefore, pralsetinib was not recommended for routine use.

Filgotinib for Treating Moderately to Severely Active Ulcerative Colitis: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

The National Institute for Health and Care Excellence invited the manufacturer (Galapagos) of filgotinib (Jyseleca®), as part of the Single Technology Appraisal process, to submit evidence for the clinical effectiveness and cost effectiveness of filgotinib for treating moderately to severely active ulcerative colitis in adults who have had an inadequate response, loss of response or were intolerant to a previous biologic agent or conventional therapy. Kleijnen Systematic Reviews Ltd, in collaboration with Maastricht University Medical Centre+, was commissioned to act as the independent Evidence Review Group. This paper summarises the company submission, presents the Evidence Review Group's critical review on the clinical and cost-effectiveness evidence in the company submission, highlights the key methodological considerations and describes the development of the National Institute for Health and Care Excellence guidance by the Appraisal Committee. The company submission included one relevant study for the comparison of filgotinib versus placebo: the SELECTION trial. As there was no head-to-head evidence with any of the comparators, the company performed two separate network meta-analyses, one for the biologic-naïve population and one for the biologic-experienced population, and for both the induction and maintenance phases. The Evidence Review Group questioned the validity of the maintenance network meta-analysis because it assumed all active treatments to be comparators in this phase, which is not in line with clinical practice. The economic analysis used a number of assumptions that introduced substantial uncertainty, which could not be fully explored, for instance, the assumption that a risk of loss of response would be independent of health state and constant over time. Company and Evidence Review Group results indicate that at its current price, and disregarding confidential discounts for comparators and subsequent treatments, filgotinib dominates some comparators (golimumab and adalimumab in the company base case, all but intravenous and subcutaneous vedolizumab in the Evidence Review Group's base case) in the biologic-naïve population. In the biologic-experienced population, filgotinib dominates all comparators in both the company and the Evidence Review Group's base case. Results should be interpreted with caution as some important uncertainties were not included in the modelling. These uncertainties were mostly centred around the maintenance network meta-analysis, loss of response, health-related quality-of-life estimates and modelling of dose escalation. The National Institute for Health and Care Excellence recommended filgotinib within its marketing authorisation, as an option for treating moderately to severely active ulcerative colitis in adults when conventional or biological treatment cannot be tolerated, or if the disease has not responded well enough or has stopped responding to these treatments, and if the company provides filgotinib according to the commercial arrangement.

Mogamulizumab for Previously Treated Mycosis Fungoides and Sézary Syndrome: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

The National Institute for Health and Care Excellence (NICE) invited the manufacturer (Kyowa Kirin) of mogamulizumab (Poteligeo®), as part of the single technology appraisal process, to submit evidence for its clinical and cost-effectiveness for previously treated mycosis fungoides (MF) and Sézary syndrome (SS). Kleijnen Systematic Reviews Ltd, in collaboration with Maastricht University Medical Centre, was commissioned to act as the independent evidence review group (ERG). This paper summarises the company submission (CS), presents the ERG's critical review of the clinical and cost-effectiveness evidence in the CS, highlights the key methodological considerations and describes the development of the NICE guidance by the Appraisal Committee. Based on a systematic literature review, one randomised controlled trial, MAVORIC, was identified showing favourable results in patients with MF and SS. However, MAVORIC compared mogamulizumab to vorinostat, which is not standard care in the NHS, and there is uncertainty due to the study design, specifically crossover of patients. Based on a "naïve comparison of results from the vorinostat arm of the MAVORIC study and the physician's choice arm (methotrexate or bexarotene i.e. United Kingdom [UK] standard treatments) of the ALCANZA study as well as comparison to Phase II bexarotene data", the company considered vorinostat to be "a reasonable proxy for current standard of care in the NHS". The ERG considered, based on the limited data available, that the comparability of vorinostat (MAVORIC) and physician's choice (ALCANZA) could not be established. In response to the Appraisal Consultation Document, the company provided an unanchored matched adjusted indirect comparison (MAIC) of mogamulizumab with UK standard care by analysing Hospital Episode Statistics (HES) data. However, given the high risk of bias of an unanchored MAIC, these results needed to be regarded with a considerable degree of caution. The economic analysis suffered from uncertainty because there was no trial evidence on the comparator in the England and Wales National Health Service (NHS), and it was unclear to what extent the trial (MAVORIC) comparator (vorinostat) was comparable to standard care, referred to as established clinical management (ECM) in the NHS. The evidence for overall survival had not reached maturity and was confounded by treatment switching, for which different crossover adjustment methods produced large variations in life years. Caregiver utilities were applied in the analysis, but there was a lack of guidance on their application and whether these were indicated in this appraisal. After consultation, the company updated the economic analysis with the MAIC. Incremental cost-effectiveness ratios comparing mogamulizumab against ECM were (depending on whether the HES or MAVORIC comparison were used) £31,030 or £32,634 per quality-adjusted life years (QALYs) gained according to the company's base case and £38,274 or £80,555 per QALY gained according to the ERG's base case. NICE did not recommend mogamulizumab for treating MF or SS in adults who have had at least one previous systemic treatment. This decision was subsequently appealed, and an appeal decision has been reached.

Filgotinib for Moderate to Severe Rheumatoid Arthritis: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

The National Institute for Health and Care Excellence (NICE) invited the manufacturer (Gilead) of filgotinib (JyselecaTM), as part of the single technology appraisal process, to submit evidence for its clinical and cost effectiveness for the treatment of patients with moderate to severe rheumatoid arthritis (RA). Kleijnen Systematic Reviews Ltd, in collaboration with Maastricht University Medical Centre, was commissioned to act as the independent Evidence Review Group (ERG). This paper summarises the company submission (CS), presents the ERG's critical review of the clinical- and cost-effectiveness evidence in the CS, highlights the key methodological considerations, and describes the development of the NICE guidance by the NICE Appraisal Committee. The evidence for filgotinib was based on two good-quality international randomised controlled trials. In FINCH 1, filgotinib was compared with placebo, and in FINCH 2, filgotinib was compared with adalimumab and placebo. As there was no head-to-head evidence with most active comparators, the company performed two separate network meta-analyses (NMAs), one for the conventional disease-modifying antirheumatic drugs-inadequate response population and one for the biological disease-modifying antirheumatic drugs-inadequate response population. The outcomes analysed were American College of Rheumatology response criteria at weeks 12 and 24, and European League Against Rheumatism response criteria at 24 weeks. The statistical methods used to perform the NMAs were valid and were in line with previous NICE appraisals. Results of the NMAs are confidential and cannot be reported here, but they were uncertain due to heterogeneity of the included studies. The economic analysis of the patient population with moderate RA suffered from limited evidence on the progression from moderate to severe health states. For the moderate RA population, the final analyses comparing filgotinib, with or without methotrexate, against standard of care resulted in incremental cost-effectiveness ratios of around £20,000 per quality-adjusted life-year gained in the company's and ERG's base-case and scenario analyses. NICE recommended filgotinib in combination with methotrexate or as monotherapy when methotrexate is contraindicated, or if people cannot tolerate it, for patients with moderate RA whose disease had responded inadequately to two or more conventional disease-modifying antirheumatic drugs (DMARDs). For the severe RA population, in view of the higher or similar net health benefits that filgotinib provided versus its comparators, NICE recommended filgotinib with or without methotrexate for patients whose disease had responded inadequately to two or more conventional DMARDs, who had been treated with one or more biological DMARDs, if rituximab was not an option, or after treatment with rituximab.

SpaceOAR Hydrogel Spacer for Reducing Radiation Toxicity During Radiotherapy for Prostate Cancer. A Systematic Review.

OBJECTIVE: To evaluate the association between SpaceOAR and radiation dosing, toxicity and quality-of-life vs no spacer across all radiotherapy modalities for prostate cancer. METHODS: A systematic search of the Cochrane Central Register of Controlled Trials, MEDLINE, and Embase was performed from database inception through May 2020. Two reviewers independently screened titles/abstracts and full papers. Data extraction was performed, and quality assessed by 1 reviewer and checked by a second, using a third reviewer as required. The synthesis was narrative. RESULTS: 19 studies (3,622 patients) were included (only 1 randomized controlled trial, in image-guided intensity-modulated radiotherapy (IG-IMRT), 18 comparatives non-randomized controlled trials in external-beam radiotherapy (EBRT), brachytherapy, and combinations thereof). No hypofractionation studies were found. Regardless of radiotherapy type, SpaceOAR significantly reduced rectal radiation dose (eg, V40 average difference -6.1% in high dose-rate brachytherapy plus IG-IMRT to -9.1% in IG-IMRT) and reduced gastrointestinal and genitourinary toxicities (eg, late gastrointestinal toxicity 1% vs 6% (P = .01), late genitourinary toxicity of 15% vs 32% (P < .001) in stereotactic body radiotherapy). Improvements were observed in most Expanded Prostate Cancer Index Composite quality-of-life domains (eg, bowel function score decrease at 3 and 6 months: Average change of zero vs -6.25 and -3.57 respectively in low dose-rate brachytherapy plus EBRT). CONCLUSION: The randomized controlled trial in IG-IMRT demonstrated that SpaceOAR reduces rectal radiation dose and late gastrointestinal and genitourinary toxicities, with urinary, bowel, and sexual quality-of-life improvement. These advantages were verified in observational studies in various radiotherapy types. Further research is required in hypofractionation.