Scoping Meta-Review of Methods Used to Assess Artificial Intelligence-Based Medical Devices for Heart Failure.

Artificial intelligence and machine learning (AI/ML) are playing increasingly important roles, permeating the field of medical devices (MDs). This rapid progress has not yet been matched by the Health Technology Assessment (HTA) process, which still needs to define a common methodology for assessing AI/ML-based MDs. To collect existing evidence from the literature about the methods used to assess AI-based MDs, with a specific focus on those used for the management of heart failure (HF), the International Federation of Medical and Biological Engineering (IFMBE) conducted a scoping meta-review. This manuscript presents the results of this search, which covered the period from January 1974 to October 2022. After careful independent screening, 21 reviews, mainly conducted in North America and Europe, were retained and included. Among the findings were that deep learning is the most commonly utilised method and that electronic health records and registries are among the most prevalent sources of data for AI/ML algorithms. Out of the 21 included reviews, 19 focused on risk prediction and/or the early diagnosis of HF. Furthermore, 10 reviews provided evidence of the impact on the incidence/progression of HF, and 13 on the length of stay. From an HTA perspective, the main areas requiring improvement are the quality assessment of studies on AI/ML (included in 11 out of 21 reviews) and their data sources, as well as the definition of the criteria used to assess the selection of the most appropriate AI/ML algorithm.

Scanning the horizon in a decentralized healthcare system: the Canadian experience.

Canada has a highly decentralized health care system with 13 provinces and territories delivering health care within their own respective jurisdictions. Decisions regarding which innovative health technologies to adopt are often driven by the unique health care priorities of each jurisdiction's population. To understand these needs, the Canadian Agency for Drugs and Technologies in Health's (CADTH's) Early Awareness Service has expanded its activities. In addition to proactively scanning the horizon for new and emerging health technologies, the Early Awareness Service also scans the horizon for national and jurisdictional health policy issues. This paper looks at CADTH's process for identifying and monitoring policy issues at a national and jurisdictional level. CADTH's Early Awareness Service delivers timely information on emerging health care concerns and technologies that may affect health care finances, facilities, operations, and patient care. The identification of important policy issues can help determine which new and emerging technologies will have the most significant impact on the health care system. The information that CADTH scans can also be used to help decision-makers prepare for potential developments and events that may have an impact on health care systems. By improving its capability to identify and share policy issues across and within jurisdictions, CADTH is better situated to provide information that can be used by policy-makers to help them plan and anticipate for the introduction of new technologies and future developments affecting the unique health care needs of their jurisdictions.

The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies.

OBJECTIVES: The English language is generally perceived to be the universal language of science. However, the exclusive reliance on English-language studies may not represent all of the evidence. Excluding languages other than English (LOE) may introduce a language bias and lead to erroneous conclusions. STUDY DESIGN AND SETTING: We conducted a comprehensive literature search using bibliographic databases and grey literature sources. Studies were eligible for inclusion if they measured the effect of excluding randomized controlled trials (RCTs) reported in LOE from systematic review-based meta-analyses (SR/MA) for one or more outcomes. RESULTS: None of the included studies found major differences between summary treatment effects in English-language restricted meta-analyses and LOE-inclusive meta-analyses. Findings differed about the methodological and reporting quality of trials reported in LOE. The precision of pooled estimates improved with the inclusion of LOE trials. CONCLUSIONS: Overall, we found no evidence of a systematic bias from the use of language restrictions in systematic review-based meta-analyses in conventional medicine. Further research is needed to determine the impact of language restriction on systematic reviews in particular fields of medicine.

Pan-Canadian consensus recommendations for proton beam therapy access in Canada.

PURPOSE: Proton Beam Therapy (PBT)is a treatment option for select cancer patients. It is currently not available in Canada. Assessment and referral processes for out-of-country treatment for eligible patients vary by jurisdiction, leading to variability in access to this treatment for Canadian cancer patients. The purpose of this initiative was to develop a framework document to inform consistent and equitable PBT access for appropriate patients through the creation of pan-Canadian PBT access consensus recommendations. MATERIALS AND METHODS: A modified Delphiprocess was used to develop pan-Canadian recommendations with input from 22 PBT clinical and administrative experts across all provinces, external peer-review by provincial cancer and system partners, and feedback from a targeted community consultation. This was conducted by electronic survey and live discussion. Consensus threshold was set at 70% agreement. RESULTS: Fourconsensus rounds resulted in a final set of 27 recommendations divided into three categories: patient eligibility (n = 9); program level (n = 10); and system level (n = 8). Patient eligibility included: anatomic site (n = 4), patient characteristics (n = 3), clinical efficacy (n = 2). Program level included: regulatory and staff requirements (n = 5), equipment and technologies (n = 4), quality assurance (n = 1). System level included: referral process (n = 5), costing, budget impact and quality adjusted life years (n = 2), eligible patient estimates (n = 1). Recommendations were released nationally in June 2021 and distributed to all 43 cancer programs in Canada. CONCLUSION: A pan-Canadian consensus-building approach was successful in creating an evidence-based, peer-reviewed suite of recommendations thatsupportapplication of consistent clinical criteria to inform treatment options, facility set-up and access to high quality proton therapy.

Public health measures to reduce the risk of SARS-CoV-2 transmission in Canada during the early days of the COVID-19 pandemic: a scoping review.

OBJECTIVE: The main objectives of this study were to synthesise and compare pandemic preparedness strategies issued by the federal and provincial/territorial (P/T) governments in Canada and to assess whether COVID-19 public health (PH) measures were tailored towards priority populations, as defined by relevant social determinants of health. METHODS: This scoping review searched federal and P/T websites on daily COVID-19 pandemic preparedness strategies between 30 January and 30 April 2020. The PROGRESS-Plus equity-lens framework was used to define priority populations. All definitions, policies and guidelines of PH strategies implemented by the federal and P/T governments to reduce risk of SARS-CoV-2 transmission were included. PH measures were classified using a modified Public Health Agency of Canada Framework for Canadian Pandemic Influenza Preparedness. RESULTS: A total of 722 COVID-19 PH measures were issued during the study period. Of these, home quarantine (voluntary) (n=13.0%; 94/722) and retail/commerce restrictions (10.9%; n=79/722) were the most common measures introduced. Many of the PH orders, including physical distancing, cancellation of mass gatherings, school closures or retail/commerce restrictions began to be introduced after 11 March 2020. Lifting of some of the PH orders in phases to reopen the economy began in April 2020 (6.5%; n=47/722). The majority (68%, n=491/722) of COVID-19 PH announcements were deemed mandatory, while 32% (n=231/722) were recommendations. Several PH measures (28.0%, n=202/722) targeted a variety of groups at risk of socially produced health inequalities, such as age, religion, occupation and migration status. CONCLUSIONS: Most PH measures centred on limiting contact between people who were not from the same household. PH measures were evolutionary in nature, reflecting new evidence that emerged throughout the pandemic. Although ~30% of all implemented COVID-19 PH measures were tailored towards priority groups, there were still unintended consequences on these populations.

Economic assessment of different modalities of immunoglobulin replacement therapy.

The delivery (intravenous or subcutaneous), location (home or hospital), and other factors of immunoglobulin replacement therapy are examined for cost effectiveness. Cost-minimization studies from several countries are reviewed and analyzed. A Canadian cost-minimization study is performed. Although common themes emerge, there are cost differences between the various countries.

A Review and Comparative Analysis of Information Targeted to the General Public on the Websites of Breast Screening Programs in Canada.

Organized breast screening programs in Canada recommend that women, usually 50-74 years of age, are screened regularly with mammography to reduce their risk of breast cancer death. There is increasing evidence that estimates of mortality reduction are overestimated and harms under-reported. This article will report on a review of the websites of 12 breast screening programs in Canada. The primary goal is to determine what information is provided to enable women to make an informed decision about mammography and whether choice is emphasized. All publicly available English language information was extracted from the 12 websites by two independent reviewers, using a data extraction sheet. Information extracted included eligible age, screening interval and potential benefits and harms. This review is relevant to policy makers and breast screening program staff so they can determine what additional or alternative information is required on their websites to enable women to make informed decisions.

Efficacy of omega-3 fatty acids in preventing age-related macular degeneration: a systematic review.

TOPIC: What is the evidence for efficacy of dietary and/or supplemental omega-3 fatty acids in preventing age-related macular degeneration (AMD)? CLINICAL RELEVANCE: Age-related macular degeneraion is the leading cause of blindness and vision impairment in persons older than 50 years living in North America. There is no cure for AMD, and treatment does not usually restore vision but only prevents disease progression to a modest degree. omega-3 fatty acids are considered potentially important antioxidants and are being considered as an arm of the Age-Related Eye Disease Study II clinical trial. METHODS/LITERATURE REVIEWED: Keywords were searched in Medline, Pre-Medline, Embase, and the Cochrane Library on Ovid. There was no restriction on the year or language of publication. RESULTS: There were 6 observational studies found, but the specific outcomes, exposures, and covariates studied all varied greatly. CONCLUSION: There is some clinical evidence for protection of AMD from omega-3 fatty acids. However, the results are not consistent. Hence, our conclusion is that this issue is neither clearly supported nor refuted by the present world literature. This is an intriguing and extremely important question but needs further study first with prospective cohort designs and, if positive, randomized clinical trials.

Many quality measurements, but few quality measures assessing the quality of breast cancer care in women: a systematic review.

BACKGROUND: Breast cancer in women is increasingly frequent, and care is complex, onerous and expensive, all of which lend urgency to improvements in care. Quality measurement is essential to monitor effectiveness and to guide improvements in healthcare. METHODS: Ten databases, including Medline, were searched electronically to identify measures assessing the quality of breast cancer care in women (diagnosis, treatment, followup, documentation of care). Eligible studies measured adherence to standards of breast cancer care in women diagnosed with, or in treatment for, any histological type of adenocarcinoma of the breast. Reference lists of studies, review articles, web sites, and files of experts were searched manually. Evidence appraisal entailed dual independent assessments of data (e.g., indicators used in quality measurement). The extent of each quality indicator's scientific validation as a measure was assessed. The American Society of Clinical Oncology (ASCO) was asked to contribute quality measures under development. RESULTS: Sixty relevant reports identified 58 studies with 143 indicators assessing adherence to quality breast cancer care. A paucity of validated indicators (n = 12), most of which assessed quality of life, only permitted a qualitative data synthesis. Most quality indicators evaluated processes of care. CONCLUSION: While some studies revealed patterns of under-use of care, all adherence data require confirmation using validated quality measures. ASCO's current development of a set of quality measures relating to breast cancer care may hold the key to conducting definitive studies.

An alternative to the hand searching gold standard: validating methodological search filters using relative recall.

BACKGROUND: Search filters or hedges play an important role in evidence-based medicine but their development depends on the availability of a "gold standard"--a reference standard against which to establish the performance of the filter. We demonstrate the feasibility of using relative recall of included studies from multiple systematic reviews to validate methodological search filters as an alternative to validation against a gold standard formed through hand searching. METHODS: We identified 105 Cochrane reviews that used the Highly Sensitive Search Strategy (HSSS), included randomized or quasi-randomized controlled trials, and reported their included studies. We measured the ability of two published and one novel variant of the HSSS to retrieve the MEDLINE-index studies included in these reviews. RESULTS: The systematic reviews were comprehensive in their searches. 72% of included primary studies were indexed in MEDLINE. Relative recall of the three strategies ranged from .98 to .91 across all reviews and more comprehensive strategies showed higher recall. CONCLUSION: An approach using relative recall instead of a hand searching gold standard proved feasible and produced recall figures that were congruent with previously published figures for the HSSS. This technique would permit validation of a methodological filter using a collection of approximately 100 studies of the chosen design drawn from the included studies of multiple systematic reviews that used comprehensive search strategies.

Can electronic search engines optimize screening of search results in systematic reviews: an empirical study.

BACKGROUND: Most electronic search efforts directed at identifying primary studies for inclusion in systematic reviews rely on the optimal Boolean search features of search interfaces such as DIALOG and Ovid. Our objective is to test the ability of an Ultraseek search engine to rank MEDLINE records of the included studies of Cochrane reviews within the top half of all the records retrieved by the Boolean MEDLINE search used by the reviewers. METHODS: Collections were created using the MEDLINE bibliographic records of included and excluded studies listed in the review and all records retrieved by the MEDLINE search. Records were converted to individual HTML files. Collections of records were indexed and searched through a statistical search engine, Ultraseek, using review-specific search terms. Our data sources, systematic reviews published in the Cochrane library, were included if they reported using at least one phase of the Cochrane Highly Sensitive Search Strategy (HSSS), provided citations for both included and excluded studies and conducted a meta-analysis using a binary outcome measure. Reviews were selected if they yielded between 1000-6000 records when the MEDLINE search strategy was replicated. RESULTS: Nine Cochrane reviews were included. Included studies within the Cochrane reviews were found within the first 500 retrieved studies more often than would be expected by chance. Across all reviews, recall of included studies into the top 500 was 0.70. There was no statistically significant difference in ranking when comparing included studies with just the subset of excluded studies listed as excluded in the published review. CONCLUSION: The relevance ranking provided by the search engine was better than expected by chance and shows promise for the preliminary evaluation of large results from Boolean searches. A statistical search engine does not appear to be able to make fine discriminations concerning the relevance of bibliographic records that have been pre-screened by systematic reviewers.

The evidence for efficacy of omega-3 fatty acids in preventing or slowing the progression of retinitis pigmentosa: a systematic review.

BACKGROUND: Studies in preterm and term human infants have suggested that a dietary supply of omega-3 fatty acids is essential for optimal visual development. Several basic science studies support the hypothesis that omega-3 fatty acids may be useful therapeutic agents for pathologies of the retina and lens. As part of a systematic review of the effect of omega-3 fatty acids on eye health, the purpose of this study was to conduct a systematic review of the scientific-medical literature to appraise and synthesize the evidence for the effects of omega-3 fatty acids in preventing the development or progression of retinitis pigmentosa. METHODS: A comprehensive search was undertaken in MEDLINE, PREMEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, Global Health, and Dissertation Abstracts. Unpublished literature was sought through manual searches of reference lists of included studies and key review articles and from the files of content experts. Searches were not restricted by language of publication, publication type, or study design. Eligibility criteria were applied to screen eligible studies on two levels. Data extraction and quality assessment were performed. RESULTS: Six studies published between 1995 and 2004 met eligibility criteria in investigating the question of the possible value of omega-3 fatty acids in slowing the progression of retinitis pigmentosa. Meta-analysis was not performed because there was not enough available information for formal quantitative analysis. INTERPRETATION: There are trends in improvement of some retinitis pigmentosa outcomes with omega-3 fatty acids in the higher quality studies. Clinical research is preliminary in this field, however. Accordingly, definitive answers will require significantly more observational and interventional clinical research.

Pharmacologic management of neovascular age-related macular degeneration: systematic review of economic evidence and primary economic evaluation.

OBJECTIVE: To examine the economic implications for the Canadian health system of pharmacologic treatment of neovascular age-related macular degeneration (AMD). DESIGN: Systematic review of economic literature and a primary economic evaluation. PARTICIPANTS: Economic literature search identified 392 potentially relevant articles, 12 of which were included for final review. METHODS: Studies were included if they met the following criteria: (i) provision of a summary measure of the trade-off between costs and consequences; (ii) participants of 40 years and older with neovascular AMD; (iii) interventions and comparators: comparison of photodynamic therapy using verteporfin (V-PDT), pegaptanib, bevacizumab, ranibizumab, anecortave acetate, intravitreal triamcinolone, placebo, or clinically relevant combinations; and (iv) outcome reported as an incremental measure of the implication of moving from the comparator to the intervention. The following databases were searched through the OVID interface: MEDLINE, EMBASE, BIOSIS Previews, CINAHL, PubMed, Health Economic Evaluations Database (HEED), and the Cochrane Library. For the economic evaluation, we took a decision analytic approach and modeled a cost-utility analysis, conducting it as a microsimulation of a Markov model. RESULTS: In general, V-PDT is more cost effective than conventional macular laser, and pegaptanib is likely more cost effective than V-PDT. The primary economic analysis revealed ranibizumab to be effective but at an unacceptably high cost per quality-adjusted life year (QALY)(>$50,000 per QALY). CONCLUSIONS: Although ranibizumab is effective for wet AMD, its cost is unacceptably high based on cost-utility theory.

Health technology assessment: a review of international activity and examples of approaches with computed tomographic colonography.

The growth of health technology assessment (HTA) internationally is currently reflected in the growing membership of the International Network of Agencies for Health Technology Assessment. Many national and regional HTA institutions emerged in the 1980s and 1990s, and more recently, HTA has emerged in newly industrialized countries and in European Union member states in transition. Health technology assessment activities are becoming an increasingly important part of health care culture, with the appearance of HTA units in hospitals and hospital departments. This article provides a brief overview of who conducts HTA internationally and looks at how HTA is conducted and how this information is used. To highlight the different structures, processes, and methods available, a portion of this article is dedicated to describing different approaches that have been observed with respect to the assessment of computed tomographic colonography in North America for population-based colorectal cancer screening.

A pragmatic critical appraisal instrument for search filters: introducing the CADTH CAI.

OBJECTIVE: To identify or develop a critical appraisal instrument (CAI) to aid in the selection of search filters for use in systematic review searching. The CAI is to be used by experienced searchers without specialized training in statistics or search filter design. METHODS: Through extensive searching and consultation, one candidate instrument was identified. Through expert consultation and several rounds of testing, the instrument was extensively revised to become the Canadian Agency for Drugs and Technologies in Health (CADTH) CAI. RESULTS: The CADTH CAI consists of ten questions and can be applied by experienced searchers with a moderate knowledge of search filter methodology. CONCLUSION: The CADTH CAI provides experienced searchers with a means of selecting the search filter that is most methodologically sound.

Evidence for the effect of omega-3 fatty acids on progression of age-related macular degeneration: a systematic review.

BACKGROUND: As part of a larger systematic review on the effect of omega-3 fatty acids on eye health, the aim of this report was to appraise and synthesize the evidence for the effects of omega-3 fatty acids in slowing down the progression of age-related macular degeneration (AMD) and/or decreasing the rate of progression to advanced forms of AMD. METHODS: A comprehensive search was undertaken in six databases (MEDLINE, PreMEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, CAB Health, and Dissertation Abstracts). RESULTS: Two unique studies, one randomized clinical trial (RCT) and one prospective cohort study, satisfied the eligibility criteria and were included in the review. The RCT reported evidence on the effect of omega-3 fatty acids in slowing down the progression of AMD. The prospective cohort study addressed the question: what is the evidence that omega-3 fatty acids decrease the rate of progression to advanced forms of AMD? CONCLUSIONS: Clinical research on this topic is scarce. Only two studies were eligible to be included in this review. Although one study result indicated efficacy of preventing AMD progression to its advanced form, this result needs to be duplicated and supported by future research.

Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review.

OBJECTIVE: To determine whether the adoption of the CONSORT checklist is associated with improvement in the quality of reporting of randomised controlled trials (RCTs). DATA SOURCES: MEDLINE, EMBASE, Cochrane CENTRAL, and reference lists of included studies and of experts were searched to identify eligible studies published between 1996 and 2005. STUDY SELECTION: Studies were eligible if they (a) compared CONSORT-adopting and non-adopting journals after the publication of CONSORT, (b) compared CONSORT adopters before and after publication of CONSORT, or (c) a combination of (a) and (b). Outcomes examined included reports for any of the 22 items on the CONSORT checklist or overall trial quality. DATA SYNTHESIS: 1128 studies were retrieved, of which 248 were considered possibly relevant. Eight studies were included in the review. CONSORT adopters had significantly better reporting of the method of sequence generation (risk ratio [RR], 1.67; 95% CI, 1.19-2.33), allocation concealment (RR, 1.66; 95% CI, 1.37-2.00) and overall number of CONSORT items than non-adopters (standardised mean difference, 0.83; 95% CI, 0.46-1.19). CONSORT adoption had less effect on reporting of participant flow (RR, 1.14; 95% CI, 0.89-1.46) and blinding of participants (RR, 1.09; 95% CI, 0.84-1.43) or data analysts (RR, 5.44; 95% CI, 0.73-36.87). In studies examining CONSORT-adopting journals before and after the publication of CONSORT, description of the method of sequence generation (RR, 2.78; 95% CI, 1.78-4.33), participant flow (RR, 8.06; 95% CI, 4.10-15.83), and total CONSORT items (standardised mean difference, 3.67 items; 95% CI, 2.09-5.25) were improved after adoption of CONSORT by the journal. CONCLUSIONS: Journal adoption of CONSORT is associated with improved reporting of RCTs.