Use of real-world data and evidence for medical devices: a qualitative study of key informant interviews.

INTRODUCTION: Health Canada is committed to the modernization of the use of real-world data (RWD) and evidence (RWE) to support regulatory decisions. As such, telephone interviews with stakeholders, including government decision makers, health technology assessment (HTA) producers, industry, and patients, to understand their experiences with and perspectives on how to enhance RWE use for medical devices were performed. METHODS: Thirty-four semi-structured telephone interviews with forty key informants were conducted. Transcripts were reviewed independently by one individual to identify, define, and categorize key concepts and were verified by a second reviewer. KEY FINDINGS: There are expectations for Health Canada to provide a framework and guidance on RWE use, identify relevant outcomes for data collection and criteria for data quality, conduct post-market surveillance more systematically, and partner with HTA organizations to develop methods for RWE generation. Stakeholders interviewed support the RWE use for regulatory decisions and HTA recommendations. Moreover, robust scientific methods for RWE generation will be critical to ensure that relevant questions are asked and rigorous statistical analyses are done to answer them. Patients are likely to consent to share their anonymized or de-identified medical information for nonprofit purposes. CONCLUSIONS: Key concepts from the interviews centered on the current and future RWE use for medical devices, considerations for the organizational, medical, scientific, and legal aspects and privacy issues of RWD collection or RWE generation, and options to implement the use of RWD and RWE. Our study findings will help inform the development of an RWE framework for regulatory decisions and HTA recommendations.

Disinvestment Activities and Candidates in the Health Technology Assessment Community: An Online Survey.

OBJECTIVES: As healthcare decision makers continue to face challenges in health services delivery to their patients, disinvestment programs are being established for a sustainable healthcare system. This study aimed to collect data and information by means of a survey of disinvestment candidates and ongoing disinvestment projects in the health technology assessment (HTA) community. METHODS: An online survey was conducted to collect information on disinvestment candidates and activities from members of the Health Technology Assessment International Disinvestment & Early Awareness Interest Group, the EuroScan International Network and International Network of Agencies for Health Technology Assessment. RESULTS: Among the 362 invitees, twenty-four unique responses were received, and almost 70 percent were involved in disinvestment initiatives. The disinvestment candidates identified represented a range of health technologies. Evidence or signaling of clinical ineffectiveness or inappropriate use typically led to the nomination of disinvestment candidates. Health technology assessments and reassessments were usually conducted to evaluate the technology in question, and decisions usually led to the limited use of the technology. Barriers to disinvestment decisions included the strength of interest and advocacy groups, insufficient data for assessments, a systematic decision process and political challenges, while obstacles to their implementation were clinicians' reluctance and insufficient funding and incentives. CONCLUSIONS: The survey results suggested that disinvestment activities are occurring in the HTA community, especially in the public sector. Future research can further investigate the processes and methods used to reach and implement disinvestment decisions from our survey respondents and explore to form closer ties between the HTA and clinical communities.

ENVIRONMENTAL IMPACT ASSESSMENT OF A HEALTH TECHNOLOGY: A SCOPING REVIEW.

INTRODUCTION: The Health Technology Expert Review Panel is an advisory body to Canadian Agency for Drugs and Technologies in Health (CADTH) that develops recommendations on health technology assessments (HTAs) for nondrug health technologies using a deliberative framework. The framework spans several domains, including the environmental impact of the health technology(ies). Our research objective was to identify articles on frameworks, methods or case studies on the environmental impact assessment of health technologies. METHODS: A literature search in major databases and a focused gray literature search were conducted. The main search concepts were HTA and environmental impact/sustainability. Eligible articles were those that described a conceptual framework or methods used to conduct an environmental assessment of health technologies, and case studies on the application of an environmental assessment. RESULTS: From the 1,710 citations identified, thirteen publications were included. Two articles presented a framework to incorporate environmental assessment in HTAs. Other approaches described weight of evidence practices and comprehensive and integrated environmental impact assessments. Central themes derived include transparency and repeatability, integration of components in a framework or of evidence into a single outcome, data availability to ensure the accuracy of findings, and familiarity with the approach used. CONCLUSIONS: Each framework and methods presented have different foci related to the ecosystem, health economics, or engineering practices. Their descriptions suggested transparency, repeatability, and the integration of components or of evidence into a single outcome as their main strengths. Our review is an initial step of a larger initiative by CADTH to develop the methods and processes to address the environmental impact question in an HTA.

HEALTH TECHNOLOGY ASSESSMENT METHODS GUIDELINES FOR MEDICAL DEVICES: HOW CAN WE ADDRESS THE GAPS? THE INTERNATIONAL FEDERATION OF MEDICAL AND BIOLOGICAL ENGINEERING PERSPECTIVE.

OBJECTIVES: Current health technology assessment (HTA) methods guidelines for medical devices may benefit from contributions by biomedical and clinical engineers. Our study aims to: (i) review and identify gaps in the current HTA guidelines on medical devices, (ii) propose recommendations to optimize the impact of HTA for medical devices, and (iii) reach a consensus among biomedical engineers on these recommendations. METHODS: A gray literature search of HTA agency Web sites for assessment methods guidelines on devices was conducted. The International Federation of Medical and Biological Engineers (IFMBE) then convened a structured focus group, with experts from different fields, to identify potential gaps in the current HTA guidelines, and to develop recommendations to fill these perceived gaps. The thirty recommendations generated from the focus group were circulated in a Delphi survey to eighty-five biomedical and clinical engineers. RESULTS: Thirty-two panelists, from seventeen countries, participated in the Delphi survey. The responses showed a strong agreement on twenty-seven of thirty recommendations. Some uncertainties remain about the methods to accurately assess the effectiveness and safety, and interoperability of a medical device with other devices or within the clinical setting. CONCLUSIONS: As medical devices differ from drug therapies, current HTA methods may not accurately reflect the conclusions of their assessment. Recommendations informed by the focus group discussions and Delphi survey responses aimed to address the perceived gaps, and to provide a more integrated approach in medical device assessments in combining engineering with other perspectives, such as clinical, economic, patient, human factors, ethical, and environmental.

Engaging the Canadian public on reimbursement decision-making for drugs for rare diseases: a national online survey.

BACKGROUND: Funding of drugs for rare diseases (DRDs) requires decisions that balance fairness for all individuals within the healthcare system with compassion for affected individuals. Our study objective was to conduct a national online survey to determine the Canadian public's perspective, including regional variations, associated with DRD decision-making. METHODS: The survey collected responses from 1631 Canadians. Respondents were asked to rank at least three and up to five DRD decision-making priorities, out of a total of eight priorities presented. They were also asked to compare and rate their agreement level on a 5-point Likert scale with four funding scenarios described. The frequency of each priority, independent of where it was ranked in relation to the other priorities, was calculated. Regression analyses were conducted to measure the association between respondents' demographics and selected priorities with their agreement level for each funding scenario. RESULTS: Among the survey respondents, Improved Quality of Life and Effective Health Care were most frequently selected as top priorities. Also, 79.2% of respondents agreed with equal access to DRDs across Canada, and 73.0% agreed with DRD funding if additional expenses are justified in the DRD's cost-effectiveness. Approximately half agreed to pay for DRDs independent of their effectiveness. There were no geographic differences in priorities. Selecting Effective Health Care in the top priorities was positively associated with both prioritizing other programs over programs for rare diseases and DRD funding only if deemed as cost-effective. Respondents, who selected National Access as one of the top priorities, were less likely to agree to fund DRDs only if deemed as cost-effective and were more likely to agree with the scenario to provide national access to DRDs. CONCLUSIONS: The survey results suggest the level of public support for funding decisions and programs that incorporate assessment of the effectiveness of drugs for improving quality of life, and to promote similar access across Canada. The responses anticipate public responses to different policy scenarios and the priorities that underlie them. Decision-makers may find it useful to consider whether and how to incorporate these results into policy decisions and their justification to citizens and patients.

EVALUATION OF PATIENT AND PUBLIC INVOLVEMENT INITIATIVES IN HEALTH TECHNOLOGY ASSESSMENT: A SURVEY OF INTERNATIONAL AGENCIES.

OBJECTIVES: Although there is increased awareness of patient and public involvement (PPI) among health technology assessment (HTA) organizations, evaluations of PPI initiatives are relatively scarce. Our objective as members of Health Technology Assessment International's (HTAi's) Patient and Citizen Involvement Group (PCIG) was to advance understanding of the range of evaluation strategies adopted by HTA organizations and their potential usefulness. METHODS: In March 2016, a survey was sent to fifty-four HTA organizations through the International Network of Agencies for Health Technology Assessment (INAHTA) and contacts of members of HTAi's PCIG. Respondents were asked about their organizational structure; how patients and members of the public are involved; whether and how PPI initiatives have been evaluated, and, if so, which facilitators and challenges to evaluation were found and how results were used and disseminated. RESULTS: Fifteen (n = 15) programs from twelve countries responded (response rate 27.8 percent) that involved patients (14/15) and members of the public (10/15) in HTA activities. Seven programs evaluated their PPI activities, including participant satisfaction (5/7), process (5/7) and impact evaluations (4/7). Evaluation results were used to improve PPI activities, identify education and training needs, and direct strategic priorities. Facilitators and challenges revolved around the need for stakeholder buy-in, sufficient resources, senior leadership, and including patients in evaluations. CONCLUSIONS: A small but diverse set of HTA organizations evaluate their PPI activities using a range of strategies that reflect the range of rationales and approaches to PPI in HTA. It will be important for HTA organizations to draw on evaluation theories and methods.

Health Technology Assessment for Molecular Diagnostics: Practices, Challenges, and Recommendations from the Medical Devices and Diagnostics Special Interest Group.

BACKGROUND: Health technology assessments (HTAs) are increasingly used to inform coverage, access, and utilization of medical technologies including molecular diagnostics (MDx). Although MDx are used to screen patients and inform disease management and treatment decisions, there is no uniform approach to their evaluation by HTA organizations. OBJECTIVES: The International Society for Pharmacoeconomics and Outcomes Research Devices and Diagnostics Special Interest Group reviewed diagnostic-specific HTA programs and identified elements representing common and best practices. METHODS: MDx-specific HTA programs in Europe, Australia, and North America were characterized by methodology, evaluation framework, and impact. Published MDx HTAs were reviewed, and five representative case studies of test evaluations were developed: United Kingdom (National Institute for Health and Care Excellence's Diagnostics Assessment Programme, epidermal growth factor receptor tyrosine kinase mutation), United States (Palmetto's Molecular Diagnostic Services Program, OncotypeDx prostate cancer test), Germany (Institute for Quality and Efficiency in Healthcare, human papillomavirus testing), Australia (Medical Services Advisory Committee, anaplastic lymphoma kinase testing for non-small cell lung cancer), and Canada (Canadian Agency for Drugs and Technologies in Health, Rapid Response: Non-invasive Prenatal Testing). RESULTS: Overall, the few HTA programs that have MDx-specific methods do not provide clear parameters of acceptability related to clinical and analytic performance, clinical utility, and economic impact. The case studies highlight similarities and differences in evaluation approaches across HTAs in the performance metrics used (analytic and clinical validity, clinical utility), evidence requirements, and how value is measured. Not all HTAs are directly linked to reimbursement outcomes. CONCLUSIONS: To improve MDx HTAs, organizations should provide greater transparency, better communication and collaboration between industry and HTA stakeholders, clearer links between HTA and funding decisions, explicit recognition of and rationale for differential approaches to laboratory-developed versus regulatory-approved test, and clear evidence requirements.

LOCAL HEALTH TECHNOLOGY ASSESSMENT IN CANADA: CURRENT STATE AND NEXT STEPS.

OBJECTIVES: Canada has witnessed expansion of the health technology assessment (HTA) infrastructure in the last 25 years. Local HTA entities at the hospital or regional level are emerging to assist decision makers in the acquisition, implementation, maintenance, and disinvestment of healthcare technologies. There is a need to facilitate collaboration and exchange of expertise and knowledge between these entities regarding the role of local HTA in Canada. METHODS: In November 2013, the pan-Canadian Collaborative hosted a symposium, Hospital/Regional HTA: Local Evidence-based Decisions for Health Care Sustainability, bringing together over 60 HTA producers, researchers, stakeholders, and manufacturers involved in local HTA across Canada. The objective was to showcase the diversity of local HTA in Canada, while highlighting common gaps to be addressed. RESULTS: The Symposium focused on current practices in local HTA in Canada to support informed decision making, and opportunities for information sharing and provide equal access to timely evidence-based information to decision makers. The main themes included assessment of evidence for local HTA, contextualization, stakeholder engagement in local HTA, knowledge translation and impact of recommendations, and challenges and opportunities for local HTA. CONCLUSIONS: Local HTA in Canada complements HTAs conducted at the provincial and federal levels to improve the efficient and effective health service delivery in institutions or regions faced with limited resources. Some challenges faced by local HTA producers to influence hospital policies and clinical practice involve the engagement of healthcare professionals and potential lack of training and support necessary for the introduction of a new technology.

How can we improve the recognition, reporting and resolution of medical device-related incidents in hospitals? A qualitative study of physicians and registered nurses.

BACKGROUND: To explore factors that influence and to identify initiatives to improve the recognition, reporting and resolution of device-related incidents. METHODS: Semi-structured telephone interviews with 16 health professionals in two tertiary care hospitals were conducted. Purposive sampling was used to identify appropriate study participants. Transcribed interviews were read independently by one individual to identify, define and organize themes and verified by another reviewer. RESULTS: Themes related to incident recognition were the hospital staff's knowledge and professional experience, medical device performance and clinical manifestations of patients, while incident reporting was influenced by error severity, personal attitudes of clinicians, feedback received on the error reported. Physicians often discontinued using medical devices if they malfunctioned. Education and training and the implementation of registries were discussed as important initiatives to improve medical device surveillance in clinical practice. CONCLUSIONS: Results from the telephone interviews suggest that multiple factors that influence participation in medical device surveillance activities are consistent with results for medical errors as reported in previous studies. The study results helped to propose a conceptual framework for a medical device surveillance system in a hospital context that would enhance patient safety and health care delivery.

Rapid review: an emerging approach to evidence synthesis in health technology assessment.

BACKGROUND: Increasingly, healthcare decision makers demand quality evidence in a short timeframe to support urgent and emergent decisions related to procurement, clinical practice, and policy. Health technology assessment (HTA) producers are responding by developing innovative approaches to evidence synthesis that can be executed more quickly than traditional systematic review. These approaches, and the broader implications they bring to bear on health decision making and policy development, however, are generally neither well-understood nor well-described. This study intends to contribute to an emerging literature around methodological approaches to rapid review in HTA by outlining those developed and implemented by the Canadian Agency for Drugs and Technologies in Health (CADTH). METHODS: Since 2005, CADTH has developed and implemented a rapid review approach that synthesizes evidence to support informed healthcare decisions and policy. Rapid Response reports are tailored to the identified needs of Canadian health decision makers, representing a range of options with regard to depth, breadth, and time-to-delivery. RESULTS: Preliminary observations indicate that CADTH's approach to rapid evidence review is generally well-received by Canadian health decision makers; real-world case studies provide pragmatic examples of how health decision makers have used Rapid Response reports to support evidence-informed health decisions across Canada. CONCLUSIONS: Rapid review is becoming an increasingly important approach to evidence synthesis, both within and external to the field of HTA. Transparent reporting of the methods used to develop rapid review products will be critical to the assessment of their relevance, utility and effects in a range of contexts.

Case studies that illustrate disinvestment and resource allocation decision-making processes in health care: a systematic review.

OBJECTIVE: Technological change accounts for approximately 25 percent of health expenditure growth. To date, limited research has been published on case studies of disinvestment and resource allocation decision making in clinical practice. Our research objective is to systematically review and catalogue the application of frameworks and tools for disinvestment and resource allocation decision making in health care. METHODS: An electronic literature search was executed for studies on disinvestment, obsolete and ineffective technologies, and priority healthcare setting, published from January 1990 until January 2012. Databases searched were MEDLINE, MEDLINE In-Process and Other Non-Indexed Citations, Embase, The Cochrane Library, PubMed, and HEED. RESULTS: Fourteen case studies on the application of frameworks and tools for disinvestment and resource allocation decisions were included. Most studies described the application of program budgeting and marginal analysis (PBMA), and two reports used health technology assessment (HTA) methods for coverage decisions in a national fee-for-service structure. Numerous healthcare technologies and services were covered across the studies. We describe the multiple criteria considered for decision making, and the strengths and limitations of these frameworks and tools are highlighted. CONCLUSIONS: Disinvestment and resource allocation decisions require evidence to ensure their transparency and objectivity. PBMA was used to assess resource allocation of health services and technologies in a fixed budget jurisdiction, while HTA reviews focused on specific technologies, principally in fee-for-service structures. Future research can review the data requirements and explore opportunities to increase the quantity of available evidence for disinvestment and resource allocation decisions.

The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies.

OBJECTIVES: The English language is generally perceived to be the universal language of science. However, the exclusive reliance on English-language studies may not represent all of the evidence. Excluding languages other than English (LOE) may introduce a language bias and lead to erroneous conclusions. STUDY DESIGN AND SETTING: We conducted a comprehensive literature search using bibliographic databases and grey literature sources. Studies were eligible for inclusion if they measured the effect of excluding randomized controlled trials (RCTs) reported in LOE from systematic review-based meta-analyses (SR/MA) for one or more outcomes. RESULTS: None of the included studies found major differences between summary treatment effects in English-language restricted meta-analyses and LOE-inclusive meta-analyses. Findings differed about the methodological and reporting quality of trials reported in LOE. The precision of pooled estimates improved with the inclusion of LOE trials. CONCLUSIONS: Overall, we found no evidence of a systematic bias from the use of language restrictions in systematic review-based meta-analyses in conventional medicine. Further research is needed to determine the impact of language restriction on systematic reviews in particular fields of medicine.

Clinical effectiveness of rapid tests for methicillin resistant Staphylococcus aureus (MRSA) in hospitalized patients: a systematic review.

BACKGROUND: Methicillin resistant Staphylococcus aureus (MRSA) are often resistant to multiple classes of antibiotics. The research objectives of this systematic review were to evaluate the clinical effectiveness of polymerase chain reaction (PCR) versus chromogenic agar for MRSA screening, and PCR versus no screening for several clinical outcomes, including MRSA colonization and infection rates. METHODS: An electronic literature search was conducted on studies evaluating polymerase chain reaction techniques and methicillin (also spelled meticillin) resistant Staphylococcus aureus that were published from 1993 onwards using Medline, Medline In-Process & Other Non-Indexed Citations, BIOSIS Previews, and EMBASE. Due to the presence of heterogeneity in the selected studies, the clinical findings of individual studies were described. RESULTS: Nine studies that compared screening for MRSA using PCR versus screening using chromogenic agar in a hospital setting, and two studies that compared screening using PCR with no or targeted screening were identified. Some studies found lower MRSA colonization and acquisition, infection, and transmission rates in screening with PCR versus screening with chromogenic agar, and the turnaround time for screening test results was lower for PCR. One study reported a lower number of unnecessary isolation days with screening using PCR versus screening with chromogenic agar, but the proportion of patients isolated was similar between both groups. The turnaround time for test results and number of isolation days were lower for PCR versus chromogenic agar for MRSA screening. CONCLUSIONS: The use of PCR for MRSA screening demonstrated a lower turnaround time and number of isolation days compared with chromogenic agar. Given the mixed quality and number of studies (11 studies), gaps remain in the published literature and the evidence remains insufficient. In addition to screening, factors such as the number of contacts between healthcare workers and patients, number of patients attended by one healthcare worker per day, probability of colonization among healthcare workers, and MRSA status of hospital shared equipment and hospital environment must be considered to control the transmission of MRSA in a hospital setting.

Public health measures to reduce the risk of SARS-CoV-2 transmission in Canada during the early days of the COVID-19 pandemic: a scoping review.

OBJECTIVE: The main objectives of this study were to synthesise and compare pandemic preparedness strategies issued by the federal and provincial/territorial (P/T) governments in Canada and to assess whether COVID-19 public health (PH) measures were tailored towards priority populations, as defined by relevant social determinants of health. METHODS: This scoping review searched federal and P/T websites on daily COVID-19 pandemic preparedness strategies between 30 January and 30 April 2020. The PROGRESS-Plus equity-lens framework was used to define priority populations. All definitions, policies and guidelines of PH strategies implemented by the federal and P/T governments to reduce risk of SARS-CoV-2 transmission were included. PH measures were classified using a modified Public Health Agency of Canada Framework for Canadian Pandemic Influenza Preparedness. RESULTS: A total of 722 COVID-19 PH measures were issued during the study period. Of these, home quarantine (voluntary) (n=13.0%; 94/722) and retail/commerce restrictions (10.9%; n=79/722) were the most common measures introduced. Many of the PH orders, including physical distancing, cancellation of mass gatherings, school closures or retail/commerce restrictions began to be introduced after 11 March 2020. Lifting of some of the PH orders in phases to reopen the economy began in April 2020 (6.5%; n=47/722). The majority (68%, n=491/722) of COVID-19 PH announcements were deemed mandatory, while 32% (n=231/722) were recommendations. Several PH measures (28.0%, n=202/722) targeted a variety of groups at risk of socially produced health inequalities, such as age, religion, occupation and migration status. CONCLUSIONS: Most PH measures centred on limiting contact between people who were not from the same household. PH measures were evolutionary in nature, reflecting new evidence that emerged throughout the pandemic. Although ~30% of all implemented COVID-19 PH measures were tailored towards priority groups, there were still unintended consequences on these populations.

Home telehealth for chronic disease management: a systematic review and an analysis of economic evaluations.

OBJECTIVES: The research objectives were two-fold: first, to systematically review the literature on the cost-effectiveness of home telehealth for chronic diseases, and second to develop a framework for the conduct of economic evaluation of home telehealth projects for patients with chronic diseases. METHODS: A comprehensive literature search identified twenty-two studies (n = 4,871 patients) on home telehealth for chronic diseases published between 1998 and 2008. Studies were reviewed in terms of their methodological quality and their conclusions. RESULTS: Home telehealth was found to be cost saving from the healthcare system and insurance provider perspectives in all but two studies, but the quality of the studies was generally low. An evaluative framework was developed which provides a basis to improve the quality of future studies to facilitate improved healthcare decision making, and an application of the framework is illustrated using data from an existing program evaluation of a home telehealth program. CONCLUSIONS: Current evidence suggests that home telehealth has the potential to reduce costs, but its impact from a societal perspective remains uncertain until higher quality studies become available.

Economic analysis of reprocessing single-use medical devices: a systematic literature review.

BACKGROUND: Although an increasing number of medical devices are labeled "for single use only," cleaning and reuse of single-use medical devices continues, because of the economic incentive. We conducted a survey of the economic literature to obtain the current evidence available and to assess the costs and benefits of reusing single-use medical devices. METHODS: A comprehensive literature search was carried out to identify articles that compared single use and reuse of single-use medical devices and that met specific scientific criteria, including evaluation of economic outcomes. Each selected article was independently reviewed by 2 reviewers to extract cost and clinical outcome data and to assess the quality of the study. RESULTS: Nine published articles met the selection criteria. The savings were about 49% of the direct cost. These savings would be offset by adverse-event costs, but none were detected. However, quality of the studies was generally poor. CONCLUSIONS: There is little available evidence of quality in the published literature to assess the practice of reuse of single-use medical devices. Moreover, data on clinical outcomes are missing and, where available, cannot be attributed specifically to the reuse of single-use medical devices.

Reprocessing and reuse of single-use medical devices: a national survey of Canadian acute-care hospitals.

A national survey investigated the current practices of reprocessing and reusing single-use medical devices in Canadian acute-care hospitals. Our response rate was 72% (413 responses), and 28% of hospitals reprocess single-use devices. The results showed that Canadian hospital practices have not changed much in the past decade.

A Review and Comparative Analysis of Information Targeted to the General Public on the Websites of Breast Screening Programs in Canada.

Organized breast screening programs in Canada recommend that women, usually 50-74 years of age, are screened regularly with mammography to reduce their risk of breast cancer death. There is increasing evidence that estimates of mortality reduction are overestimated and harms under-reported. This article will report on a review of the websites of 12 breast screening programs in Canada. The primary goal is to determine what information is provided to enable women to make an informed decision about mammography and whether choice is emphasized. All publicly available English language information was extracted from the 12 websites by two independent reviewers, using a data extraction sheet. Information extracted included eligible age, screening interval and potential benefits and harms. This review is relevant to policy makers and breast screening program staff so they can determine what additional or alternative information is required on their websites to enable women to make informed decisions.

A third of systematic reviews changed or did not specify the primary outcome: a PROSPERO register study.

OBJECTIVES: To examine outcome reporting bias of systematic reviews registered in PROSPERO. STUDY DESIGN AND SETTING: Retrospective cohort study. The primary outcomes from systematic review publications were compared with those reported in the corresponding PROSPERO records; discrepancies in the primary outcomes were assessed as upgrades, additions, omissions, or downgrades. Relative risks (RRs) and 95% confidence intervals (CI) were calculated to determine the likelihood of having a change in primary outcome when the meta-analysis result was favorable and statistically significant. RESULTS: Ninety-six systematic reviews were published. A discrepancy in the primary outcome occurred in 32% of the included reviews and 39% of the reviews did not explicitly specify a primary outcome(s); 6% of the primary outcomes were omitted. There was no significant increased risk of adding/upgrading (RR, 2.14; 95% CI: 0.53, 8.63) or decreased risk of downgrading (RR, 0.76; 95% CI: 0.27, 2.17) an outcome when the meta-analysis result was favorable and statistically significant. As well, there was no significant increased risk of adding/upgrading (RR, 0.89; 95% CI: 0.31, 2.53) or decreased risk of downgrading (RR, 0.56; 95% CI: 0.29, 1.08) an outcome when the conclusion was positive. CONCLUSIONS: We recommend review authors carefully consider primary outcome selection, and journals are encouraged to focus acceptance on registered systematic reviews.

Factors that influence the recognition, reporting and resolution of incidents related to medical devices and other healthcare technologies: a systematic review.

BACKGROUND: Medical devices have improved the treatment of many medical conditions. Despite their benefit, the use of devices can lead to unintended incidents, potentially resulting in unnecessary harm, injury or complications to the patient, a complaint, loss or damage. Devices are used in hospitals on a routine basis. Research to date, however, has been primarily limited to describing incidents rates, so the optimal design of a hospital-based surveillance system remains unclear. Our research objectives were twofold: i) to explore factors that influence device-related incident recognition, reporting and resolution and ii) to investigate interventions or strategies to improve the recognition, reporting and resolution of medical device-related incidents. METHODS: We searched the bibliographic databases: MEDLINE, Embase, the Cochrane Central Register of Controlled Trials and PsycINFO database. Grey literature (literature that is not commercially available) was searched for studies on factors that influence incident recognition, reporting and resolution published and interventions or strategies for their improvement from 2003 to 2014. Although we focused on medical devices, other health technologies were eligible for inclusion. RESULTS: Thirty studies were included in our systematic review, but most studies were concentrated on other health technologies. The study findings indicate that fear of punishment, uncertainty of what should be reported and how incident reports will be used and time constraints to incident reporting are common barriers to incident recognition and reporting. Relevant studies on the resolution of medical errors were not found. Strategies to improve error reporting include the use of an electronic error reporting system, increased training and feedback to frontline clinicians about the reported error. CONCLUSIONS: The available evidence on factors influencing medical device-related incident recognition, reporting and resolution by healthcare professionals can inform data collection and analysis in future studies. Since evidence gaps on medical device-related incidents exist, telephone interviews with frontline clinicians will be conducted to solicit information about their experiences with medical devices and suggested strategies for device surveillance improvement in a hospital context. Further research also should investigate the impact of human, system, organizational and education factors on the development and implementation of local medical device surveillance systems.