Frameworks, Dimensions, Definitions of Aspects, and Assessment Methods for the Appraisal of Quality of Health Data for Secondary Use: Comprehensive Overview of Reviews.

BACKGROUND: Health care has not reached the full potential of the secondary use of health data because of-among other issues-concerns about the quality of the data being used. The shift toward digital health has led to an increase in the volume of health data. However, this increase in quantity has not been matched by a proportional improvement in the quality of health data. OBJECTIVE: This review aims to offer a comprehensive overview of the existing frameworks for data quality dimensions and assessment methods for the secondary use of health data. In addition, it aims to consolidate the results into a unified framework. METHODS: A review of reviews was conducted including reviews describing frameworks of data quality dimensions and their assessment methods, specifically from a secondary use perspective. Reviews were excluded if they were not related to the health care ecosystem, lacked relevant information related to our research objective, and were published in languages other than English. RESULTS: A total of 22 reviews were included, comprising 22 frameworks, with 23 different terms for dimensions, and 62 definitions of dimensions. All dimensions were mapped toward the data quality framework of the European Institute for Innovation through Health Data. In total, 8 reviews mentioned 38 different assessment methods, pertaining to 31 definitions of the dimensions. CONCLUSIONS: The findings in this review revealed a lack of consensus in the literature regarding the terminology, definitions, and assessment methods for data quality dimensions. This creates ambiguity and difficulties in developing specific assessment methods. This study goes a step further by assigning all observed definitions to a consolidated framework of 9 data quality dimensions.

Success Factors for Scaling Up the Adoption of Digital Therapeutics Towards the Realization of P5 Medicine.

Introduction: Digital therapeutics (DTx) can be a valuable contribution to the successful scale up of P5 Medicine (personalized, participatory, predictive, preventive, precision medicine) as they offer powerful means of delivering personalization and active patient participation in disease self-management. We investigated how the approval and adoption of DTx within health systems have been approached in five selected European countries and regions, with a view to proposing success factors scaling up their adoption. Methodology: Preliminary research established best countries or region candidates as being Germany, UK, France, Belgium, and the Spanish Region of Catalonia. The research was informed by a literature review, interviews with public bodies and industry, and a multi-stakeholder workshop to validate the findings and fill in existing gaps. Results: To authorize the use of digital technologies, the countries and regions passed legislation and developed policy instruments, appointed bodies to assess and certify the products and formalized mechanisms for permitting reimbursement. While DTx is not a commonly used nomenclature, there are digital health technology types defined that have similar requirements as DTx. Assessment and certification frameworks are usually built around the Medical Device Regulation with additional criteria. Reimbursement considerations often observe reimbursement of therapeutic devices and/or medicines. To be integrated into reimbursement systems, countries require manufacturers to demonstrate clinical value and cost-effectiveness. As there are currently very few DTx approved in practice, there is resistance toward clinical acceptance and organizational change, and change management is highly needed to integrate DTx into healthcare systems. The integration and secondary use of DTx data is not encountered in daily practice. Although some enablers exist, there remain technical and legal barriers. Discussion: DTx strategies should be considered as an integral part of digital health strategies and legislation, and specific DTx pathways with clear and transparent assessment and guidelines that balance regulation and innovation should be defined. To help manufacturers, countries should recommend and list methods that are widely accepted and ensure scientific robustness, aligned to the MDR requirements to support transfer of relevant and comparable data across countries. To facilitate rapid uptake of innovation, countries should add flexibility to the framework by allowing temporary market authorization to enable data collection that can support the clinical and socio-economic evaluation and data gathering phase. Certification should trigger rapid price setting and reimbursement mechanisms, and dynamic ways to adjust price and reimbursement levels in time should be established. Relevant stakeholders should be approached on the potential impacts of DTx through transparent communication and change management strategies should be considered. These findings should be validated with a wider range of stakeholders.

Quality of Hospital Electronic Health Record (EHR) Data Based on the International Consortium for Health Outcomes Measurement (ICHOM) in Heart Failure: Pilot Data Quality Assessment Study.

BACKGROUND: There is increasing recognition that health care providers need to focus attention, and be judged against, the impact they have on the health outcomes experienced by patients. The measurement of health outcomes as a routine part of clinical documentation is probably the only scalable way of collecting outcomes evidence, since secondary data collection is expensive and error-prone. However, there is uncertainty about whether routinely collected clinical data within electronic health record (EHR) systems includes the data most relevant to measuring and comparing outcomes and if those items are collected to a good enough data quality to be relied upon for outcomes assessment, since several studies have pointed out significant issues regarding EHR data availability and quality. OBJECTIVE: In this paper, we first describe a practical approach to data quality assessment of health outcomes, based on a literature review of existing frameworks for quality assessment of health data and multistakeholder consultation. Adopting this approach, we performed a pilot study on a subset of 21 International Consortium for Health Outcomes Measurement (ICHOM) outcomes data items from patients with congestive heart failure. METHODS: All available registries compatible with the diagnosis of heart failure within an EHR data repository of a general hospital (142,345 visits and 12,503 patients) were extracted and mapped to the ICHOM format. We focused our pilot assessment on 5 commonly used data quality dimensions: completeness, correctness, consistency, uniqueness, and temporal stability. RESULTS: We found high scores (>95%) for the consistency, completeness, and uniqueness dimensions. Temporal stability analyses showed some changes over time in the reported use of medication to treat heart failure, as well as in the recording of past medical conditions. Finally, the investigation of data correctness suggested several issues concerning the characterization of missing data values. Many of these issues appear to be introduced while mapping the IMASIS-2 relational database contents to the ICHOM format, as the latter requires a level of detail that is not explicitly available in the coded data of an EHR. CONCLUSIONS: Overall, results of this pilot study revealed good data quality for the subset of heart failure outcomes collected at the Hospital del Mar. Nevertheless, some important data errors were identified that were caused by fundamentally different data collection practices in routine clinical care versus research, for which the ICHOM standard set was originally developed. To truly examine to what extent hospitals today are able to routinely collect the evidence of their success in achieving good health outcomes, future research would benefit from performing more extensive data quality assessments, including all data items from the ICHOM standards set and across multiple hospitals.

Propelling Health Care into the Twenties.

The scope and potential of personalised health care are underappreciated and underrealised, often because of resistance to change. The consequence is that many inadequacies of health care in Europe persist unnecessarily, and many opportunities for improvement are neglected. This article identifies the principal challenges, outlines possible approaches to resolving them, and highlights the benefits that could result from greater adoption of personalised health care. It locates the discussion in the context of European policy, focusing particularly on the most recent and authoritative reviews of health care in the EU Member States, and on the newly acquired spirit of readiness and pragmatism among European officials to embrace change and innovative technologies in a new decade. It highlights the attention now being given by policymakers to incentives, innovation, and investment as levers to improve European citizens' prospects in a rapidly evolving world, and how these distinct and disruptive themes contribute to a renaissance in thinking about delivering optimal health care in Europe. It explores the chances offered to patients by specific initiatives in health domains such as cancer and antimicrobial resistance, and by innovative science, novel therapies, earlier diagnosis tools, and deeper understanding of health promotion and prevention. And it reflects on how health care providers could benefit from a shift towards better primary care and towards deploying health data more effectively, including the use of artificial intelligence, coupled with a move to a smoother organisational/regulatory structure and realigned professional responsibilities. The conclusion is that preparing Europe's health care systems for the inevitable strains of the coming years is both possible and necessary. A more courageous approach to embracing personalised health care could guarantee the sustainability of Europe's health care systems before rising demands and exponential costs overwhelm them - an exercise in future-proofing, in ensuring that they are equipped to withstand whatever lies ahead. A focus on the potential and implementation of personalised care would permit more efficient use of resources and deliver better quality health-preserving care.

Raising the Impact of Real World Evidence.

In an environment in which most regulatory and authoritative health strategy decisions are made on the basis of randomised control trials, real-world evidence (RWE), primarily derived from electronic health records, remains a second-class citizen. Real world evidence is widely taken to include Pragmatic clinical trials and insights derived from the distributed analysis of large volumes of routinely collected health data and registry data (so called big data). This presentation will look at the growing scale and reputation of big health data, the ways in which good governance principles and better quality data are creating reusable data at scale, how platforms and tools are enabling better quality evidence generation, and the perspectives of different stakeholders towards the positioning of RWE in decision making: by regulators, health technology assessment agencies, outcomes benchmarking and value based care. This talk will review how we are presently able to generate trustworthy real world evidence, what we mean by that, and the barriers that remain to trusting it. These remaining barriers will need to be tackled by future health informatics research.

[Value of hospital electronic health records for clinical research: contribution of the European project EHR4CR]. / Intérêt des dossiers de santé électroniques hospitaliers pour la recherche clinique - Le projet européen EHR4CR.

Electronic health records in hospitals contribute to improving the quality of care by enabling better management of clinical information. The databases thus constituted facilitate the exchange of health information with healthcare providers and optimize multidisciplinary coordination for better therapeutic results. The EHR4CR (Electronic Health Records for Clinical Research) European project has developed an innovative pilot platform enabling the reuse of this digital information for clinical research. By enhancing and speeding up clinical research procedures, this innovative approach makes it possible to conduct clinical trials more efficiently, faster, and more economically.

The positive impacts of Real-World Data on the challenges facing the evolution of biopharma.

Demand for healthcare services is unprecedented. Society is struggling to afford the cost. Pricing of biopharmaceutical products is under scrutiny, especially by payers and Health Technology Assessment agencies. As we discuss here, rapidly advancing technologies, such as Real-World Data (RWD), are being utilized to increase understanding of disease. RWD, when captured and analyzed, produces the Real-World Evidence (RWE) that underpins the economic case for innovative medicines. Furthermore, RWD can inform the understanding of disease, help identify new therapeutic intervention points, and improve the efficiency of research and development (R&D), especially clinical trials. Pursuing precompetitive collaborations to define shared requirements for the use of RWD would equip service-providers with the specifications needed to implement cloud-based solutions for RWD acquisition, management and analysis. Only this approach would deliver cost-effective solutions to an industry-wide problem.

Cost-benefit assessment of using electronic health records data for clinical research versus current practices: Contribution of the Electronic Health Records for Clinical Research (EHR4CR) European Project.

INTRODUCTION: The widespread adoption of electronic health records (EHR) provides a new opportunity to improve the efficiency of clinical research. The European EHR4CR (Electronic Health Records for Clinical Research) 4-year project has developed an innovative technological platform to enable the re-use of EHR data for clinical research. The objective of this cost-benefit assessment (CBA) is to assess the value of EHR4CR solutions compared to current practices, from the perspective of sponsors of clinical trials. MATERIALS AND METHODS: A CBA model was developed using an advanced modeling approach. The costs of performing three clinical research scenarios (S) applied to a hypothetical Phase II or III oncology clinical trial workflow (reference case) were estimated under current and EHR4CR conditions, namely protocol feasibility assessment (S1), patient identification for recruitment (S2), and clinical study execution (S3). The potential benefits were calculated considering that the estimated reduction in actual person-time and costs for performing EHR4CR S1, S2, and S3 would accelerate time to market (TTM). Probabilistic sensitivity analyses using Monte Carlo simulations were conducted to manage uncertainty. RESULTS: Should the estimated efficiency gains achieved with the EHR4CR platform translate into faster TTM, the expected benefits for the global pharmaceutical oncology sector were estimated at €161.5m (S1), €45.7m (S2), €204.5m (S1+S2), €1906m (S3), and up to €2121.8m (S1+S2+S3) when the scenarios were used sequentially. CONCLUSIONS: The results suggest that optimizing clinical trial design and execution with the EHR4CR platform would generate substantial added value for pharmaceutical industry, as main sponsors of clinical trials in Europe, and beyond.

Analysis of Requirements for the Medication Profile to Be Used in Clinical Research: Protocol Feasibility Studies and Patient Recruitment.

A "Medication Profile," the information about the medicines a person is using and has used, is a core part of many electronic health record systems and summaries. However, there is little objective research into the data elements that the profile should contain to support the uses it must serve. With the increasing emphasis on secondary uses of electronic health information, as well as supporting the requirements to support direct to patient care, the Medication Profile should also support the requirements from clinical research. However, there is little, if any, description of these available. This paper describes an analysis of a set of study eligibility criteria that was undertaken to investigate which medication-related data elements would be required to support two clinical research use cases: the parameters to query a patient's Medication Profile to assess their suitability for entry into a trial (patient recruitment) and the parameters to query a set of Medication Profiles in a data warehouse to assess whether the eligibility criteria as described would yield a reasonable cohort of patients as potential subjects (protocol feasibility). These medication-related data elements then become information requirements that a Medication Profile should ideally meet, in order to be able to support these two uses in the clinical research domain.

The collection and utilisation of patient ethnicity data in general practices and hospitals in the United Kingdom: a qualitative case study.

BACKGROUND: Although the collection of patient ethnicity data is a requirement of publicly funded healthcare providers in the UK, recording of ethnicity is sub-optimal for reasons that remain poorly understood. AIMS AND OBJECTIVES: We sought to understand enablers and barriers to the collection and utilisation of ethnicity data within electronic health records, how these practices have developed and what benefit this information provides to different stakeholder groups. METHODS: We undertook an in-depth, qualitative case study drawing on interviews and documents obtained from participants working as academics, managers and administrators within the UK. RESULTS: Information regarding patient ethnicity was collected and coded as administrative patient data, and/or in narrative form within clinical records. We identified disparities in the classification of ethnicity, approaches to coding and levels of completeness due to differing local, regional and national policies and processes. Most participants could not identify any clinical value of ethnicity information and many did not know if and when data were shared between services or used to support quality of care and research. CONCLUSIONS: Findings highlighted substantial variations in data classification, and practical challenges in data collection and usage that undermine the integrity of data collected. Future work needs to focus on explaining the uses of these data to frontline clinicians, identifying resources that can support busy professionals to collect standardised data and then, once collected, maximising the utility of these data.

National evaluation of the benefits and risks of greater structuring and coding of the electronic health record: exploratory qualitative investigation.

OBJECTIVE: We aimed to explore stakeholder views, attitudes, needs, and expectations regarding likely benefits and risks resulting from increased structuring and coding of clinical information within electronic health records (EHRs). MATERIALS AND METHODS: Qualitative investigation in primary and secondary care and research settings throughout the UK. Data were derived from interviews, expert discussion groups, observations, and relevant documents. Participants (n=70) included patients, healthcare professionals, health service commissioners, policy makers, managers, administrators, systems developers, researchers, and academics. RESULTS: Four main themes arose from our data: variations in documentation practice; patient care benefits; secondary uses of information; and informing and involving patients. We observed a lack of guidelines, co-ordination, and dissemination of best practice relating to the design and use of information structures. While we identified immediate benefits for direct care and secondary analysis, many healthcare professionals did not see the relevance of structured and/or coded data to clinical practice. The potential for structured information to increase patient understanding of their diagnosis and treatment contrasted with concerns regarding the appropriateness of coded information for patients. CONCLUSIONS: The design and development of EHRs requires the capture of narrative information to reflect patient/clinician communication and computable data for administration and research purposes. Increased structuring and/or coding of EHRs therefore offers both benefits and risks. Documentation standards within clinical guidelines are likely to encourage comprehensive, accurate processing of data. As data structures may impact upon clinician/patient interactions, new models of documentation may be necessary if EHRs are to be read and authored by patients.

Quality metrics for detailed clinical models.

OBJECTIVE: To develop quality metrics for detailed clinical models (DCMs) and test their validity. METHODS: Based on existing quality criteria which did not include formal metrics, we developed quality metrics by applying the ISO/IEC 9126 software quality evaluation model. The face and content validity of the initial quality metrics were assessed by 9 international experts. Content validity was defined as agreement by over 70% of the panelists. For eliciting opinions and achieving consensus of the panelists, a two round Delphi survey was conducted. Valid quality metrics were considered reliable if agreement between two evaluators' assessments of two example DCMs was over 0.60 in terms of the kappa coefficient. After reliability and validity were tested, the final DCM quality metrics were selected. RESULTS: According to the results of the reliability test, the degree of agreement was high (a kappa coefficient of 0.73). Based on the results of the reliability test, 8 quality evaluation domains and 29 quality metrics were finalized as DCM quality metrics. CONCLUSION: Quality metrics were validated by a panel of international DCM experts. Therefore, we expect that the metrics, which constitute essential qualitative and quantitative quality requirements for DCMs, can be used to support rational decision-making by DCM developers and clinical users.

Data resource profile: cardiovascular disease research using linked bespoke studies and electronic health records (CALIBER).

The goal of cardiovascular disease (CVD) research using linked bespoke studies and electronic health records (CALIBER) is to provide evidence to inform health care and public health policy for CVDs across different stages of translation, from discovery, through evaluation in trials to implementation, where linkages to electronic health records provide new scientific opportunities. The initial approach of the CALIBER programme is characterized as follows: (i) Linkages of multiple electronic heath record sources: examples include linkages between the longitudinal primary care data from the Clinical Practice Research Datalink, the national registry of acute coronary syndromes (Myocardial Ischaemia National Audit Project), hospitalization and procedure data from Hospital Episode Statistics and cause-specific mortality and social deprivation data from the Office of National Statistics. Current cohort analyses involve a million people in initially healthy populations and disease registries with ∼10(5) patients. (ii) Linkages of bespoke investigator-led cohort studies (e.g. UK Biobank) to registry data (e.g. Myocardial Ischaemia National Audit Project), providing new means of ascertaining, validating and phenotyping disease. (iii) A common data model in which routine electronic health record data are made research ready, and sharable, by defining and curating with meta-data >300 variables (categorical, continuous, event) on risk factors, CVDs and non-cardiovascular comorbidities. (iv) Transparency: all CALIBER studies have an analytic protocol registered in the public domain, and data are available (safe haven model) for use subject to approvals. For more information, e-mail s.denaxas@ucl.ac.uk.

A review of the empirical evidence of the value of structuring and coding of clinical information within electronic health records for direct patient care.

BACKGROUND: The case has historically been presented that structured and/or coded electronic health records (EHRs) benefit direct patient care, but the evidence base for this is not well documented. METHODS: We searched for evidence of direct patient care value from the use of structured and/or coded information within EHRs. We interrogated nine international databases from 1990 to 2011. Value was defined using the Institute of Medicine's six areas for improvement for healthcare systems: effectiveness, safety, patient-centredness, timeliness, efficiency and equitability. We included studies satisfying the Cochrane Effective Practice and Organisation of Care (EPOC) group criteria. RESULTS: Of 5016 potentially eligible papers, 13 studies satisfied our criteria: 10 focused on effectiveness, with eight demonstrating potential for improved proxy and actual clinical outcomes if a structured and/or coded EHR was combined with alerting or advisory systems in a focused clinical domain. Three studies demonstrated improvement in safety outcomes. No studies were found reporting value in relation to patient-centredness, timeliness, efficiency or equitability. CONCLUSIONS: We conclude that, to date, there has been patchy effort to investigate empirically the value from structuring and coding EHRs for direct patient care. Future investments in structuring and coding of EHRs should be informed by robust evidence as to the clinical scenarios in which patient care benefits may be realised.