The experiences of 33 national COVID-19 dashboard teams during the first year of the pandemic in the World Health Organization European Region: A qualitative study.

Background: Governments across the World Health Organization (WHO) European Region have prioritised dashboards for reporting COVID-19 data. The ubiquitous use of dashboards for public reporting is a novel phenomenon. Objective: This study explores the development of COVID-19 dashboards during the first year of the pandemic and identifies common barriers, enablers and lessons from the experiences of teams responsible for their development. Methods: We applied multiple methods to identify and recruit COVID-19 dashboard teams, using a purposive, quota sampling approach. Semi-structured group interviews were conducted from April to June 2021. Using elaborative coding and thematic analysis, we derived descriptive and explanatory themes from the interview data. A validation workshop was held with study participants in June 2021. Results: Eighty informants participated, representing 33 national COVID-19 dashboard teams across the WHO European Region. Most dashboards were launched swiftly during the first months of the pandemic, February to May 2020. The urgency, intense workload, limited human resources, data and privacy constraints and public scrutiny were common challenges in the initial development stage. Themes related to barriers or enablers were identified, pertaining to the pre-pandemic context, pandemic itself, people and processes and software, data and users. Lessons emerged around the themes of simplicity, trust, partnership, software and data and change. Conclusions: COVID-19 dashboards were developed in a learning-by-doing approach. The experiences of teams reveal that initial underpreparedness was offset by high-level political endorsement, the professionalism of teams, accelerated data improvements and immediate support with commercial software solutions. To leverage the full potential of dashboards for health data reporting, investments are needed at the team, national and pan-European levels.

Features Constituting Actionable COVID-19 Dashboards: Descriptive Assessment and Expert Appraisal of 158 Public Web-Based COVID-19 Dashboards.

BACKGROUND: Since the outbreak of COVID-19, the development of dashboards as dynamic, visual tools for communicating COVID-19 data has surged worldwide. Dashboards can inform decision-making and support behavior change. To do so, they must be actionable. The features that constitute an actionable dashboard in the context of the COVID-19 pandemic have not been rigorously assessed. OBJECTIVE: The aim of this study is to explore the characteristics of public web-based COVID-19 dashboards by assessing their purpose and users ("why"), content and data ("what"), and analyses and displays ("how" they communicate COVID-19 data), and ultimately to appraise the common features of highly actionable dashboards. METHODS: We conducted a descriptive assessment and scoring using nominal group technique with an international panel of experts (n=17) on a global sample of COVID-19 dashboards in July 2020. The sequence of steps included multimethod sampling of dashboards; development and piloting of an assessment tool; data extraction and an initial round of actionability scoring; a workshop based on a preliminary analysis of the results; and reconsideration of actionability scores followed by joint determination of common features of highly actionable dashboards. We used descriptive statistics and thematic analysis to explore the findings by research question. RESULTS: A total of 158 dashboards from 53 countries were assessed. Dashboards were predominately developed by government authorities (100/158, 63.0%) and were national (93/158, 58.9%) in scope. We found that only 20 of the 158 dashboards (12.7%) stated both their primary purpose and intended audience. Nearly all dashboards reported epidemiological indicators (155/158, 98.1%), followed by health system management indicators (85/158, 53.8%), whereas indicators on social and economic impact and behavioral insights were the least reported (7/158, 4.4% and 2/158, 1.3%, respectively). Approximately a quarter of the dashboards (39/158, 24.7%) did not report their data sources. The dashboards predominately reported time trends and disaggregated data by two geographic levels and by age and sex. The dashboards used an average of 2.2 types of displays (SD 0.86); these were mostly graphs and maps, followed by tables. To support data interpretation, color-coding was common (93/158, 89.4%), although only one-fifth of the dashboards (31/158, 19.6%) included text explaining the quality and meaning of the data. In total, 20/158 dashboards (12.7%) were appraised as highly actionable, and seven common features were identified between them. Actionable COVID-19 dashboards (1) know their audience and information needs; (2) manage the type, volume, and flow of displayed information; (3) report data sources and methods clearly; (4) link time trends to policy decisions; (5) provide data that are "close to home"; (6) break down the population into relevant subgroups; and (7) use storytelling and visual cues. CONCLUSIONS: COVID-19 dashboards are diverse in the why, what, and how by which they communicate insights on the pandemic and support data-driven decision-making. To leverage their full potential, dashboard developers should consider adopting the seven actionability features identified.

An expert-based mapping of healthcare system strategies to support rational drug prescribing in primary care across 13 European countries.

BACKGROUND: Irrational prescribing has received increasing attention among policy-makers to improve drug safety and effectiveness while avoiding economic waste. The policies intended to rationalise prescribing have been grouped by WHO under a taxonomy, classifying them into two types of strategies - (1) targeted approaches (micro level) and (2) system-oriented approaches (macro level). The extent to which countries implement strategies and the existing types is currently unknown. This paper explores the following research question via expert opinions: to what extent have European countries implemented strategies to support rational prescribing (targeted and system oriented) and what are the types implemented? METHODS: We assessed the available information on policies intended to promote rational prescribing. We used the WHO taxonomy to explore our research question as the basis for a standardised questionnaire. The data were collected between August 2018 and April 2019. The questionnaire consisted of questions that solicited the opinion of experts on the implementation of prescribing control mechanisms in primary care in their respective countries. Experts were identified through the literature and relevant networks. The questionnaire was sent to 17 identified country experts from 17 different countries; 15 responded and 13 were used in our analysis. Answers were validated through follow-up correspondence, interviews and presentation at an OECD meeting. RESULTS: Expert-reported data shows that all 13 countries included in our study have several mechanisms in place for enhancing rational prescribing in primary care. All approaches were reported to have been implemented in at least two countries. We identified two groups of countries, namely a small group of countries (n = 3) with fewer mechanisms in place and a larger group of countries (n = 10) with a large number of strategies with accompanying instruments at both the micro and macro levels. CONCLUSIONS: The data reported by the experts suggests that all 13 countries included in our study have several mechanisms in place for enhancing rational prescribing in primary care on both the micro and macro levels. With respect to the extent of mechanisms being in place, two groups of countries were identified. This initial mapping of strategies forms a basis for more in-depth research to be able to assess the impact of bundles of strategies on system and targeted level on rational drug prescribing in primary care in Europe.

EQ-5D studies in nervous system diseases in eight Central and East European countries: a systematic literature review.

BACKROUND: Guidelines for economic analyses of health care technologies require local input data for reimbursement decisions in the countries of Central and Eastern Europe (CEE). The aim of this study was to systematically review and analyse the available empirical studies using the EQ-5D instrument as a measure of the health-related quality of life (HRQoL) in patients with neurological diseases. METHODS: A systematic literature search was performed up to 1st April 2018 to identify relevant studies in eight selected CEE countries. Original articles reporting on studies of neurological diseases using the EQ-5D instrument were analysed. RESULTS: Thirty-six articles, describing the results of 38 samples of patients and a total of 13,005 patients were included in the review. Most studies were from Hungary (44.4%) and none from Romania or Slovakia. EQ-5D utility scores were reported in 33 (91.7%) articles. In multiple sclerosis (MS) being the most represented disease, the average utility scores ranged from 0.49 in Austria to 0.80 in Poland with a weighted average of 0.69. EQ VAS scores for MS ranged from 39 in Czech Republic to 72.0 in Poland, with weighted average of 59.1. MS patients, together with epilepsy and essential tremor patients, estimated their HRQoL among the highest. CONCLUSIONS: EQ-5D research activity in neurology has been increasing through the years in studied CEE countries. There are clinical areas with the significant social burden, such as a migraine or meningitis, that are completely lacking data, other areas, such as stroke or epilepsy, with very scarce data.

Cost-of-illness studies in nine Central and Eastern European countries.

BACKGROUND: To date, a multi-country review evaluating the cost-of-illness (COI) studies from the Central and Eastern European (CEE) region has not yet been published. Our main objective was to provide a general description about published COI studies from CEE. METHODS: A systematic search was performed between 1 January 2006 and 1 June 2017 in Medline, EMBASE, The Cochrane Library, CINAHL, and Web of Science to identify all relevant COI studies from nine CEE countries. COI studies reporting costs without any restrictions by age, co-morbidities, or treatment were included. Methodology, publication standards, and cost results were analysed. RESULTS: We identified 58 studies providing 83 country-specific COI results: Austria (n = 9), Bulgaria (n = 16), Croatia (n = 3), the Czech Republic (n = 10), Hungary (n = 24), Poland (n = 11), Romania (n = 3), Slovakia (n = 3), and Slovenia (n = 4). Endocrine, nutritional, and metabolic diseases (18%), neoplasms (12%), infections (11%), and neurological disorders (11%) were the most frequently studied clinical areas, and multiple sclerosis was the most commonly studied disease. Overall, 57 (98%) of the studies explicitly stated the source of resource use data, 45 (78%) the study perspective, 34 (64%) the costing method, and 24 (58%) reported at least one unit costs. Regardless of methodological differences, a positive relationship was observed between costs of diseases and countries' per capita GDP. CONCLUSIONS: Cost-of-illness studies varied considerably in terms of methodology, publication practice, and clinical areas. Due to these heterogeneities, transferability of the COI results is limited across Central and Eastern European countries.

Shared decision making between patient and GP about referrals from primary care: Does gatekeeping make a difference?

Primary care faces challenging times in many countries, mainly caused by an ageing population. The GPs' role to match patients' demand with medical need becomes increasingly complex with the growing multiple conditions population. Shared decision-making (SDM) is recognized as ideal to the treatment decision making process. Understanding GPs' perception on SDM about patient referrals and whether patients' preferences are considered, becomes increasingly important for improving health outcomes and patient satisfaction. This study aims to 1) understand whether countries vary in how GPs perceive SDM, in patients' referral, 2) describe to what extent SDM in GPs' referrals differ between gatekeeping and non-gatekeeping systems, and 3) identify what factors GPs consider when referring to specialists and describing how this differs between gatekeeping and non-gatekeeping systems. Data were collected between October 2011 and December 2013 in 32 countries through the QUALICOPC study (Quality and Costs of Primary Care in Europe). The first question was answered by assessing GPs' perception on who takes the referral decision. For the second question, a multilevel logistic model was applied. For the third question we analysed the GPs' responses on what patient logistics and need arguments they consider in the referral process. We found: 1) variation in GPs reported SDM- 90% to 35%, 2) a negative correlation between gatekeeper systems and SDM-however, some countries strongly deviate and 3) GPs in gatekeeper systems more often consider patient interests, whereas in non-gatekeeping countries the GP's value more own experience with specialists and benchmarking information. Our findings imply that GPs in gatekeeper systems seem to be less inclined to SDM than GPs in a non-gatekeeping system. The relation between gatekeeping/non-gatekeeping and SDM is not straightforward. A more contextualized approach is needed to understand the relation between gatekeeping as a system design feature and its relation with and/or impact on SDM.

Rationalizing the introduction and use of pharmaceutical products: The role of managed entry agreements in Central and Eastern European countries.

This paper aims to provide an overview of the rationalization strategies for the introduction and use of pharmaceuticals, focusing on the role of managed entry agreements (MEA) in Central and Eastern European (CEE) countries, namely Bulgaria, the Czech Republic, Croatia, Hungary, Poland and Romania. We developed a conceptual framework on MEAs that was used as the basis for a standardized assessment questionnaire sent to country experts to capture their perceptions on their countries' rationalization strategies and MEAs. Our study shows that the main role of MEAs and other related policies embedded in the health care system is to limit the budget impact of drugs in all examined 6 countries. Uncertainty about outcomes and appropriate utilization seem to be of lower priority. Finance-based MEAs are used by all countries. Performance-based MEAs are scarce and used to a limited extent by Hungary and Poland. The overall transparency of the existence and details of MEAs is limited. Expansion of the use and increased transparency of MEAs is recommended. Still, the informational infrastructure and competencies in implementing MEA's need to be developed further.

EQ-5D studies in cardiovascular diseases in eight Central and Eastern European countries: a systematic review of the literature.

BACKGROUND: The measurement of health-related quality of life (HRQoL) by validated generic instruments, such as EQ-5D, has become an increasingly important tool for the assessment of health care in a wide range of diagnoses. AIM: We aimed to systematically review EQ-5D literature on cardiovascular diseases in eight Central and Eastern European (CEE) countries. METHODS: A structured literature search was conducted in MEDLINE, EMBASE, Web of Science, CINAHL, PsycINFO, Cochrane Library, and the EuroQol website up to November 2016. Original cardiovascular-related studies that reported EQ-5D results were included. RESULTS: Of the 36 papers, 17 reported EQ-5D index scores. Most studies were performed in Poland (n = 24, 67%). The most common diagnosis regarding the number of publications and population size was ischaemic heart disease (n = 13, N = 6394), followed by atrial fibrillation (n = 4, N = 1052). The average EQ-5D index scores ranged from 0.61 to 0.88 and from 0.66 to 0.95 for patients before and after cardiac procedure/surgery, respectively (including angioplasty, coronary artery bypass grafting, ablation, surgical correction of septal defects, transcatheter aortic valve implantation [TAVI]). In all studies baseline scores were lower than the repeated assessments after the procedure, with the most substantial improvement of 0.24 in high-risk elderly patients after TAVI. Studies which did not assess invasive treatment reported mean EQ-5D index scores ranging from 0.18 to 0.80. CONCLUSIONS: The number of cardiovascular-related studies reporting HRQoL using EQ-5D has consistently increased in CEE countries over the past decade and is outstanding compared with other clinical fields. The EQ-5D index and EQ VAS scores varied based on the disease severity, patient characteristics, and treatment protocol.

EQ-5D studies in musculoskeletal and connective tissue diseases in eight Central and Eastern European countries: a systematic literature review and meta-analysis.

EQ-5D is becoming the preferred instrument to measure health-state utilities involved in health technology assessment. The objective of this study is to assess the state of EQ-5D research in musculoskeletal disorders in 8 Central and Eastern European countries and to provide a meta-analysis of EQ-5D index scores. Original research articles published in any language between Jan 2000 and Sept 2016 were included, if they reported any EQ-5D outcome from at least two musculoskeletal patients from Austria, Bulgaria, the Czech Republic, Hungary, Poland, Romania, Slovakia, or Slovenia. Risk of bias was assessed with the Cochrane Collaboration's tool. Twenty-nine articles (5992 patients) were included on rheumatoid arthritis (n = 7), osteoporosis (n = 5), chronic pain (n = 5), osteoarthritis (n = 4), ankylosing spondylitis (n = 2), psoriatic arthritis (n = 2), total hip replacement (n = 2), and scleroderma (n = 2). Low back pain was under-represented, while studies in neck pain, systemic lupus erythematosus, gout, and childhood disorders were lacking. EQ-5D index scores were reported in 24 studies, while the version of the instrument and the value-set was not specified in 41% and 46% of the articles, respectively. Meta-analysis was performed on 24 disease states involving 6876 observation points. Intervention effect was reported in 22 subgroups, out of which risk of bias was low in 41%. This review provides recommendations to improve reporting standards of EQ-5D results and highlights potential areas for future research. Coordinated research in conditions with greatest public health impact as well as a development of a regional value-set could provide locally relevant health-state utilities that are transferable among countries within the region.

EQ-5D in Central and Eastern Europe: 2000-2015.

OBJECTIVE: Cost per quality-adjusted life year data are required for reimbursement decisions in many Central and Eastern European (CEE) countries. EQ-5D is by far the most commonly used instrument to generate utility values in CEE. This study aims to systematically review the literature on EQ-5D from eight CEE countries. METHODS: An electronic database search was performed up to 1 July 2015 to identify original EQ-5D studies from the countries of interest. We analysed the use of EQ-5D with respect to clinical areas, methodological rigor, population norms and value sets. RESULTS: We identified 143 studies providing 152 country-specific results with a total sample size of 81,619: Austria (n = 11), Bulgaria (n = 6), Czech Republic (n = 18), Hungary (n = 47), Poland (n = 51), Romania (n = 2), Slovakia (n = 3) and Slovenia (n = 14). Cardiovascular (21 %), neurologic (17 %), musculoskeletal (15 %) and endocrine, nutritional and metabolic diseases (13 %) were the most frequently studied clinical areas. Overall, 112 (78 %) of the studies reported EQ VAS results and 86 (60 %) EQ-5D index scores, of which 27 (31 %) did not specify the applied tariff. Hungary, Poland and Slovenia have population norms. Poland and Slovenia also have a national value set. CONCLUSIONS: Increasing use of EQ-5D is observed throughout CEE. The spread of health technology assessment activities in countries seems to be reflected in the number of EQ-5D studies. However, improvement in informed use and methodological quality of reporting is needed. In jurisdictions where no national value set is available, in order to ensure comparability we recommend to apply the most frequently used UK tariff. Regional collaboration between CEE countries should be strengthened.

Reporting and use of the OECD Health Care Quality Indicators at national and regional level in 15 countries.

QUALITY PROBLEM OR ISSUE: OECD member states are involved since 2003 in a project coordinated by the OECD on Health Care Quality Indicators (HCQI). All OECD countries are biennially requested by the OECD to deliver national data on the quality indicators for international benchmarking purposes. INITIAL ASSESSMENT: Currently, there is no knowledge whether the OECD HCQI information is used by the countries themselves for healthcare system accountability and improvement purposes. CHOICE OF SOLUTION: The objective of the study is to explore the reporting and use of OECD HCQI in OECD member-states. IMPLEMENTATION: Data were collected through a questionnaire sent to all OECD member-states containing factual questions on the reporting on all OECD HCQ-indicators. Responses were received between June and December 2014. In this timeframe, two reminders were sent to the participants. The work progress was presented during HCQI Meetings in November 2014 and May 2015. EVALUATION: Fifteen countries reported to have a total of 163 reports in which one or more HCQIs were reported. One hundred and sixteen were national and 47 were regional reports. Forty-nine reports had a general system focus, 80 were disease specific, 10 referred to a specific type of care setting, 22 were thematic and 2 were a combination of two (disease specific for a particular type of care and thematic for a specific type of care). Most reports were from Canada: 49. All 15 countries use one or more OECD indicators. LESSONS LEARNED: The OECD quality indicators have acquired a clear place in national and regional monitoring activities. Some indicators are reported more often than others. These differences partly reflect differences between healthcare systems. Whereas some indicators have become very common, such as cancer care indicators, others, such as mental healthcare and patient experience indicators are relatively new and require some more time to be adopted more widely.

Health technology assessment in Poland, the Czech Republic, Hungary, Romania and Bulgaria.

This paper describes and discusses the development and use of health technology assessment (HTA) in five Central and Eastern European countries (CEE): Poland, the Czech Republic, Hungary, Romania and Bulgaria. It provides a general snapshot of HTA policies in the selected CEE countries to date by focusing on country case-studies based on document analysis and expert opinion. It offers an overview of similarities and differences between the individual CEE countries and discusses in detail the role of HTA by assessing its formalization and institutionalization, standardization of methodology, the use of HTA in practice and the degree of professionalization of HTA in the region. It finds that HTA has been to some extent implemented in all five countries studied, with methodologies in accordance with international standards, but that challenges remain when it comes to the role of HTA in health care decision-making as well as to human resource capacities of the countries. This paper suggests that coming years will show whether CEE countries develop adequate national analytical capacity to assess and appraise technologies in the context of local need and affordability, instead of using HTA as a mere administrative procedure to fulfill (inter)national requirements. Finally, suggestions are provided to strengthen HTA in CEE countries through cooperation, mutual learning, a common accreditation of HTA bodies and increased network building among CEE HTA experts.