What can we learn from general practitioners who left Spain? A mixed methods international study.

BACKGROUND: International mobility of health workforce affects the performance of health systems and has major relevance in human resources for health policy and planning. To date, there has been little research exploring the reasons why general practitioners (GPs) migrate. This mixed methods study aimed to investigate the reasons why Spain-trained GPs migrate and develop GP retention and recruitment health policy recommendations relevant to Spanish primary care. METHODS: The study followed an explanatory sequential mixed methods study design combining surveys with semi-structured interviews and focus groups with GPs who qualified in Spain and were living overseas at the time of the study. The survey data examined the reasons why GPs left Spain and their intention to return and were analysed using quantitative methods. The transcripts from interviews and focus groups centred on GPs' insights to enhance retention and recruitment in Spain and were analysed thematically. RESULTS: The survey had 158 respondents with an estimated 25.4% response rate. Insufficient salary (75.3%), job insecurity and temporality (67.7%), excessive workload (67.7%), poor primary care governance (55.7%), lack of flexibility in the workplace (43.7%) and personal circumstances (43.7%) were the main reasons for leaving Spain. Almost half of the respondents (48.7%) would consider returning to Spanish general practice if their working conditions improved. Interviews and focus groups with respondents (n = 24) pointed towards the need to improve the quality of employment contracts, working conditions, opportunities for professional development, and governance in primary care for effective retention and recruitment. CONCLUSION: Efforts to improve GP retention and recruitment in Spain should focus on salary, job security, flexibility, protected workload, professional development, and governance. We draw ten GP retention and recruitment recommendations expected to inform urgent policy action to tackle existing and predicted GP shortages in Spanish primary care.

Health system performance assessment: embedding resilience through performance intelligence.

Health systems around the world are facing challenges in achieving their goals. In the wake of the coronavirus disease pandemic, the need for resilient health systems has become even more apparent. This article argues that embedding resilience into health system performance assessment (HSPA) frameworks can be a valuable approach for improving health system performance. This perspective examines key challenges threatening health systems and makes a case for the continued relevance of HSPA by embedding resilience-related performance intelligence.

Acute care pathway assessed through performance indicators during the COVID-19 pandemic in OECD countries (2020-2021): a scoping review.

BACKGROUND: The COVID-19 pandemic severely impacted care for non-COVID patients. Performance indicators to monitor acute care, timely reported and internationally accepted, lacked during the pandemic in OECD countries. This study aims to summarize the performance indicators available in the literature to monitor changes in the quality of acute care in OECD countries during the first year and a half of the pandemic (2020-July 2021) and to assess their trends. METHODS: Scoping review. Search in Embase and MEDLINE (07-07-2022). Acute care performance indicators and indicators related to acute general surgery were collected and collated following a care pathway approach. Indicators assessing identical clinical measures were grouped under a common indicator title. The trends from each group of indicators were collated (increase/decrease/stable). RESULTS: A total of 152 studies were included. 2354 indicators regarding general acute care and 301 indicators related to acute general surgery were included. Indicators focusing on pre-hospital services reported a decreasing trend in the volume of patients: from 225 indicators, 110 (49%) reported a decrease. An increasing trend in pre-hospital treatment times was reported by most of the indicators (n = 41;70%) and a decreasing trend in survival rates of out-of-hospital cardiac arrest (n = 61;75%). Concerning care provided in the emergency department, most of the indicators (n = 752;71%) showed a decreasing trend in admissions across all levels of urgency. Concerning the mortality rate after admission, most of the indicators (n = 23;53%) reported an increasing trend. The subset of indicators assessing acute general surgery showed a decreasing trend in the volume of patients (n = 50;49%), stability in clinical severity at admission (n = 36;53%), and in the volume of surgeries (n = 14;47%). Most of the indicators (n = 28;65%) reported no change in treatment approach and stable mortality rate (n = 11,69%). CONCLUSION: This review signals relevant disruptions across the acute care pathway. A subset of general surgery performance indicators showed stability in most of the phases of the care pathway. These results highlight the relevance of assessing this care pathway more regularly and systematically across different clinical entities to monitor disruptions and to improve the resilience of emergency services during a crisis.

Changes in the quality of cancer care as assessed through performance indicators during the first wave of the COVID-19 pandemic in 2020: a scoping review.

BACKGROUND: Cancer comprises a high burden on health systems. Performance indicators monitoring cancer outcomes are routinely used in OECD countries. However, the development of process and cancer-pathway based information is essential to guide health care delivery, allowing for better monitoring of changes in the quality of care provided. Assessing the changes in the quality of cancer care during the COVID-19 pandemic requires a structured approach considering the high volume of publications. This study aims to summarize performance indicators used in the literature to evaluate the impact of the COVID-19 pandemic on cancer care (January-June 2020) in OECD countries and to assess changes in the quality of care as reported via selected indicators. METHODS: Search conducted in MEDLINE and Embase databases. Performance indicators and their trends were collated according to the cancer care pathway. RESULTS: This study included 135 articles, from which 1013 indicators were retrieved. Indicators assessing the diagnostic process showed a decreasing trend: from 33 indicators reporting on screening, 30 (91%) signalled a decrease during the pandemic (n = 30 indicators, 91%). A reduction was also observed in the number of diagnostic procedures (n = 64, 58%) and diagnoses (n = 130, 89%). The proportion of diagnoses in the emergency setting and waiting times showed increasing trends (n = 8, 89% and n = 14, 56%, respectively). A decreasing trend in the proportion of earliest stage cancers was reported by 63% of indicators (n = 9), and 70% (n = 43) of indicators showed an increasing trend in the proportion of advanced-stage cancers. Indicators reflecting the treatment process signalled a reduction in the number of procedures: 79%(n = 82) of indicators concerning surgeries, 72%(n = 41) of indicators assessing radiotherapy, and 93%(n = 40) of indicators related to systemic therapies. Modifications in cancer treatment were frequently reported: 64%(n = 195) of indicators revealed changes in treatment. CONCLUSIONS: This study provides a summary of performance indicators used in the literature to assess the cancer care pathway from January 2020 to June 2020 in OECD countries, and the changes in the quality of care signalled by these indicators. The trends reported inform on potential bottlenecks of the cancer care pathway. Monitoring this information closely could contribute to identifying moments for intervention during crises.

Implementing structured follow-up of neonatal and paediatric patients: an evaluation of three university hospital case studies using the functional resonance analysis method.

BACKGROUND: In complex critical neonatal and paediatric clinical practice, little is known about long-term patient outcomes and what follow-up care is most valuable for patients. Emma Children's Hospital, Amsterdam UMC (Netherlands), implemented a follow-up programme called Follow Me for neonatal and paediatric patient groups, to gain more insight into long-term outcomes and to use such outcomes to implement a learning cycle for clinical practice, improve follow-up care and facilitate research. Three departments initiated re-engineering and change processes. Each introduced multidisciplinary approaches to long-term follow-up, including regular standardised check-ups for defined age groups, based on medical indicators, developmental progress, and psychosocial outcomes in patients and their families. This research evaluates the implementation of the three follow-up programmes, comparing predefined procedures (work-as-imagined) with how the programmes were implemented in practice (work-as-done). METHODS: This study was conducted in 2019-2020 in the outpatient settings of the neonatal intensive care, paediatric intensive care and paediatric surgery departments of Emma Children's Hospital. It focused on the organisational structure of the follow-up care. The functional resonance analysis method (FRAM) was applied, using documentary analysis, semi-structured interviews, observations and feedback sessions. RESULTS: One work-as-imagined model and four work-as-done models were described. The results showed vast data collection on medical, developmental and psychosocial indicators in all work-as-done models; however, process indicators for programme effectiveness and performance were missing. In practice there was a diverse allocation of roles and responsibilities and their interrelations to create a multidisciplinary team; there was no one-size-fits-all across the different departments. Although control and feedback loops for long-term outcomes were specified with respect to the follow-up groups within the programmes, they were found to overlap and misalign with other internal and external long-term outcome monitoring practices. CONCLUSION: Implementing structured long-term follow-up may provide insights for improving daily practice and follow-up care, with the precondition of standardised measurements. Lessons learned from practice are (1) to address fragmentation in data collection and storage, (2) to incorporate the diverse ways to create a multidisciplinary team in practice, and (3) to include timely actionable indicators on programme effectiveness and performance, alongside medical, developmental and psychosocial indicators.

How do Patient-reported Outcome Scores in International Hip and Knee Arthroplasty Registries Compare?

BACKGROUND: Patient-reported outcome measures (PROMs) are the only systematic approach through which the patient's perspective can be considered by surgeons (in determining a procedure's efficacy or appropriateness) or healthcare systems (in the context of value-based healthcare). PROMs in registries enable international comparison of patient-centered outcomes after total joint arthroplasty, but the extent to which those scores may vary between different registry populations has not been clearly defined. QUESTIONS/PURPOSES: (1) To what degree do mean change in general and joint-specific PROM scores vary across arthroplasty registries, and to what degree is the proportion of missing PROM scores in an individual registry associated with differences in the mean reported change scores? (2) Do PROM scores vary with patient BMI across registries? (3) Are comorbidity levels comparable across registries, and are they associated with differences in PROM scores? METHODS: Thirteen national, regional, or institutional registries from nine countries reported aggregate PROM scores for patients who had completed PROMs preoperatively and 6 and/or 12 months postoperatively. The requested aggregate PROM scores were the EuroQol-5 Dimension Questionnaire (EQ-5D) index values, on which score 1 reflects "full health" and 0 reflects "as bad as death." Joint-specific PROMs were the Oxford Knee Score (OKS) and the Oxford Hip Score (OHS), with total scores ranging from 0 to 48 (worst-best), and the Hip Disability and Osteoarthritis Outcome Score-Physical Function shortform (HOOS-PS) and the Knee Injury and Osteoarthritis Outcome Score-Physical Function shortform (KOOS-PS) values, scored 0 to 100 (worst-best). Eligible patients underwent primary unilateral THA or TKA for osteoarthritis between 2016 and 2019. Registries were asked to exclude patients with subsequent revisions within their PROM collection period. Raw aggregated PROM scores and scores adjusted for age, gender, and baseline values were inspected descriptively. Across all registries and PROMs, the reported percentage of missing PROM data varied from 9% (119 of 1354) to 97% (5305 of 5445). We therefore graphically explored whether PROM scores were associated with the level of data completeness. For each PROM cohort, chi-square tests were performed for BMI distributions across registries and 12 predefined PROM strata (men versus women; age 20 to 64 years, 65 to 74 years, and older than 75 years; and high or low preoperative PROM scores). Comorbidity distributions were evaluated descriptively by comparing proportions with American Society of Anesthesiologists (ASA) physical status classification of 3 or higher across registries for each PROM cohort. RESULTS: The mean improvement in EQ-5D index values (10 registries) ranged from 0.16 to 0.33 for hip registries and 0.12 to 0.25 for knee registries. The mean improvement in the OHS (seven registries) ranged from 18 to 24, and for the HOOS-PS (three registries) it ranged from 29 to 35. The mean improvement in the OKS (six registries) ranged from 15 to 20, and for the KOOS-PS (four registries) it ranged from 19 to 23. For all PROMs, variation was smaller when adjusting the scores for differences in age, gender, and baseline values. After we compared the registries, there did not seem to be any association between the level of missing PROM data and the mean change in PROM scores. The proportions of patients with BMI 30 kg/m 2 or higher ranged from 16% to 43% (11 hip registries) and from 35% to 62% (10 knee registries). Distributions of patients across six BMI categories differed across hip and knee registries. Further, for all PROMs, distributions also differed across 12 predefined PROM strata. For the EQ-5D, patients in the younger age groups (20 to 64 years and 65 to 74 years) had higher proportions of BMI measurements greater than 30 kg/m 2 than older patients, and patients with the lowest baseline scores had higher proportions of BMI measurements more than 30 kg/m 2 compared with patients with higher baseline scores. These associations were similar for the OHS and OKS cohorts. The proportions of patients with ASA Class at least 3 ranged across registries from 6% to 35% (eight hip registries) and from 9% to 42% (nine knee registries). CONCLUSION: Improvements in PROM scores varied among international registries, which may be partially explained by differences in age, gender, and preoperative scores. Higher BMI tended to be associated with lower preoperative PROM scores across registries. Large variation in BMI and comorbidity distributions across registries suggest that future international studies should consider the effect of adjusting for these factors. Although we were not able to evaluate its effect specifically, missing PROM data is a recurring challenge for registries. Demonstrating generalizability of results and evaluating the degree of response bias is crucial in using registry-based PROMs data to evaluate differences in outcome. Comparability between registries in terms of specific PROMs collection, postoperative timepoints, and demographic factors to enable confounder adjustment is necessary to use comparison between registries to inform and improve arthroplasty care internationally. LEVEL OF EVIDENCE: Level III, therapeutic study.

Status of the health information system in Ireland and its fitness to support health system performance assessment: a multimethod assessment based on stakeholder involvement.

BACKGROUND: Between 2019 and 2021, the first Irish health system performance assessment (HSPA) framework was developed. As routinely collected health data are necessary to continuously populate indicators of an HSPA framework, a purpose-driven assessment of the health information system (HIS) in Ireland and its fitness to support the implementation of an HSPA framework was conducted. This study reports on the status of the Irish HIS through a multimethod assessment based on continuous broad stakeholder involvement. METHODS: Between May and November 2020, over 50 informants were engaged in individual and group interviews and stakeholder consultation workshops as part of the HIS assessment process. Descriptive themes and high-level data availability heatmaps were derived from interview and workshop data using thematic analysis. Indicator "passports" for the HSPA framework were populated during stakeholder consultation workshops and analysed using univariate descriptive statistics. RESULTS: The HIS in Ireland was able to provide administrative, survey and registry-based data for public sector acute care services, focusing on structure, process and output metrics. Significant data availability gaps, most notably from primary care, private hospitals and community care, were reported, with little availability of electronic health record and people-reported data. Data on outcome metrics were mostly missing, as were linkage possibilities across datasets for care pathway monitoring. The COVID-19 pandemic highlighted the national HIS's shortcomings but also the capacity for rapid development and improvement. CONCLUSIONS: A tailor-made assessment of the HIS in Ireland, involving a broad set of relevant stakeholders, revealed strengths, weaknesses and areas for improvement in the Irish health data landscape. It also contributed to the development of a national HSPA framework and momentum to further strengthen data infrastructure and governance, while working towards a more data-driven and person-centred healthcare system. This work demonstrates the utility of an inclusive HIS assessment process and is applicable beyond Ireland, where this case study was conducted.

Citizen engagement in healthcare procurement decision-making by healthcare insurers: recent experiences in the Netherlands.

BACKGROUND: In insurance-based healthcare systems, healthcare insurers are interested in engaging citizens in care procurement to contract healthcare services that matter to people. In the Netherlands, an amendment to the Health Insurance Act was set forth in 2021 to formalize and strengthen the engagement of the insured population with healthcare insurers' procurement cycles. This study explores the role of Dutch healthcare insurers in operationalizing citizen engagement in procurement cycles before changes occur linked to the amendment to the Health Insurance Act. METHODS: A phenomenological qualitative design was employed in two phases: (1) we consulted academics and policy experts on the role of healthcare insurers regarding citizen engagement; (2) we conducted focus groups with representatives of healthcare insurers to understand how citizens' engagement is being operationalized. Transcripts of the interviews with experts and detailed notes of focus group meetings were analysed using a qualitative inductive approach. Selected excerpts were analysed on discourse and content and organized by a coding scheme following a rigorous and accelerated data reduction technique. RESULTS: We identified four strategies used by healthcare insurers to operationalize citizen engagement: (1) broadening their population health orientation; (2) developing and improving mechanisms for engaging citizens; (3) strengthening features of data governance for effective use of value-driven data; (4) implementing financial and incentive mechanisms among healthcare providers in support of value-based healthcare. However, regulated market mechanisms and low institutional trust in healthcare insurers undermine their transition from merely funding healthcare towards becoming people-centred value-based healthcare purchasers. CONCLUSION: Dutch healthcare insurers seem to be strengthening the community orientation of their functioning while enhancing the end-to-end experience of the insured. The expected practical effects of the amendment to the Health Insurance Act include broadening the role of the council of insurees in decision-making processes and systematically documenting the efforts set forth by healthcare insurers in engaging citizens. Further research is needed to better understand how the regulated competitive market could be hampering the engagement of citizens in healthcare procurement decision-making and value creation from the citizens' perspective.

Improving performance intelligence for governing an integrated health and social care delivery network: a case study on the Amsterdam Noord district.

BACKGROUND: A guiding principle of a successful integrated health and social care delivery network is to establish a governance approach based on learning, grounded in a data and knowledge infrastructure. The 'Krijtmolen Alliantie' is a network of health and social care providers with the ambition to realize such a performance intelligence driven governance model in line with the Triple Aim. This study seeks to identify what performance intelligence is available and how it can be improved. METHODS: This case study was conducted in the district of Amsterdam Noord, the Netherlands, and employed 23 semi-structured interviews with stakeholders in health and social care, a feasibility analysis of available administrative data, and a reflection meeting with board members of the 'Krijtmolen Alliantie'. Information needs for performance intelligence by the stakeholders were mapped and a data landscape of the district covered by the network was drafted. Finally, in the reflection meeting with board members of the 'Krijtmolen Alliantie' the information needs and data landscape were aligned with governing needs, resulting in priority domains around which to strengthen the data infrastructure for governance of the integrated health and social care delivery network. RESULTS: The 'Krijtmolen Alliantie' encompasses a network of providers with a diverse range of catchment areas. There are indicators on population health and welfare, however they have limited actionability for providers due to a misalignment with their respective catchment areas. There is a barrier in data exchange between health and social care providers. It is difficult to construct one indicator for per capita cost in the Dutch health data infrastructure as health and social care are subdivided in financing siloes. Priority domains for improvement of performance intelligence for the 'Krijtmolen Alliantie' are: 1) Per capita and per patient cost data integration that would allow combined accountability through aligning financial incentives to facilitate integrated care, and 2) combined patient experience and outcome measures to reflect network quality of care and patient experience performance. CONCLUSION: Available performance intelligence lacks actionability for the governance of integrated care networks. Our recommendation is to align performance intelligence with the regional governance responsibilities of stakeholders for health and social care delivery.

The current and potential uses of Electronic Medical Record (EMR) data for primary health care performance measurement in the Canadian context: a qualitative analysis.

BACKGROUND: Electronic Medical Records (EMRs) are a rich data source to measure and improve quality of care. As Canadian primary health care (PHC) EMRs mature, there is increasing potential use of EMR data for performance measurement. This study identifies and describes current uses of EMR data for performance measurement and considerations to further its potential in the Canadian context. METHODS: We applied a qualitative case study design and descriptive assessment in three phases, consulting multiple data sources including scientific and grey literature, system leaders (n = 41), and clinician/researchers (n = 20). Phases included a multimethod approach to identify initiatives using EMR data for performance measurement across Canadian jurisdictions; in-depth review of current initiatives identified from a healthcare performance intelligence lens; and triangulation and thematic analysis across data sources to explore considerations for advancing performance measurement uses of EMR data in the Canadian context. RESULTS: Six initiatives of EMR data use for performance measurement were identified: one multi-jurisdictional; five jurisdiction-specific in the provinces of British Columbia, Manitoba and Ontario. EMR data uses were predominately for micro-level PHC physician and team performance improvement, with some use for meso-level organization/network-wide improvement. Indicator sets varied in number, though shared emphasis on chronic disease management and prevention/screening and to a lesser extent medication management. Key considerations for governing, resourcing and implementing EMR data for performance measurement were identified. CONCLUSIONS: The extent of EMR data use for performance measurement varies across Canada. To further its potential, pan-Canadian data and privacy standards, performance intelligence competencies and renewed core PHC indicators should be prioritized. Experiences across countries, coupled with increasing momentum for performance measurement using real-world data, should be leveraged to avoid unnecessarily slow progress in Canada and abroad.

Patient-reported indicators in mental health care: towards international standards among members of the OECD.

BACKGROUND: Achieving people-centred health care systems requires new and innovative strategies to capture information about whether, and to what degree, health care is successful in improving health from the perspective of the patient. Patient-reported outcome measures (PROMs) and Patient-reported experience measures (PREMs) can bring some of these new insights, and are increasingly used in research, clinical care, and policymaking. METHODS: This paper reflects the ongoing discussions and findings of the OECD PaRIS Working Group on Patient-reported Indicators for Mental Health Care. RESULTS: The OECD has been measuring quality of care for mental health conditions over the last 14 years through the Health Care Quality and Outcomes (HCQO) program; nonetheless, information on how persons with mental health problems value the services they receive, and impact of the services, remains limited. As of 2018, a survey from the OECD showed that only five of the twelve countries surveyed (Australia, Israel, Netherlands, Sweden, United Kingdom) reported PROMs and PREMs collection on a regular basis in mental health settings. The paper details some of the challenges specific to the collection and use of PROMs and PREMs in mental health care, and examples from countries which have implemented comprehensive programmes to gather information about PROMs and PREMs for individuals receiving mental health services. CONCLUSIONS: Given the health and economic impact of mental ill-health across all OECD countries, there is significant value to being able to assess the quality and outcomes of care in this area using internationally-comparable measures. Continued international harmonisation of PROMs and PREMs for mental health through international coordination is a key way to facilitate the sharing of national experiences, promote the use of PROMs and PREMs, and create meaningful indicators for national and international benchmarking.

Measuring patient voice matters: setting the scene for patient-reported indicators.

BACKGROUND: Achieving people-centred health care systems requires new and innovative strategies to capture information about whether, and to what degree, health care is successful in improving health from the perspective of the patient. Patient-reported outcome measures (PROMs) and Patient-reported experience measures (PREMs) can bring some of these new insights, and are increasingly used in research, clinical care, and policymaking. METHODS: This paper reflects the ongoing discussions and findings of the OECD PaRIS Working Group on Patient-reported Indicators for Mental Health Care. RESULTS: The OECD has been measuring quality of care for mental health conditions over the last 14 years through the Health Care Quality and Outcomes (HCQO) program; nonetheless, information on how persons with mental health problems value the services they receive, and impact of the services, remains limited. As of 2018, a survey from the OECD showed that only five of the twelve countries surveyed (Australia, Israel, Netherlands, Sweden, United Kingdom) reported PROMs and PREMs collection on a regular basis in mental health settings. The paper details some of the challenges specific to the collection and use of PROMs and PREMs in mental health care, and examples from countries which have implemented comprehensive programmes to gather information about PROMs and PREMs for individuals receiving mental health services. CONCLUSIONS: Given the health and economic impact of mental ill-health across all OECD countries, there is significant value to being able to assess the quality and outcomes of care in this area using internationally-comparable measures. Continued international harmonisation of PROMs and PREMs for mental health through international coordination is a key way to facilitate the sharing of national experiences, promote the use of PROMs and PREMs, and create meaningful indicators for national and international benchmarking.

Patients Place More of an Emphasis on Physical Recovery Than Return to Work or Financial Recovery.

BACKGROUND: Value-based healthcare models aim to incentivize healthcare providers to offer interventions that address determinants of health. Understanding patient priorities for physical and socioeconomic recovery after injury can help determine which services and resources are most useful to patients. QUESTIONS/PURPOSES: (1) Do trauma patients consistently identify a specific aspect/domain of recovery as being most important at 6 weeks, 6 months, and 12 months after an injury? (2) Does the relative importance of those domains change within the first year after injury? (3) Are differences in priorities greater between patients than for a given patient over time? (4) Are different recovery priorities associated with identifiable biopsychosocial factors? METHODS: Between June 2018 and December 2018, 504 adult patients with fractures of the extremities or pelvis were surgically treated at the study site. For this prospective longitudinal study, we purposefully sampled patients from 6 of the 12 orthopaedic attendings' postoperative clinics. The participating surgeons surgically treated 243 adult patients with fractures of the extremities or pelvis. Five percent (11 of 243) of patients met inclusion criteria but missed their appointments during the 6-week recruitment window and could not be consented. We excluded 4% (9 of 243) of patients with a traumatic brain injury, 1% (2) of patients with a spinal cord injury, and 5% (12) of non-English-speaking patients (4% Spanish speaking [10]; 1% other languages [2]). Eighty-six percent of eligible patients (209 of 243) were approached for consent, and 5% (11 of 209) of those patients refused to participate. All remaining 198 patients consented and completed the baseline survey; 83% (164 of 198 patients) completed at least 6 months of follow-up, and 68% (134 of 198 patients) completed the 12-month assessment. The study participants' mean age was 44 ± 17 years, and 63% (125 of 198) were men. The primary outcome was the patient's recovery priorities, assessed at 6 weeks, 6 months, and 12 months after fracture using a discrete choice experiment. Discrete choice experiments are a well-established method for eliciting decisional preferences. In this technique, respondents are presented with a series of hypothetical scenarios, described by a set of plausible attributes or outcomes, and asked to select their preferred scenario. We used hierarchical Bayesian modeling to calculate individual-level estimates of the relative importance of physical recovery, work-related recovery, and disability benefits, based on the discrete choice experiment responses. The hierarchical Bayesian model improves upon more commonly used regression techniques by accounting for the observed response patterns of individual patients and the sequence of scenarios presented in the discrete choice experiment when calculating the model estimates. We computed the coefficient of variation for the three recovery domains and compared the between-patient versus within-patient differences using asymptotic tests. Separate prognostic models were fit for each of the study's three recovery domains to assess marginal changes in the importance of the recovery domain based on patient characteristics and factors that remained constant over the study (such as sex or preinjury work status) and patient characteristics and factors that varied over the study (including current work status or patient-reported health status). We previously published the 6-week results. This paper expands upon the prior publication to evaluate longitudinal changes in patient recovery priorities. RESULTS: Physical recovery was the respondents' main priority at all three timepoints, representing 60% ± 9% of their overall concern. Work-related recovery and access to disability benefits were of secondary importance and were associated with 27% ± 6% and 13% ± 7% of the patients' concern, respectively. The patients' concern for physical recovery was 6% (95% CrI 4% to 7%) higher at 12 months after fracture that at 6 weeks postfracture. The mean concern for work-related recovery increased by 7% (95% CrI 6% to 8%) from 6 weeks to 6 months after injury. The mean importance of disability benefits increased by 2% (95% CrI 1% to 4%) from 6 weeks to 6 months and remained 2% higher (95% CrI 0% to 3%) at 12 months after the injury. Differences in priorities were greater within a given patient over time than between patients as measured using the coefficient of variation (physical recovery [245% versus 7%; p < 0.001], work-related recovery [678% versus 12%; p < 0.001], and disability benefits [620% versus 33%; p < 0.001]. There was limited evidence that biopsychosocial factors were associated with variation in recovery priorities. Patients' concern for physical recovery was 2% higher for every 10-point increase in their Patient-reported Outcome Measure Information System (PROMIS) physical health status score (95% CrI 1% to 3%). A 10-point increase in the patient's PROMIS mental health status score was associated with a 1% increase in concern for work-related recovery (95% CrI 0% to 2%). CONCLUSION: Work-related recovery and accessing disability benefits were a secondary concern compared with physical recovery in the 12 months after injury for patients with fractures. However, the importance of work-related recovery was elevated after the subacute phase. Priorities were highly variable within a given patient in the year after injury compared with between-patient differences. Given this variation, orthopaedic surgeons should consider assessing and reassessing the socioeconomic well-being of their patients throughout their continuum of care. LEVEL OF EVIDENCE: Level II, therapeutic study.

Exploring Changes to the Actionability of COVID-19 Dashboards Over the Course of 2020 in the Canadian Context: Descriptive Assessment and Expert Appraisal Study.

BACKGROUND: Public web-based COVID-19 dashboards are in use worldwide to communicate pandemic-related information. Actionability of dashboards, as a predictor of their potential use for data-driven decision-making, was assessed in a global study during the early stages of the pandemic. It revealed a widespread lack of features needed to support actionability. In view of the inherently dynamic nature of dashboards and their unprecedented speed of creation, the evolution of dashboards and changes to their actionability merit exploration. OBJECTIVE: We aimed to explore how COVID-19 dashboards evolved in the Canadian context during 2020 and whether the presence of actionability features changed over time. METHODS: We conducted a descriptive assessment of a pan-Canadian sample of COVID-19 dashboards (N=26), followed by an appraisal of changes to their actionability by a panel of expert scorers (N=8). Scorers assessed the dashboards at two points in time, July and November 2020, using an assessment tool informed by communication theory and health care performance intelligence. Applying the nominal group technique, scorers were grouped in panels of three, and evaluated the presence of the seven defined features of highly actionable dashboards at each time point. RESULTS: Improvements had been made to the dashboards over time. These predominantly involved data provision (specificity of geographic breakdowns, range of indicators reported, and explanations of data sources or calculations) and advancements enabled by the technologies employed (customization of time trends and interactive or visual chart elements). Further improvements in actionability were noted especially in features involving local-level data provision, time-trend reporting, and indicator management. No improvements were found in communicative elements (clarity of purpose and audience), while the use of storytelling techniques to narrate trends remained largely absent from the dashboards. CONCLUSIONS: Improvements to COVID-19 dashboards in the Canadian context during 2020 were seen mostly in data availability and dashboard technology. Further improving the actionability of dashboards for public reporting will require attention to both technical and organizational aspects of dashboard development. Such efforts would include better skill-mixing across disciplines, continued investment in data standards, and clearer mandates for their developers to ensure accountability and the development of purpose-driven dashboards.

Features Constituting Actionable COVID-19 Dashboards: Descriptive Assessment and Expert Appraisal of 158 Public Web-Based COVID-19 Dashboards.

BACKGROUND: Since the outbreak of COVID-19, the development of dashboards as dynamic, visual tools for communicating COVID-19 data has surged worldwide. Dashboards can inform decision-making and support behavior change. To do so, they must be actionable. The features that constitute an actionable dashboard in the context of the COVID-19 pandemic have not been rigorously assessed. OBJECTIVE: The aim of this study is to explore the characteristics of public web-based COVID-19 dashboards by assessing their purpose and users ("why"), content and data ("what"), and analyses and displays ("how" they communicate COVID-19 data), and ultimately to appraise the common features of highly actionable dashboards. METHODS: We conducted a descriptive assessment and scoring using nominal group technique with an international panel of experts (n=17) on a global sample of COVID-19 dashboards in July 2020. The sequence of steps included multimethod sampling of dashboards; development and piloting of an assessment tool; data extraction and an initial round of actionability scoring; a workshop based on a preliminary analysis of the results; and reconsideration of actionability scores followed by joint determination of common features of highly actionable dashboards. We used descriptive statistics and thematic analysis to explore the findings by research question. RESULTS: A total of 158 dashboards from 53 countries were assessed. Dashboards were predominately developed by government authorities (100/158, 63.0%) and were national (93/158, 58.9%) in scope. We found that only 20 of the 158 dashboards (12.7%) stated both their primary purpose and intended audience. Nearly all dashboards reported epidemiological indicators (155/158, 98.1%), followed by health system management indicators (85/158, 53.8%), whereas indicators on social and economic impact and behavioral insights were the least reported (7/158, 4.4% and 2/158, 1.3%, respectively). Approximately a quarter of the dashboards (39/158, 24.7%) did not report their data sources. The dashboards predominately reported time trends and disaggregated data by two geographic levels and by age and sex. The dashboards used an average of 2.2 types of displays (SD 0.86); these were mostly graphs and maps, followed by tables. To support data interpretation, color-coding was common (93/158, 89.4%), although only one-fifth of the dashboards (31/158, 19.6%) included text explaining the quality and meaning of the data. In total, 20/158 dashboards (12.7%) were appraised as highly actionable, and seven common features were identified between them. Actionable COVID-19 dashboards (1) know their audience and information needs; (2) manage the type, volume, and flow of displayed information; (3) report data sources and methods clearly; (4) link time trends to policy decisions; (5) provide data that are "close to home"; (6) break down the population into relevant subgroups; and (7) use storytelling and visual cues. CONCLUSIONS: COVID-19 dashboards are diverse in the why, what, and how by which they communicate insights on the pandemic and support data-driven decision-making. To leverage their full potential, dashboard developers should consider adopting the seven actionability features identified.

Engaging citizens in the development of a health system performance assessment framework: a case study in Ireland.

BACKGROUND: The launch in 2017 of the Irish 10-year reform programme Sláintecare represents a key commitment in the future of the health system. An important component of the programme was the development of a health system performance assessment (HSPA) framework. In 2019, the Department of Health of Ireland (DoH) and Health Service Executive (HSE) commissioned the technical support of researchers to develop an outcome-oriented HSPA framework which should reflect the shared priorities of multiple stakeholders, including citizens. This study describes the method applied in the Irish context and reflects on the added value of using a citizen panel in the development of an HSPA framework. METHODS: A panel of 15 citizens was convened, recruited by a third-party company using a sampling strategy to achieve a balanced mix representing the Irish society. Panellists received lay-language preparatory materials before the meeting. Panellists used a three-colour scheme to signal the importance of performance measures. An exit questionnaire was administered to understand how participants experienced being part of the panel. The citizen panel was the first in a series of three panels towards the development of the HSPA framework, followed by panels including representatives of the DoH and HSE, and representatives from professional associations and special interest groups. RESULTS: The citizen panel generated 249 health performance measures ranging across 13 domains. Top-ranking domains to the citizen panel (people-centredness, coordination of care, and coverage) were less prioritized by the other panels; domains less prioritized by the citizen panel, such as accessibility, responsiveness, efficiency, and effectiveness, were of higher priority in the other panels. Citizen panellists shared a similar understanding of what a citizen panel involves and described their experience at the panel as enjoyable, interesting, and informative. CONCLUSIONS: The priorities of the citizen panel were accounted for during all phases of developing the HSPA framework. This was possible by adopting an inclusive development process and by engaging citizens early on. Citizen engagement in HSPA development is essential for realizing citizen-driven healthcare system performance and generating trust and ownership in performance intelligence. Future research could expand the use of citizen panels to assess, monitor, and report on the performance of healthcare systems.

International comparison of pressure ulcer measures in long-term care facilities: Assessing the methodological robustness of 4 approaches to point prevalence measurement.

INTRODUCTION: Pressure ulcer indicators are among the most frequently used performance measures in long-term care settings. However, measurement systems vary and there is limited knowledge about the international comparability of different measurement systems. The aim of this analysis was to identify possible avenues for international comparisons of data on pressure ulcer prevalence among residents of long-term care facilities. MATERIAL AND METHODS: A descriptive analysis of the four point prevalence measurement systems programs used in 28 countries on three continents was performed. The criteria for the description and analysis were based on the scientific literature on criteria for indicator selection, on issues in international comparisons of data and on specific challenges of pressure ulcer measurements. RESULTS: The four measurement systems use a prevalence measure based on very similar numerator and denominator definitions. All four measurement systems also collect data on patient mobility. They differ in the pressure ulcer classifications used and the requirements for a head-to-toe resident examination. The regional or country representativeness of long-term care facilities also varies among the four measurement systems. CONCLUSIONS: Methodological differences among the point prevalence measurement systems are an important barrier to reliable comparisons of pressure ulcer prevalence data. The alignment of the methodologies may be improved by implementing changes to the study protocols, such as aligning the classification of pressure ulcers and requirements for a head-to-toe resident skin assessment. The effort required for each change varies. All these elements need to be considered by any initiative to facilitate international comparison and learning.

More public health service providers are experiencing job burnout than clinical care providers in primary care facilities in China.

BACKGROUND: Health workers are at high risk of job burnout. Primary care in China has recently expanded its scope of services to a broader range of public health services in addition to clinical care. This study aims to measure the prevalence of burnout and identify its associated factors among clinical care and public health service providers at primary care facilities. METHODS: A cross-sectional survey (2018) was conducted among 17,816 clinical care and public health service providers at 701 primary care facilities from six provinces. Burnout was measured by the Chinese version of the Maslach Burnout Inventory-General Scale, and multilevel linear regression analysis was conducted to identify burnout's association with demographics, as well as occupational and organisational factors. RESULTS: Overall, half of the providers (50.09%) suffered from burnout. Both the presence of burnout and the proportion of severe burnout among public health service providers (58.06% and 5.25%) were higher than among clinical care providers (47.55% and 2.26%, respectively). Similar factors were associated with burnout between clinical care and public health service providers. Younger, male, lower-educated providers and providers with intermediate professional title, permanent contract or higher working hours were related to a higher level of burnout. Organisational environment, such as the presence of a performance-based salary system, affected job burnout. CONCLUSIONS: Job burnout is prevalent among different types of primary care providers in China, indicating the need for actions that encompass the entirety of primary care. We recommend strengthening the synergy between clinical care and public health services and transforming the performance-based salary system into a more quality-based system that includes teamwork incentives.

A problem-oriented systems approach to primary care system development: development and initial testing of the problem-oriented primary care system development record.

BACKGROUND: There is significant global policy interest related to enabling a data-driven approach for evidence-based primary care system development. This paper describes the development and initial testing of a prototype tool (the Problem-Oriented Primary Care System Development Record, or PCSDR) that enables a data-driven and contextualized approach to primary care system development. METHODS: The PCSDR is an electronic record that enables the systematic input, classification, structuring, storage, processing and analysis of different types of data related to the structure, function and performance of primary care systems over time. Data inputted into the PCSDR was coded using the WHO's PHC-IMPACT framework and classification system. The PCSDR's functionalities were tested by using a case study of primary care system development in Tajikistan. RESULTS: Tajikistan's case study demonstrated that the PCSDR is a potentially effective and conceptually-sound tool for the input, classification, structuring and storage of different data types from myriad sources. The PCSDR is therefore a basic data entry and data management system that enables query and analytics functions for health services research and evidence-based primary care system development functions. CONCLUSIONS: The PCSDR is a data system that enables a contextualized approach to evidence-based primary care system development. It represents a coherent and effective synthesis of the fields of primary care system development and performance assessment. The PCSDR enables analysts to leverage primary care performance assessment frameworks for a broad range of functions related to health systems analysis, improvement and the development of learning health systems.

Individual, institutional, and scientific environment factors associated with questionable research practices in the reporting of messages and conclusions in scientific health services research publications.

BACKGROUND: Health Services Research findings (HSR) reported in scientific publications may become part of the decision-making process on healthcare. This study aimed to explore associations between researcher's individual, institutional, and scientific environment factors and the occurrence of questionable research practices (QRPs) in the reporting of messages and conclusions in scientific HSR publications. METHODS: We employed a mixed-methods study design. We identified factors possibly contributing to QRPs in the reporting of messages and conclusions through a literature review, 14 semi-structured interviews with HSR institutional leaders, and 13 focus-groups amongst researchers. A survey corresponding with these factors was developed and shared with 172 authors of 116 scientific HSR publications produced by Dutch research institutes in 2016. We assessed the included publications for the occurrence of QRPs. An exploratory factor analysis was conducted to identify factors within individual, institutional, and environmental domains. Next, we conducted bivariate analyses using simple Poisson regression to explore factors' association with the number of QRPs in the assessed HSR publications. Factors related to QRPs with a p-value < .30 were included in four multivariate models tested through a multiple Poisson regression. RESULTS: In total, 78 (45%) participants completed the survey (51.3% first authors and 48.7% last authors). Twelve factors were included in the multivariate analyses. In all four multivariate models, a higher score of "pressure to create societal impact" (Exp B = 1.28, 95% CI [1.11, 1.47]), was associated with higher number of QRPs. Higher scores on "specific training" (Exp B = 0.85, 95% CI [0.77-0.94]) and "co-author conflict of interest" (Exp B = 0.85, 95% CI [0.75-0.97]) factors were associated with a lower number of QRPs. Stratification between first and last authors indicated different factors were related to the occurrence of QRPs for these groups. CONCLUSION: Experienced pressure to create societal impact is associated with more QRPs in the reporting of messages and conclusions in HSR publications. Specific training in reporting messages and conclusions and awareness of co-author conflict of interests are related to fewer QRPs. Our results should stimulate awareness within the field of HSR internationally on opportunities to better support reporting in scientific HSR publications.