Comparing methods for assessing the reliability of health care quality measures.

Quality measurement plays an increasing role in U.S. health care. Measures inform quality improvement efforts, public reporting of variations in quality of care across providers and hospitals, and high-stakes financial decisions. To be meaningful in these contexts, measures should be reliable and not heavily impacted by chance variations in sampling or measurement. Several different methods are used in practice by measure developers and endorsers to evaluate reliability; however, there is uncertainty and debate over differences between these methods and their interpretations. We review methods currently used in practice, pointing out differences that can lead to disparate reliability estimates. We compare estimates from 14 different methods in the case of two sets of mental health quality measures within a large health system. We find that estimates can differ substantially and that these discrepancies widen when sample size is reduced.

DataDerm: Improving trends in performance measurement.

BACKGROUND: Medicare's legacy quality reporting programs were consolidated into the Merit-Based Incentive Payment System (MIPS) in 2015. PURPOSE: The DataDerm registry of the American Academy of Dermatology was examined to understand the potential for and subsequent rate of improvement across 23 performance measures. METHODS: We examined the level of performance across 23 performance measures with at least 20 clinicians reporting on at least 50 patients' experience. We calculated the following values: the aggregate performance rate for each measure and the overall aggregate performance rate. RESULTS: The aggregate performance rate for each measure ranged from 20.4% for AAD 1 (Psoriasis: Assessment of Disease Activity), to 99.9% for measure ACMS 1 (Avoidance of Opioid Prescriptions for Reconstruction After Skin Resection). Three of 23 measures had an aggregate performance over 95%. The overall aggregate performance rate across all 23 measures was 81.2%, indicating an aggregate potential for improvement of 18.8% across the 23 measures. Nine performance measures reported across the first five years of DataDerm's existence were tracked through time to understand trends in performance through time. The performance across the nine performance measures meeting the inclusion criteria consistently improved in the initial years (2016 through 2018) of DataDerm participation and showed some variation in 2019 and 2020. CONCLUSIONS: These data provide evidence that the very act of participation in a multi-institutional registry and tracking compliance with performance measures can lead to improvements in compliance with the performance measures and therefore improvements in quality of care.

Health system performance assessment: embedding resilience through performance intelligence.

Health systems around the world are facing challenges in achieving their goals. In the wake of the coronavirus disease pandemic, the need for resilient health systems has become even more apparent. This article argues that embedding resilience into health system performance assessment (HSPA) frameworks can be a valuable approach for improving health system performance. This perspective examines key challenges threatening health systems and makes a case for the continued relevance of HSPA by embedding resilience-related performance intelligence.

Identifying performance indicators to measure overall performance of telephone triage - a scoping review.

OBJECTIVE: This article aims to summarize performance indicators used in telephone triage services research, and make recommendations for the selection of valid indicators to measure the performance of telephone triage. We describe what kind of frameworks, performance indicators, or variables have been used for evaluating telephone triage performance by systematically mapping the telephone triage performance measurement. The objective was to find measures for each Triple Aim dimension. DESIGN: A scoping review method was used following Joanna Briggs Institute guidelines. Using this method, we defined indicators to measure the performance of telephone triage. We used the Triple Aim framework to identify indicators to measure the overall performance of telephone triage. The Triple Aim framework consists of improving the patient experience of care, improving the health of populations, and reducing cost per capita. SETTING: The scoping review was performed using CINAHL, Medline, EBSCOhost, and PubMed electronic databases. The eligibility criterion was research published in English between 2015 and 2023. The inclusion focused on the use and performance of telephone triage services and system-focused studies. RESULTS: A total of 1098 papers were screened for inclusion, with 57 papers included in our review. We identified 13 performance indicators covering all Triple Aim dimensions: waiting times, access, patient satisfaction, the accuracy of triage decision, severity and urgency of the symptoms, triage response, patient compliance with the advice given, follow-up healthcare service use, and running costs of service. We didn't find any earlier framework covering all Triple Aim dimensions properly. CONCLUSIONS: Measuring the performance of telephone triage requires an extensive and comprehensive approach. We presented performance indicators that may be included in the framework for measuring the performance of telephone triage to support overall performance measurements of telephone triage.

Assessing public health sector performance in Bangladesh: Implications for health systems strengthening and universal health coverage - A secondary data analysis.

The Ministry of Health and Family Welfare has established a health systems strengthening initiative for measuring the performance of public sector health facilities in Bangladesh. The objective of the performance management initiative is to establish routine systems for measuring and scoring health facility performance and promote best practices in public health service management. The performance initiative includes a set of assessments conducted across the four tiers of the public health sector. The findings of assessments demonstrate improvements in the quality of health services and a sharp increase in the utilisation of services across all tiers during the period 2017-2019. The performance management initiative has also identified areas for improvement in the supply-side health system readiness, including ensuring an adequate supply of human resources, essential medicines, and functioning medical equipment and technologies. This initiative outlines the need to systematically address the issue of high health workforce vacancy rates through effective human resource planning and management strategies. The reporting of these ongoing health systems successes and challenges through the performance management initiative in Bangladesh provides an opportunity to develop evidence-based policy reforms for strengthening supply-side health systems. The initiative results, particularly in the context of growing public demand for services, also justifies a monitoring and evaluation mechanism focusing on the quality and coverage of frontline health facilities and the development of more integrated health systems. The performance management initiative will facilitate the maintenance of essential health services while addressing emergency health needs and tracking progress towards achieving the Universal Health Coverage goal.

Kinematic and Physiological Analysis of Medieval Combat Sport Using Motion Analysis, Blood Lactate Measurement, and Heart Rate Monitoring: A Case Study.

Medieval combat sport is a form of mixed martial art in which combatants engage in fighting using offensive and defensive equipment while dressed in full armor. The sport is considered extremely taxing, making it nearly impossible to maintain the same level of performance. However, this form of sport has not been thoroughly analyzed, and its impact on human physical response is largely unknown. To address this gap, the study reported here aimed to introduce and test a procedure for analyzing human physical responses within the framework of the sport. To accomplish this, two experienced combatants were asked to engage in a series of strikes, performed in the form of a set duel simulating a professional fight competition. The kinematic aspect of the procedure was examined using motion analysis with the help of an IMU suit, while the physiological aspect was evaluated based on blood lactate levels and heart rate measurements. Furthermore, an ergometer test conducted in a laboratory setting aimed to determine the lactate threshold. The duel results showed noticeable decreases in the kinematic aspects of the strikes, such as the velocity of impact, and a dramatic rise in physiological aspects, such as heart rate and blood lactate levels. During the duel sets, the blood lactate surpassed the threshold level, and at the end, the heart rate exceeded the maximum age-related level. Practicing medieval combat sport has been shown to impose an extreme physical load on the bodies of combatants, noticeably affecting their performance levels.

A mixed-method approach to examining the experiences of allied health clinicians and managers of the performance, appraisal and development framework in a regional health care service.

INTRODUCTION: Evaluating employee performance is important for organisational success however, limited research exists across the allied health workforce. OBJECTIVE: This study investigated allied health clinician and manager experiences and perceptions of the appropriateness of the performance and development (PAD) framework. DESIGN: A mixed methods design with purposive sampling was conducted in a medium size hospital and health service in regional Queensland to investigate the appropriateness of the PAD framework. A 29 item online survey collected likert scale and open ended responses. Identified respondents also completed a semi-structured interview. Aggregated satisfaction scores identifying PAD alignment, processes, education, resources and perceptions were analysed to identify organisational PAD appropriateness. Themes from interviews provided concurrent experiential data. FINDINGS: Eighty-five respondents completed the online survey and 20 interviews were conducted. A Chi-Square Goodness of Fit test revealed a significant (p < 0.05) clinician (n = 75) response of moderate dissatisfaction (36%) for the PAD framework, while managers (n = 10) were neither dissatisfied nor satisfied (50%) or moderately satisfied (40%). Clinician aggregated agreement responses were significantly (p < 0.05) higher for PAD alignment (45%), processes (63%), education (49%) and lower for feedback (33%) and resources (28%). Manager aggregated agreement responses were high for PAD alignment (85%), processes (55%), education (57%), feedback (65%) and perceptions (50%). Interviews (n = 20) revealed three themes: uncertainty around PAD framework; expectations for experienced leadership and importance of discipline specific knowledge. DISCUSSION: This study draws on both clinician and manager input to assess the overall PAD framework with responses favouring clinician dissatisfaction to the current process. CONCLUSION: The PAD framework is valuable for staff development when there is positive rapport and discipline alignment between both clinician and manager. Successful staff appraisals require efficient and appropriate documentation that promotes individual development, clinical performance and organisational priorities.

Between-Hospital and Between-Physician Variation in Outcomes and Costs in High- and Low-Complex Surgery: A Nationwide Multilevel Analysis.

OBJECTIVES: Clinicians and policy makers are increasingly exploring strategies to reduce unwarranted variation in outcomes and costs. Adequately accounting for case mix and better insight into the levels at which variation exists is crucial for such strategies. This nationwide study investigates variation in surgical outcomes and costs at the level of hospitals and individual physicians and evaluates whether these can be reliably compared on performance. METHODS: Variation was analyzed using 92 330 patient records collected from 62 Dutch hospitals who underwent surgery for colorectal cancer (n = 6640), urinary bladder cancer (n = 14 030), myocardial infarction (n = 31 870), or knee osteoarthritis (n = 39 790) in the period 2018 to 2019. Multilevel regression modeling with and without case-mix adjustment was used to partition variation in between-hospital and between-physician components for in-hospital mortality, intensive care unit admission, length of stay, 30-day readmission, 30-day reintervention, and in-hospital costs. Reliability was calculated for each treatment-outcome combination at both levels. RESULTS: Across outcomes, hospital-level variation relative to total variation ranged between ≤ 1% and 15%, and given the high caseloads, this typically yielded high reliability (> 0.9). In contrast, physician-level variation components were typically ≤ 1%, with limited opportunities to make reliable comparisons. The impact of case-mix adjustment was limited, but nonnegligible. CONCLUSIONS: It is not typically possible to make reliable comparisons among physicians due to limited partitioned variation and low caseloads. Nevertheless, for hospitals, the opposite often holds. Although variation-reduction efforts directed at hospitals are thus more likely to be successful, this should be approached cautiously, partly because level-specific variation and the impact of case mix vary considerably across treatments and outcomes.

Optimizing Sanitation Network Upgrading Projects in Slum Areas.

Infrastructure upgrading projects are a key element in enhancing the livelihood of residents in slum areas. These projects face significant constructability challenges common to dense-urban construction coupled with the unique socioeconomic challenges of operating in slums. This research focuses on sanitation network upgrading projects in slum areas and proposes a novel methodology capable of (1) accounting for the unique constructability challenges for these projects, (2) accelerating the provision of sanitation services, and (3) optimizing construction decisions. The key contribution of this research to the body of knowledge is in developing a comprehensive construction planning framework capable of achieving these three objectives. The proposed framework focuses specifically on sewer lines upgrading within the larger sanitation networks upgrading projects. This framework consists of five main models that can guide planners in selecting the appropriate equipment sizes, trench system configuration, and optimal equipment routing, in addition to identifying all possible execution sequences along with the corresponding construction cost and duration of each sequence. Most notably, this framework proposes an approach to assess the serviceability of different construction plans measured by how fast sanitary services can be provided to slum dwellers. A multi-objective, genetic algorithms optimization model is developed to identify the optimal construction plans that accelerate the sanitary service provision to residents while minimizing construction costs. A real-world example is presented to demonstrate the model capabilities in optimizing construction plans.

Performance measurement of nonhomogeneous Hong Kong hospitals using directional distance functions.

Cook et al. (Oper Res 61(3):666-676, 2013) propose a DEA-based model for the performance evaluation of non-homogeneous decision making units (DMUs) based on constant returns to scale (CRS), extended by Li et al. (Health Care Manag Sci 22(2):215-228, 2019) to variable returns to scale (VRS). This paper locates these models into more general DDF models to deal with nonhomogeneous DMUs and applies these to Hong Kong hospitals. The production process of each hospital is divided into subunits which have the same inputs and outputs and hospital performance is measured using the subunits. The paper provides CRS and VRS versions of DDF models and compares them with Cook et al. (Oper Res 61(3):666-676, 2013) and Li et al. (Health Care Manag Sci 22(2):215-228, 2019). A kernel-based method is used to estimate the distributions as well as a DEA-based efficiency analysis adapted by Simar and Zelenyuk to test the distributions. Both DDF CRS and VRS versions produce results similar to Cook et al. (Oper Res 61(3):666-676, 2013) and Li et al. (Health Care Manag Sci 22(2):215-228, 2019) respectively. However, the statistical tests find differences for the different technologies assumed as would be expected. For hospital managers, the more generalised DDF models expand their range of options in terms of directional improvements and priorities as well as dealing with non-homogeneity.

Feedback on clinical team performance: how does it work, in what contexts, for whom, and for what changes? A critical realist qualitative multiple case study.

BACKGROUND: Feedback on clinical performance aims to provide teams in health care settings with structured results about their performance in order to improve these results. Two systematic reviews that included 147 randomized studies showed unresolved variability in professional compliance with desired clinical practices. Conventional recommendations for improving feedback on clinical team performance generally appear decontextualized and, in this regard, idealized. Feedback involves a complex and varied arrangement of human and non-human entities and interrelationships. To explore this complexity and improve feedback, we sought to explain how feedback on clinical team performance works, for whom, in what contexts, and for what changes. Our goal in this research was to present a realistic and contextualized explanation of feedback and its outcomes for clinical teams in health care settings. METHODS: This critical realist qualitative multiple case study included three heterogeneous cases and 98 professionals from a university-affiliated tertiary care hospital. Five data collection methods were used: participant observation, document retrieval, focus groups, semi-structured interviews, and questionnaires. Intra- and inter-case analysis performed during data collection involved thematic analysis, analytical questioning, and systemic modeling. These approaches were supported by critical reflexive dialogue among the research team, collaborators, and an expert panel. RESULTS: Despite the use of a single implementation model throughout the institution, results differed on contextual decision-making structures, responses to controversy, feedback loop practices, and use of varied technical or hybrid intermediaries. Structures and actions maintain or transform interrelationships and generate changes that are in line with expectations or the emergence of original solutions. Changes are related to the implementation of institutional and local projects or indicator results. However, they do not necessarily reflect a change in clinical practice or patient outcomes. CONCLUSIONS: This critical realist qualitative multiple case study offers an in-depth explanation of feedback on clinical team performance as a complex and open-ended sociotechnical system in constant transformation. In doing so, it identifies reflexive questions that are levers for the improvement of team feedback.

Measuring and improving performance of clinicians: an application of patient-based records.

BACKGOUND: Efforts to measure performance and identify its driving factors among clinicians are needed for building a high-quality clinician workforce. The availability of data is the most challenging thing. This paper presented a summary performance measure for clinicians and its application on examining factors that influence performance using routine patient-based records. METHODS: Perfomance indicators and difficulty score were extracted from electronic medical records (EMRs). Difficulty adjustment and standardized processing were used to obtain indicators which were comparable between specialties. Principal component analysis (PCA) was used to estimate the summary performance measure. The performance measure was then used to examine the influence of person-job fit and burnout through a mediator effect model and cluster analysis. RESULTS: A valid sample of 404 clinicians were included in this study, and 244 of them had valid response in the questionnaire. PCA explained 79.37% of the total variance presented by the four adjusted performance indicators. Non-performance attributes and performance driving factors help distinguish different clusters of clinicians. Burnout mediates the relationship between person-job fit and performance in a specific group of clinicians (ß = 0.120, p = 0.008). CONCLUSIONS: We demonstrated the analytical steps to estimate clinicians' performance and its practical application using EMRs. Our findings provide insight into personnel classified management. Such practice can be applied in countries where electronic medical record systems are relatively less developed to continuously improve the application of performance management.

Scaling Family Voices and Engagement to Measure and Improve Systems Performance and Whole Child Health: Progress and Lessons from the Child and Adolescent Health Measurement Initiative.

BACKGROUND: The 1997 legislation authorizing the United States Child Health Insurance Program sparked progress to measure and publicly report on children's healthcare services quality and system performance. To meet the moment, the national Child and Adolescent Health Measurement Initiative (CAHMI) public-private collaboration was launched to put families at the center of defining, measuring and using healthcare performance information to drive improved services quality and outcomes. METHODS: Since 1996 the CAHMI followed an intentional path of collaborative action to (1) articulate shared goals for child health and advance a comprehensive, life-course and outcomes-based healthcare performance measurement and reporting framework; (2) collaborate with families, providers, payers and government agencies to specify, validate and support national, state and local use of dozens of framework aligned measures; (3) create novel public-facing digital data query, collection and reporting tools that liberate data findings for use by families, providers, advocates, policymakers, the media and researchers (Data Resource Center, Well Visit Planner); and (4) generate field building research and systems change agendas and frameworks (Prioritizing Possibilities, Engagement In Action) to catalyze prevention, flourishing and healing centered, trauma-informed, whole child and family engaged approaches, integrated systems and supportive financing and policies. CONCLUSIONS: Lessons call for a restored, sustainable family and community engaged measurement infrastructure, public activation campaigns, and undeterred federal, state and systems leadership that implement policies to incentivize, resource, measure and remove barriers to integrated systems of care that scale family engagement to equitably promote whole child, youth and family well-being. Population health requires effective family engagement.

A Novel Inexpensive Camera-Based Photoelectric Barrier System for Accurate Flying Sprint Time Measurement.

This paper introduces a novel approach to addressing the challenge of accurately timing short distance runs, a critical aspect in the assessment of athletic performance. Electronic photoelectric barriers, although recognized for their dependability and accuracy, have remained largely inaccessible to non-professional athletes and smaller sport clubs due to their high costs. A comprehensive review of existing timing systems reveals that claimed accuracies beyond 30 ms lack experimental validation across most available systems. To bridge this gap, a mobile, camera-based timing system is proposed, capitalizing on consumer-grade electronics and smartphones to provide an affordable and easily accessible alternative. By leveraging readily available hardware components, the construction of the proposed system is detailed, ensuring its cost-effectiveness and simplicity. Experiments involving track and field athletes demonstrate the proficiency of the proposed system in accurately timing short distance sprints. Comparative assessments against a professional photoelectric cells timing system reveal a remarkable accuracy of 62 ms, firmly establishing the reliability and effectiveness of the proposed system. This finding places the camera-based approach on par with existing commercial systems, thereby offering non-professional athletes and smaller sport clubs an affordable means to achieve accurate timing. In an effort to foster further research and development, open access to the device's schematics and software is provided. This accessibility encourages collaboration and innovation in the pursuit of enhanced performance assessment tools for athletes.

Examining human factors and ergonomics aspects in a manufacturing organisation's metrics system: measuring up to stakeholder needs.

This research examines the status of human factors and ergonomics (HF/E) metrics in the case context of product realisation in an electronics manufacturing company. Interactions with 100+ stakeholders over a five year period were thematically analysed for metrics-related views and content. A disconnect between engineering metrics and HF/E metrics was evident. Engineers and HF/E specialists expressed different understandings of the gap between the disciplines and how to generate HF/E metrics that would fit the organisation. Other emerging themes provided insight for metrics development including improving indicator relatability, considerations for communication of information, and barriers to implementation of metrics. The results led to seven recommendations to help guide practitioners in developing and refining HF/E metrics as part of an organisation's metrics system. This macroergonomic case study provides key points for consideration when developing HF/E focussed metrics to support organisations being more proactive with HF/E in work system design. Practitioner summary: Metrics' presence, stakeholder views on metrics, and metrics-related content in a case organisation were thematically analysed with a macroergonomics focus. Human factors and ergonomics metrics (HF/E) were disconnected from engineering metrics thus limiting the design team's ability to handle human factors in design. Factors influencing HF/E metrics creation and integration were identified, resulting in seven recommendations for developing HF/E metrics.

Quality of acute ischemic stroke care at a tertiary Hospital in Ghana.

BACKGROUND: Information on the quality of acute ischemic stroke care provided in lower-to-middle income countries is limited. OBJECTIVE: This study was undertaken to examine the quality of acute ischemic stroke care provided at Tamale Teaching Hospital in Ghana. METHODS: The medical records of patients admitted into the medical ward of the hospital between January to October 2021 were reviewed retrospectively. Extent of compliance to 15 stroke performance indicators were determined. RESULTS: Under the study period, 105 patients were admitted at the hospital with acute ischemic stroke. The mean (±SD) age was 65 ± 12 years; 38.1% were males; 65.7% had National Health Insurance Scheme coverage. Glasgow Coma Scale was the only functional stroke rating scale used by physicians to rate stroke severity. About a quarter of the patients had CT scan performed within 24 h of admission. Less than a quarter of the patients had a last known well time documented. Rate of thrombolytic administration was 0%. Less than a quarter of the patients were prescribed venous thromboembolism prophylaxis on the day of admission or day after. Only 13.8% of patients had documented reasons for not being prescribed venous thromboembolism prophylaxis. Antiplatelet therapy was prescribed to 33.3% of the patients by the end of day 2 of admission. Anticoagulation was prescribed to all patients who had comorbid condition of atrial fibrillation as part of the discharge medications. More than half of the patients were discharged to go home with statin medications. Documented stroke education was provided to 31.4% caretakers or patients. Slightly less than half of the patients were assessed for or received rehabilitation. Less than a quarter had documented dysphagia screening within 24 h of admission. None of the patient had their stroke severity rated with National Institutes of Health Stroke Scale on arrival. No patient obtained carotid imaging assessment by end of day 2. CONCLUSION: There were several gaps in the quality of acute ischemic stroke care provided to patients at the Tamale Teaching Hospital. With the exception of discharging patients on statin medications, there was poor adherence to all other stroke performance indicators.

Rising through the pandemic: a scoping review of quality improvement in public health during the COVID-19 pandemic.

BACKGROUND: The COVID-19 pandemic generated a growing interest in and need for evidence-based tools to facilitate the implementation of emergency management strategies within public health practice. Quality improvement (QI) is a key framework and philosophy to guide organizational emergency response efforts; however, the nature and extent to which it has been used in public health settings during the COVID-19 pandemic remains unclear. METHODS: We conducted a scoping review of literature published January 2020 - February 2021 and focused on the topic of QI at public health agencies during the COVID-19 pandemic. The search was conducted using four bibliographic databases, in addition to a supplementary grey literature search through custom Google search engines and targeted website search methods. Of the 1,878 peer-reviewed articles assessed, 15 records met the inclusion criteria. An additional 11 relevant records were identified during the grey literature search, for a total of 26 records included in the scoping review. RESULTS: Records were organized into five topics: 1) collaborative problem solving and analysis with stakeholders; 2) supporting learning and capacity building in QI; 3) learning from past emergencies; 4) implementing QI methods during COVID-19; and 5) evaluating performance using frameworks/indicators. CONCLUSIONS: The literature indicates that QI-oriented activities are occurring at the organizational and program levels to enhance COVID-19 response. To optimize the benefits that QI approaches and methodologies may offer, it is important for public health agencies to focus on both widespread integration of QI as part of an organization's management philosophy and culture, as well as project level activities at all stages of the emergency management cycle.

Incentivizing performance in health care: a rapid review, typology and qualitative study of unintended consequences.

BACKGROUND: Health systems are increasingly implementing policy-driven programs to incentivize performance using contracts, scorecards, rankings, rewards, and penalties. Studies of these "Performance Management" (PM) programs have identified unintended negative consequences. However, no single comprehensive typology of the negative and positive unintended consequences of PM in healthcare exists and most studies of unintended consequences were conducted in England or the United States. The aims of this study were: (1) To develop a comprehensive typology of unintended consequences of PM in healthcare, and (2) To describe multiple stakeholder perspectives of the unintended consequences of PM in cancer and renal care in Ontario, Canada. METHODS: We conducted a rapid review of unintended consequences of PM in healthcare (n = 41 papers) to develop a typology of unintended consequences. We then conducted a secondary analysis of data from a qualitative study involving semi-structured interviews with 147 participants involved with or impacted by a PM system used to oversee 40 care delivery networks in Ontario, Canada. Participants included administrators and clinical leads from the networks and the government agency managing the PM system. We undertook a hybrid inductive and deductive coding approach using the typology we developed from the rapid review. RESULTS: We present a comprehensive typology of 48 negative and positive unintended consequences of PM in healthcare, including five novel unintended consequences not previously identified or well-described in the literature. The typology is organized into two broad categories: unintended consequences on (1) organizations and providers and on (2) patients and patient care. The most common unintended consequences of PM identified in the literature were measure fixation, tunnel vision, and misrepresentation or gaming, while those most prominent in the qualitative data were administrative burden, insensitivity, reduced morale, and systemic dysfunction. We also found that unintended consequences of PM are often mutually reinforcing. CONCLUSIONS: Our comprehensive typology provides a common language for discourse on unintended consequences and supports systematic, comparable analyses of unintended consequences across PM regimes and healthcare systems. Healthcare policymakers and managers can use the results of this study to inform the (re-)design and implementation of evidence-informed PM programs.

Development of a performance measurement system for general practitioners' office in China's primary healthcare.

BACKGROUND: General practitioners are the main providers of primary care services. To better strengthen the important role of general practitioners in primary healthcare services, China is promoting the general practitioners' office system. There is a lack of well-accepted methods to measure the performance of general practitioner offices in China. We thus aim to develop a systematic and operable performance measurement system for evaluating the general practitioner's office. METHODS: We establish an index pool of the performance measurement system of general practitioners' offices by a cross-sectional study and the literature research method and adopt the focus group method to establish the preliminary system. The Delphi method is then used to conduct three rounds of consultation to modify indices, which aims to form the final indicator system. We determine the weight of each index by the analytic hierarchy process method, which together with the final indicator system constitutes the final performance measurement system. Finally, we select three offices from three different cities in Sichuan Province, China, as case offices to conduct the case study, aiming to assess its credibility. RESULTS: Our results show that the first office scored 958.5 points, the second scored 768.1 points, and the third scored 947.7 points, which corresponds to the reality of these three offices, meaning that the performance measurement system is effective and manoeuvrable. CONCLUSIONS: Our study provides support for standardizing the functions of China's general practitioner's office, improving the health service quality of generalists, and providing a theoretical basis for the standardization of the general practitioner's office.

Big Data Health Care Innovations: Performance Dashboarding as a Process of Collective Sensemaking.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/16779.