Measuring National Immunization System Performance: A Systematic Assessment of Available Resources.

BACKGROUND: Vaccination coverage is widely used to assess immunization performance but, on its own, provides insufficient information to drive improvements. Assessing the performance of underlying components of immunization systems is less clear, with several monitoring and evaluation (M&E) resources available for use in different operational settings and for different purposes. We studied these resources to understand how immunization system performance is measured. METHODS: We reviewed peer-reviewed and gray literature published since 2000 to identify M&E resources that include national-level indicators measuring the performance of immunization systems or their components (governance, financing, regulation, information systems, vaccine logistics, workforce, service delivery, and demand generation). We summarize indicators by the system components or outcomes measured and describe findings narratively. RESULTS: We identified 20 resources to monitor immunization program objectives and guide national strategic decision-making, encompassing 631 distinct indicators. Indicators for immunization program outcomes comprised the majority (124/631 [19.7%]), largely vaccination coverage (110/124 [88.7%]). Almost all resources (19/20 [95%]) included indicators for vaccine logistics (83/631 [13.2%]), and those for regulation (19/631 [3.0%]) and demand generation (28/631 [4.4%]) were least common. There was heterogeneity in how information systems (92/563 [14.6%]) and workforce (47/631 [7.4%]) were assessed across resources. Indicators for vaccination coverage in adults, data use in decision-making, equity and diversity, effectiveness of safety surveillance, and availability of a public health workforce were notably lacking. CONCLUSIONS: Between the resources identified in this review, we identified considerable variability and gaps in indicators assessing the performance of some immunization system components. Given the multitude of indicators, policymakers may be better served by tailoring evaluation resources to their specific context to gain useful insight into health system performance and improve data use in decision-making for immunization programs.

Common Features of Selection Processes of Health System Performance Indicators in Primary Healthcare: A Systematic Review.

BACKGROUND: Health system performance indicators are widely used to assess primary healthcare (PHC) performance. Despite the numerous tools and some convergence on indicator criteria, there is not a clear understanding of the common features of indicator selection processes. We aimed to review the literature to identify papers that document indicator selection processes for health system performance indicators in PHC. METHODS: We searched the online databases Scopus, Medline, and CINAHL, as well as the grey literature, without time restrictions, initially on July 31, 2019 followed by an update November 13, 2020. Empirical studies or reports were included if they described the selection of health system performance indicators or frameworks, that included PHC indicators. A combination of the process focussed research question and qualitative analysis meant a quality appraisal tool or assessment of bias could not meaningfully be applied to assess individual studies. We undertook an inductive analysis based on potential indicator selection processes criteria, drawn from health system performance indicator appraisal tools reported in the literature. RESULTS: We identified 16 503 records of which 28 were included in the review. Most studies used a descriptive case study design. We found no consistent variations between indicator selection processes of health systems of high income and low- or lower-middle income countries. Identified common features of selection processes for indicators in PHC include literature review or adaption of an existing framework as an initial step; a consensus building process with stakeholders; structuring indicators into categories; and indicator criteria focusing on validity and feasibility. The evidence around field testing with utility and consideration of reporting burden was less clear. CONCLUSION: Our findings highlight several characteristics of health system indicator selection processes. These features provide the groundwork to better understand how to value indicator selection processes in PHC.

Expert perspectives on priorities for supporting health security in the Pacific region through health systems strengthening.

The COVID-19 pandemic has reiterated the interdependence of health security and health systems, and the need for resilient health systems to prevent large-scale impacts of infectious disease outbreaks and other acute public health events. Three years into the COVID-19 pandemic has led to discussions on how to "build back better", making it important to identify lessons to strengthen health systems and prevent future shocks from health security threats. Limited data exist on effective implementable initiatives, especially for the Pacific region. We explored the perceptions of a selection of experts with field experience in the Pacific region to identify and prioritise areas for future health system investments that strengthen health security. We conducted a qualitative cross-sectional study, collecting data using four focus group discussions. We analysed the data using a content analysis of notes recorded from each of the sessions. There were 24 participants, representing 15 research and academic institutions, nongovernment agencies, UN agencies and government as well as independent consultants. All were health sector stakeholders with field experience in the Pacific region and expertise in either health systems or health security. The analysis revealed four areas to prioritise future efforts, namely workforce development, risk communication, public health surveillance and laboratory capacity. A fifth theme, localisation, was identified as a cross cutting theme that should be applied to implementation of other identified priority areas. These findings provide a starting point to apply in practice this relatively new concept, of targeted health systems strengthening for health security development, in the Pacific. Evaluation of these initiatives will strengthen knowledge on the value of integrating these two concepts.

Factors That Influence Data Use to Improve Health Service Delivery in Low- and Middle-Income Countries.

BACKGROUND: Health service delivery indicators are designed to reveal how well health services meet a community's needs. Effective use of the data can enable targeted improvements in health service delivery. We conducted a systematic review to identify the factors that influence the use of health service delivery indicators to improve delivery of primary health care services in low- and middle-income settings. METHODS: We reviewed empirical studies published in 2005 or later that provided evidence on the use of health service delivery data at the primary care level in low- and middle-income countries. We searched Scopus, Medline, the Cochrane Library, and citations of included studies. We also searched the gray literature, using a separate strategy. We extracted information on study design, setting, study population, study objective, key findings, and any identified lessons learned. RESULTS: Twelve studies met the inclusion criteria. This small number of studies suggests there is insufficient evidence to draw reliable conclusions. However, a content analysis identified the following potentially influential factors, which we classified into 3 categories: governance (leadership, participatory monitoring, regular review of data); production of information (presentation of findings, data quality, qualitative data); and health information system resources (electronic health management information systems, organizational structure, training). Contextual factors and performance-based financing were also each found to have a role; however, discussing these as mediating factors may not be practical in terms of promoting data use. CONCLUSION: Scant evidence exists regarding factors that influence the use of health service delivery indicators to improve delivery of primary health care services in low- and middle-income countries. However, the existing evidence highlights some factors that may have a role in improving data use. Further research may benefit from comparing data use factors across different types of program indicators or using our classification as a framework for field experiments.

Implementation of the Xpert MTB/RIF assay for tuberculosis in Mongolia: a qualitative exploration of barriers and enablers.

OBJECTIVE: The aim of our study was to identify barriers and enablers to implementation of the Xpert MTB/RIF test within Mongolia's National Tuberculosis Program. METHODS: Twenty-foursemi-structured interviews were conducted between June and September 2015 with laboratory staff and tuberculosis physicians in Mongolia's capital Ulaanbaatar and regional towns where Xpert MTB/RIF testing had been implemented. Interviews were recorded, transcribed, translated and analysed thematically using NVIVO qualitative analysis software. RESULTS: Eight laboratory staff (five from the National Tuberculosis Reference Laboratory in Ulaanbaatar and three from provincial laboratories) and sixteen tuberculosis physicians (five from the Mongolian National Center for Communicable Diseases in Ulaanbaatar, four from district tuberculosis clinics in Ulaanbaatar and seven from provincial tuberculosis clinics) were interviewed. Major barriers to Xpert MTB/RIF implementation identified were: lack of awareness of program guidelines; inadequate staffing arrangements; problems with cartridge supply management; lack of local repair options for the Xpert machines; lack of regular formal training; paper based system; delayed treatment initiation due to consensus meeting and poor sample quality. Enablers to Xpert MTB/RIF implementation included availability of guidelines in the local language; provision of extra laboratory staff, shift working arrangements and additional modules; capacity for troubleshooting internally; access to experts; opportunities for peer learning; common understanding of diagnostic algorithms and decentralised testing. CONCLUSION: Our study identified a number of barriers and enablers to implementation of Xpert MTB/RIF in the Mongolian National Tuberculosis Program. Lessons learned from this study can help to facilitate implementation of Xpert MTB/RIF in other Mongolian locations as well as other low-and middle-income countries.