Tracking health system performance in times of crisis using routine health data: lessons learned from a multicountry consortium.

COVID-19 has prompted the use of readily available administrative data to track health system performance in times of crisis and to monitor disruptions in essential healthcare services. In this commentary we describe our experience working with these data and lessons learned across countries. Since April 2020, the Quality Evidence for Health System Transformation (QuEST) network has used administrative data and routine health information systems (RHIS) to assess health system performance during COVID-19 in Chile, Ethiopia, Ghana, Haiti, Lao People's Democratic Republic, Mexico, Nepal, South Africa, Republic of Korea and Thailand. We compiled a large set of indicators related to common health conditions for the purpose of multicountry comparisons. The study compiled 73 indicators. A total of 43% of the indicators compiled pertained to reproductive, maternal, newborn and child health (RMNCH). Only 12% of the indicators were related to hypertension, diabetes or cancer care. We also found few indicators related to mental health services and outcomes within these data systems. Moreover, 72% of the indicators compiled were related to volume of services delivered, 18% to health outcomes and only 10% to the quality of processes of care. While several datasets were complete or near-complete censuses of all health facilities in the country, others excluded some facility types or population groups. In some countries, RHIS did not capture services delivered through non-visit or nonconventional care during COVID-19, such as telemedicine. We propose the following recommendations to improve the analysis of administrative and RHIS data to track health system performance in times of crisis: ensure the scope of health conditions covered is aligned with the burden of disease, increase the number of indicators related to quality of care and health outcomes; incorporate data on nonconventional care such as telehealth; continue improving data quality and expand reporting from private sector facilities; move towards collecting patient-level data through electronic health records to facilitate quality-of-care assessment and equity analyses; implement more resilient and standardized health information technologies; reduce delays and loosen restrictions for researchers to access the data; complement routine data with patient-reported data; and employ mixed methods to better understand the underlying causes of service disruptions.

Effect of lifting COVID-19 restrictions on utilisation of primary care services in Nepal: a difference-in-differences analysis.

INTRODUCTION: An increasing number of studies have reported disruptions in health service utilisation due to the COVID-19 pandemic and its associated restrictions. However, little is known about the effect of lifting COVID-19 restrictions on health service utilisation. The objective of this study was to estimate the effect of lifting COVID-19 restrictions on primary care service utilisation in Nepal. METHODS: Data on utilisation of 10 primary care services were extracted from the Health Management Information System across all health facilities in Nepal. We used a difference-in-differences design and linear fixed effects regressions to estimate the effect of lifting COVID-19 restrictions. The treatment group included palikas that had lifted restrictions in place from 17 August 2020 to 16 September 2020 (Bhadra 2077) and the control group included palikas that had maintained restrictions during that period. The pre-period included the 4 months of national lockdown from 24 March 2020 to 22 July 2020 (Chaitra 2076 to Ashar 2077). Models included month and palika fixed effects and controlled for COVID-19 incidence. RESULTS: We found that lifting COVID-19 restrictions was associated with an average increase per palika of 57.5 contraceptive users (95% CI 14.6 to 100.5), 15.6 antenatal care visits (95% CI 5.3 to 25.9) and 1.6 child pneumonia visits (95% CI 0.2 to 2.9). This corresponded to a 9.4% increase in contraceptive users, 34.2% increase in antenatal care visits and 15.6% increase in child pneumonia visits. Utilisation of most other primary care services also increased after lifting restrictions, but coefficients were not statistically significant. CONCLUSIONS: Despite the ongoing pandemic, lifting restrictions can lead to an increase in some primary care services. Our results point to a causal link between restrictions and health service utilisation and call for policy makers in low- and middle-income countries to carefully consider the trade-offs of strict lockdowns during future COVID-19 waves or future pandemics.

Best and worst performing health facilities: A positive deviance analysis of perceived drivers of primary care performance in Nepal.

Primary care services are on average of low quality in Nepal. However, there is marked variation in performance of basic clinical and managerial functions between primary health care centers. The determinants of variation in primary care performance in low- and middle-income countries have been understudied relative to the prominence of primary care in national health plans. We used the positive deviance approach to identify best and worst performing primary health care centers in Nepal and investigated perceived drivers of best performance. We selected eight primary health care centers in Province 1, Nepal, using an index of basic clinical and operational activities to identify four best and four worst performing primary health care centers. We conducted semi-structured, in-depth interviews with managers and clinical staff from each of the eight primary health care centers for a total of 32 interviews. We identified the following factors that distinguished best from worst performers: 1) Managing the facility effectively, 2) engaging local leadership, 3) building active community accountability, 4) assessing and responding to facility performance, 5) developing sources of funding, 6) compensating staff fairly, 7) managing clinical staff performance, and 8) promoting uninterrupted availability of supplies and equipment. These findings can be used to inform quality improvement efforts and health system reforms in Nepal and other similarly under-resourced health systems.

COVID-19 and resilience of healthcare systems in ten countries.

Declines in health service use during the Coronavirus Disease 2019 (COVID-19) pandemic could have important effects on population health. In this study, we used an interrupted time series design to assess the immediate effect of the pandemic on 31 health services in two low-income (Ethiopia and Haiti), six middle-income (Ghana, Lao People's Democratic Republic, Mexico, Nepal, South Africa and Thailand) and high-income (Chile and South Korea) countries. Despite efforts to maintain health services, disruptions of varying magnitude and duration were found in every country, with no clear patterns by country income group or pandemic intensity. Disruptions in health services often preceded COVID-19 waves. Cancer screenings, TB screening and detection and HIV testing were most affected (26-96% declines). Total outpatient visits declined by 9-40% at national levels and remained lower than predicted by the end of 2020. Maternal health services were disrupted in approximately half of the countries, with declines ranging from 5% to 33%. Child vaccinations were disrupted for shorter periods, but we estimate that catch-up campaigns might not have reached all children missed. By contrast, provision of antiretrovirals for HIV was not affected. By the end of 2020, substantial disruptions remained in half of the countries. Preliminary data for 2021 indicate that disruptions likely persisted. Although a portion of the declines observed might result from decreased needs during lockdowns (from fewer infectious illnesses or injuries), a larger share likely reflects a shortfall of health system resilience. Countries must plan to compensate for missed healthcare during the current pandemic and invest in strategies for better health system resilience for future emergencies.

The price of quality care: cross-sectional associations between out-of-pocket payments and quality of care in six low-income countries.

OBJECTIVE: To assess the relationship between out-of-pocket (OOP) payments and primary health care quality in six low-income countries: Afghanistan, the Democratic Republic of the Congo (DRC), Haiti, Nepal, Senegal and Tanzania. METHODS: We examined the association between OOP payments and quality of care during antenatal care and sick child care visits using Service Provision Assessments data. We defined four process quality outcomes from observations of clinical care: visit duration, history-taking items asked, exam items performed, and counselling items delivered. The outcome is the total amount paid for services. We used multilevel models to test the relationship between OOP payments and each quality measure in public, private non-profit and private for-profit facilities controlling for patient, provider, and facility characteristics. RESULTS: Across the six countries, an average of 42% of the 29 677 observed clients paid for their visit. In the adjusted models, OOP payments were positively associated with the visit duration during sick child visits, with history-taking and exam items during antenatal care visits, and with counselling in private for-profit facilities for both visit types. These associations were strong particularly in Afghanistan, the DRC and Haiti; for example, a high-quality antenatal care visit in the DRC would cost approximately USD 1.12 more than a visit with median quality. CONCLUSION: Provider effort was associated with higher OOP payments for sick child and antenatal care services in the six countries studied. While many families are already spending high amounts on care, they must often spend even more to receive higher quality care.

Hospital-provision of essential primary care in 56 countries: determinants and quality.

OBJECTIVE: To estimate the use of hospitals for four essential primary care services offered in health centres in low- and middle-income countries and to explore differences in quality between hospitals and health centres. METHODS: We extracted data from all demographic and health surveys conducted since 2010 on the type of facilities used for obtaining contraceptives, routine antenatal care and care for minor childhood diarrhoea and cough or fever. Using mixed-effects logistic regression models we assessed associations between hospital use and individual and country-level covariates. We assessed competence of care based on the receipt of essential clinical actions during visits. We also analysed three indicators of user experience from countries with available service provision assessment survey data. FINDINGS: On average across 56 countries, public hospitals were used as the sole source of care by 16.9% of 126 012 women who obtained contraceptives, 23.1% of 418 236 women who received routine antenatal care, 19.9% of 47 677 children with diarrhoea and 18.5% of 82 082 children with fever or cough. Hospital use was more common in richer countries with higher expenditures on health per capita and among urban residents and wealthier, better-educated women. Antenatal care quality was higher in hospitals in 44 countries. In a subset of eight countries, people using hospitals tended to spend more, report more problems and be somewhat less satisfied with the care received. CONCLUSION: As countries work towards achieving ambitious health goals, they will need to assess care quality and user preferences to deliver effective primary care services that people want to use.