Comparative effectiveness of two disparate policies on child health: experimental evidence from the Philippines.

BACKGROUND: Should health systems invest more in access to care by expanding insurance coverage or in health care services including improving the quality of care? Comparing these options experimentally would shed light on the impact and cost-effectiveness of these strategies. METHODS: The Quality Improvement Demonstration Study (QIDS) was a randomized policy experiment conducted across 30 districts in the Philippines. The study had a control group and two policy intervention groups intended to improve the health of young children. The demand-side intervention in QIDS was universal health insurance coverage (UHC) for children aged 5 years or younger, and a supply-side intervention, a pay-for-performance (P4P) bonus for all providers who met pre-determined quality levels. In this paper, we compare the impacts of these policies from the QIDS experiment on childhood wasting by calculating DALYs averted per US$spent. RESULTS: The direct per capita costs to implement UHC and P4P are US$4.08 and US$1.98 higher, respectively, compared to control. DALYs due to wasting were reduced by 334,862 in UHC and 1,073,185 in P4P. When adjustments are made for the efficiency of higher quality, the DALYS averted per US$ spent is similar in the two arms, 1.56 and 1.58 for UHC and P4P, respectively. Since the P4P quality improvements touches all patients seen by qualifying providers (32% in UHC versus 100% in P4P), there is a larger reduction in DALYs. With similar programmatic costs for either intervention, in this study, each US$spent under P4P yielded 1.52 DALYs averted compared to the standard program, while UHC yielded only a 0.50 DALY reduction. CONCLUSION: P4P had a greater impact and was more cost-effective compared to UHC as measured by DALYs averted. While expanded insurance benefit ceilings affected only those who are covered, P4P incentivizes practice quality improvement regardless of whether children are insured or uninsured.

Misdiagnosis of obstetrical cases and the clinical and cost consequences to patients: a cross-sectional study of urban providers in the Philippines.

BACKGROUND: Misdiagnosis may be a significant and under-recognized quality of care problem. In birthing facilities located in anurban Philippine setting, we investigated the diagnostic accuracy for three obstetric conditions: cephalopelvic disproportion (CPD), post-partum hemorrhage (PPH), and pre-eclampsia. DESIGN: Identical simulated cases were used to measure diagnostic accuracy for every provider (n=103). We linked misdiagnosis - identified by the simulated cases - to obstetrical complications of the patients at the participating facilities. Patient-level data on health outcomes and costs were obtained from medical records and follow-home in-person interviews. RESULTS: The prevalence of misdiagnosis among obstetric providers was 29.8% overall, 25% for CPD, 33% for PPH, and 31% for pre-eclampsia. Linking provider decision-making to patients, we found those who misdiagnosed the simulated cases were more likely to have patients with a complication (OR 2.96; 95% CI 1.39-3.77) compared with those who did not misdiagnose. Complicated patients were significantly less likely to be referred to a hospital immediately, were more likely to be readmitted to a hospital after delivery, had significantly higher medical costs, and lost more income than non-complicated patients. CONCLUSION: Diagnosis is arguably the most important task a clinician performs because it determines the subsequent course of evaluation and treatment, with the direct and indirect costs of diagnostic error, placing large financial burdens on the patient.

Do Health Reforms to Improve Quality Have Long-Term Effects? Results of a Follow-Up on a Randomized Policy Experiment in the Philippines.

We tracked doctors who had previously participated in a randomized policy experiment in the Philippines. The original experiment involved 30 district hospitals divided equally into one control site and two intervention sites that increased insurance payments (full insurance support for children under 5 years old) or made bonus payments to hospital staff. During the 3 years of the intervention, quality-as measured by clinical performance and value vignettes-improved and was sustained in both intervention sites compared with controls. Five years after the interventions were discontinued, we remeasured the quality of care of the doctors. We found that the intervention sites continued to have significantly higher quality compared with the control sites. The previously documented quality improvement in intervention sites appears to be sustained; moreover, it was subject to a very low (less than 1% per year) rate of decay in quality scores.

Public hospital care: equal for all or equal for some? Evidence from the Philippines.

In low- and middle-income countries, government budgets are rarely sufficient to cover a public hospital's operating costs. Shortfalls are typically financed through a combination of health insurance contributions and user charges. The mixed nature of this financing arrangement potentially creates financial incentives to treat patients with equal health need unequally. Using data from the Philippines, the authors analyzed whether doctors respond to such incentives. After controlling for a patient's condition, they found that patients using insurance, paying more for hospital accommodation, and being treated in externally monitored hospitals were likely to receive more care. This highlights the worrying possibility that public hospital patients with equal health needs are not always equally treated.

The impact of performance incentives on child health outcomes: results from a cluster randomized controlled trial in the Philippines.

Improving clinical performance using measurement and payment incentives, including pay for performance (or P4P), has, so far, shown modest to no benefit on patient outcomes. Our objective was to assess the impact of a P4P programme on paediatric health outcomes in the Philippines. We used data from the Quality Improvement Demonstration Study. In this study, the P4P intervention, introduced in 2004, was randomly assigned to 10 community district hospitals, which were matched to 10 control sites. At all sites, physician quality was measured using Clinical Performance Vignettes (CPVs) among randomly selected physicians every 6 months over a 36-month period. In the hospitals randomized to the P4P intervention, physicians received bonus payments if they met qualifying scores on the CPV. We measured health outcomes 4-10 weeks after hospital discharge among children 5 years of age and under who had been hospitalized for diarrhoea and pneumonia (the two most common illnesses affecting this age cohort) and had been under the care of physicians participating in the study. Health outcomes data collection was done at baseline/pre-intervention and 2 years post-intervention on the following post-discharge outcomes: (1) age-adjusted wasting, (2) C-reactive protein in blood, (3) haemoglobin level and (4) parental assessment of child's health using general self-reported health (GSRH) measure. To evaluate changes in health outcomes in the control vs intervention sites over time (baseline vs post-intervention), we used a difference-in-difference logistic regression analysis, controlling for potential confounders. We found an improvement of 7 and 9 percentage points in GSRH and wasting over time (post-intervention vs baseline) in the intervention sites relative to the control sites (P ≤ 0.001). The results from this randomized social experiment indicate that the introduction of a performance-based incentive programme, which included measurement and feedback, led to improvements in two important child health outcomes.

A comparison of objective biomarkers with a subjective health status measure among children in the Philippines.

UNLABELLED: Large health surveys use subjective (self-reported) and objective (biomarkers) measures to assess heath status. However, the linkage or disparity of these measures has not been systematically studied in developing countries. METHOD: Using data from the Philippine Quality Improvement Demonstration Study, QIDS, this study evaluated the associations between General Self-Reported Health Status (GSRH) and height, weight, hemoglobin, red blood cell folate, C-reactive protein, and blood lead levels. The authors modeled each biomarker as a function of GSRH controlling for socioeconomic status and selection effects. Changes in biomarkers and GSRH in children who had previously been hospitalized were also examined. RESULTS: GSRH independently predicted hemoglobin, C-reactive protein, stunting, and wasting. GSRH did not vary significantly with folate deficiency and blood lead levels. CONCLUSIONS: In addition to being a measure of overall child health status, GSRH may be a useful and inexpensive screening tool for identifying children that need further health testing.

Financial incentives and measurement improved physicians' quality of care in the Philippines.

The merits of using financial incentives to improve clinical quality have much appeal, yet few studies have rigorously assessed the potential benefits. The uncertainty surrounding assessments of quality can lead to poor policy decisions, possibly resulting in increased cost with little or no quality improvement, or missed opportunities to improve care. We conducted an experiment involving physicians in thirty Philippine hospitals that overcomes many of the limitations of previous studies. We measured clinical performance and then examined whether modest bonuses equal to about 5 percent of a physician's salary, as well as system-level incentives that increased compensation to hospitals and across groups of physicians, led to improvements in the quality of care. We found that both the bonus and system-level incentives improved scores in a quality measurement system used in our study by ten percentage points. Our findings suggest that when careful measurement is combined with the types of incentives we studied, there may be a larger impact on quality than previously recognized.

Pushing on a String: How policy might encourage private doctors to compete with the public sector on the basis of quality.

Results from the Philippine Quality Improvement Demonstration Study show that a policy that expands insurance coverage improves quality of care, as measured by clinical performance vignettes, among public physicians, and induces a spillover effect that improves quality among private physicians.

Understanding predictors of postdischarge deaths: a prospective evaluation of children 5 years and younger discharged from Philippine district hospitals.

BACKGROUND: Factors that increase likelihood of readmission or mortality postdischarge from diarrhea and pneumonia cases among children is less understood. METHODS: This study investigated the deaths of 24 children from a cohort of 3275. Using logistic regression, the authors compared data from those who survived with those who died to estimate the determinants of mortality in the study population. The authors also analyzed the hospital charts and completed mortality interviews with families of the deceased children. RESULTS: Poor quality of care significantly increased the likelihood of mortality. Sicker children, those born to less-educated mothers, and those who had longer lengths of stay also had a higher likelihood of mortality. Hospital charts corroborated findings from clinical vignettes. The mortality interviews revealed delays in seeking care from onset of symptoms. CONCLUSION: Quality of care contributes to postdischarge mortality and that clinical vignettes are an effective means to identify where quality can be improved.

Evidence of a causal link between health outcomes, insurance coverage, and a policy to expand access: experimental data from children in the Philippines.

In this paper, we present evidence on the health effects of a health insurance intervention targeted to poor children using data from a randomized policy experiment known as the Quality Improvement Demonstration Study. Among study participants, using a difference-in-difference regression model, we estimated a 9-12 and 4-9 percentage point reduction in the likelihood of wasting and having an infection, respectively, as measured by a common biomarker C-reactive Protein. Interestingly, these benefits were not apparent at the time of discharge; the beneficial health effects were manifest several weeks after release from the hospital.

Quality variation and its impact on costs and satisfaction: evidence from the QIDS study.

BACKGROUND: Improving the quality of inpatient hospital care is increasingly attainable in a variety of settings. However, the relationship between rising quality and costs is unclear; similarly the relationship between varying levels of quality and a patient's satisfaction remains poorly defined. METHODS: We use data from the Quality Improvement Demonstration Study (QIDS) based in 30 district hospitals in the Philippines. There were 974 children in the study; these children were cared for by 43 physicians. To measure quality of care, the physicians completed vignettes, a valid and inexpensive measure. Patient exit surveys were given to parents of children on the day of discharge, collecting information on services and hospital charges for the inpatient stay, payment sources for the hospitalization, and the Patient Satisfaction Survey (PSQ-18). RESULTS: We found a nonlinear relationship between quality and hospital charges: at low levels of quality improvements are linked to lower hospital charges. However, as quality improves further, these changes lead to higher charges. Higher quality also demonstrated a similar nonlinear relationship with patient satisfaction. CONCLUSIONS: The U-shaped association between quality and hospital charges suggests that targeting the lowest quality providers may decrease costs. The similar relationship between patient-reported satisfaction and quality improvement suggests that investments in quality will raise satisfaction, perhaps even when charges are increased.

An unhealthy public-private tension: pharmacy ownership, prescribing, and spending in the Philippines.

Physicians' links with pharmacies may create perverse financial incentives to overprescribe, prescribe products with higher profit margins, and direct patients to their pharmacy. Interviews with pharmacy customers in the Philippines show that those who use pharmacies linked to public-sector physicians had 5.4 greater odds of having a prescription from such physicians and spent 49.3 percent more than customers using other pharmacies. For customers purchasing brand-name medicines, switching to generics would reduce drug spending by 58 percent. Controlling out-of-pocket spending on drugs requires policies to control financial links between doctors and pharmacies, as well as tighter regulation of nongeneric prescribing.

An evaluation of the cost-effectiveness of policy navigators to improve access to care for the poor in the Philippines.

OBJECTIVES: Even when health insurance coverage is available, health policies may not be effective at increasing coverage among vulnerable populations. New approaches are needed to improve access to care. We experimentally introduced a novel intervention that uses Policy Navigators to increase health insurance enrollment in a poor population. METHODS: We used data from the Quality Improvement Demonstration Study (QIDS), a randomized experiment taking place at the district level in the Visayas region of the Philippines. In two arms of the study, we compared the effects of introducing Policy Navigators to controls. The Policy Navigators advocated for improved access to care by providing regular system-level expertise directly to the policy-makers, municipal mayors and governors responsible for paying for and enrolling poor households into the health insurance program. Using regression models, we compared levels of enrollment in our intervention versus control sites. We also assessed the cost-effectiveness of marginal increases in enrollment. RESULTS: We found that Policy Navigators improved enrollment in health insurance between 39% and 102% compared to the controls. Policy navigators were cost-effective at 0.86 USD per enrollee. However, supplementary national government campaigns, which were implemented to further increase coverage, attenuated normal enrollment efforts. CONCLUSION: Policy Navigators appear to be effective in improving access to care and their success underscores the importance of local-level strategies for improving enrollment.

The health and cost impact of care delay and the experimental impact of insurance on reducing delays.

OBJECTIVE: To examine whether delays in seeking care are associated with worse health outcomes or increased treatment costs in children, and then assess whether insurance coverage reduces these delays. STUDY DESIGN: We use data on 4070 children younger than 5 years from the Quality Improvement Demonstration Study, a randomized controlled experiment assessing the effects of increasing insurance coverage. We examined whether delay in care, defined as greater than 2 days between the onset of symptoms and admission to the study district hospitals, is associated with wasting or having positive C-reactive protein levels on discharge, and with total charge for hospital admission; we also evaluated whether increased benefit coverage and enrollment reduced the likelihood of delay. RESULTS: Delay is associated with 4.2% and 11.2% percentage point increases in the likelihood of wasting (P = .08) and having positive C-reactive protein levels (P = .03), respectively, at discharge. On average, hospitalization costs were 1.9% higher with delay (P = .04). Insurance intervention results in 5 additional children in 100 not delaying going to the hospital (P = .02). CONCLUSIONS: In this population, delayed care is associated with worse health outcomes and higher costs. Access to insurance reduced delays; thus insurance interventions may have positive effects on health outcomes.

A novel method for measuring health care system performance: experience from QIDS in the Philippines.

OBJECTIVES: Measuring and monitoring health system performance is important albeit controversial. Technical, logistic and financial challenges are formidable. We introduced a system of measurement, which we call Q, to measure the quality of hospital clinical performance across a range of facilities. This paper describes how Q was developed, implemented in hospitals in the Philippines and how it compares with typical measures. METHODS: Q consists of measures of clinical performance, patient satisfaction and volume of physician services. We evaluate Q using experimental data from the Quality Improvement Demonstration Study (QIDS), a randomized policy experiment. We determined its responsiveness over time and to changes in structural measures such as staffing and supplies. We also examined the operational costs of implementing Q. RESULTS: Q was sustainable, minimally disruptive and readily grafted into existing routines in 30 hospitals in 10 provinces semi-annually for a period of 2(1/2) years. We found Q to be more responsive to immediate impacts of policy change than standard structural measures. The operational costs totalled USD2133 or USD305 per assessment per site. CONCLUSION: Q appears to be an achievable assessment tool that is a comprehensive and responsive measure of system level quality at a limited cost in resource-poor settings.

Underutilization of social insurance among the poor: evidence from the Philippines.

BACKGROUND: Many developing countries promote social health insurance as a means to eliminate unmet health needs. However, this strategy may be ineffective if there are barriers to fully utilizing insurance. METHODS: We analyzed the utilization of social health insurance in 30 hospital districts in the central regions of the Philippines between 2003 and 2007. Data for the study came from the Quality Improvement Demonstration Study (QIDS) and included detailed patient information from exit interviews of children under 5 years of age conducted in seven waves among public hospital districts located in the four central regions of the Philippines. These data were used to estimate and identify predictors of underutilization of insurance benefits--defined as the likelihood of not filing claims despite having legitimate insurance coverage--using logistic regression. RESULTS: Multivariate analyses using QIDS data from 2004 to 2007 reveal that underutilization averaged about 15% throughout the study period. Underutilization, however, declined over time. Among insured hospitalized children, increasing length of stay in the hospital and mother's education, were associated with less underutilization. Being in a QIDS intervention site was also associated with less underutilization and partially accounts for the downward trend in underutilization over time. DISCUSSION: The surprisingly high level of insurance underutilization by insured patients in the QIDS sites undermines the potentially positive impact of social health insurance on the health of the marginalized. In the Philippines, where the largest burden of health care spending falls on households, underutilization suggests ineffective distribution of public funds, failing to reach a significant proportion of households which are by and large poor. Interventions that improve benefit awareness may combat the problem of underutilization and should be the focus of further research in this area.

Should we have confidence if a physician is accredited? A study of the relative impacts of accreditation and insurance payments on quality of care in the Philippines.

It is unclear whether health provider accreditation ensures or promotes quality of care. Using baseline data from the Quality Improvement Demonstration Study (QIDS) in the Philippines we measured the quality of pediatric care provided by private and public doctors working at the district hospital level in the country's central region. We found that national level accreditation by a national insurance program influences quality of care. However, our data also show that insurance payments have a similar, strong impact on quality of care. These results suggest that accreditation alone may not be sufficient to promote high quality of care. Further improvements may be achieved with properly monitored and well-designed payment or incentive schemes.

The Quality Improvement Demonstration Study: an example of evidence-based policy-making in practice.

BACKGROUND: Randomized trials have long been the gold-standard for evaluating clinical practice. There is growing recognition that rigorous studies are similarly needed to assess the effects of policy. However, these studies are rarely conducted. We report on the Quality Improvement Demonstration Study (QIDS), an example of a large randomized policy experiment, introduced and conducted in a scientific manner to evaluate the impact of large-scale governmental policy interventions. METHODS: In 1999 the Philippine government proposed sweeping reforms in the National Health Sector Reform Agenda. We recognized the unique opportunity to conduct a social experiment. Our ongoing goal has been to generate results that inform health policy. Early on we concentrated on developing a multi-institutional collaborative effort. The QIDS team then developed hypotheses that specifically evaluated the impact of two policy reforms on both the delivery of care and long-term health status in children. We formed an experimental design by randomizing matched blocks of three communities into one of the two policy interventions plus a control group. Based on the reform agenda, one arm of the experiment provided expanded insurance coverage for children; the other introduced performance-based payments to hospitals and physicians. Data were collected in household, hospital-based patient exit, and facility surveys, as well as clinical vignettes, which were used to assess physician practice. Delivery of services and health status were evaluated at baseline and after the interventions were put in place using difference-in-difference estimation. RESULTS: We found and addressed numerous challenges conducting this study, namely: formalizing the experimental design using the existing health infrastructure; securing funding to do research coincident with the policy reforms; recognizing biases and designing the study to account for these; putting in place a broad data collection effort to account for unanticipated findings; introducing sustainable policy interventions based on the reform agenda; and providing results in real-time to policy makers through a combination of venues. CONCLUSION: QIDS demonstrates that a large, prospective, randomized controlled policy experiment can be successfully implemented at a national level as part of sectoral reform. While we believe policy experiments should be used to generate evidence-based health policy, to do this requires opportunity and trust, strong collaborative relationships, and timing. This study nurtures the growing attitude that translation of scientific findings from the bedside to the community can be done successfully and that we should raise the bar on project evaluation and the policy-making process.

Associations between cognitive function, blood lead concentration, and nutrition among children in the central Philippines.

OBJECTIVE: Because little is known about its effects on cognitive function among children in less-developed countries, we determined the impact of lead exposure from other nutritional determinants of cognitive ability. STUDY DESIGN: Data were from a cross-sectional population-based stratified random sample of 877 children (age 6 months-5 years) participating in the Quality Improvement Demonstration Study we are conducting in the Philippines. With data from validated psychometric instruments, venous blood samples, and comprehensive survey instruments, we developed multi-stage models to account for endogenous determinants of blood lead levels (BLLs) and exogenous confounders of the association between BLLs and cognitive function. RESULTS: A 1 microg/dL increase in BLL was associated with a 3.32 point decline in cognitive functioning in children aged 6 months to 3 years and a 2.47 point decline in children aged 3 to 5 years olds. BLL was inversely associated with hemoglobin and folate levels. Higher folate levels mitigated the negative association between BLL and cognitive function. CONCLUSIONS: These population-based data suggest greater lead toxicity on cognitive function than previously reported. Our findings also suggest that folate and iron deficient children are more susceptible to the negative cognitive effects of lead. Folate supplementation may offer some protective effects against lead exposure.

Elevated blood-lead levels among children living in the rural Philippines.

OBJECTIVE: Generally, lead poisoning is not considered a significant environmental hazard for children in rural areas of developing countries. With a prospectively designed policy experiment, the research community and the government are conducting a broad-based investigation to introduce and evaluate the impact of health policy reforms on children in a rural area of the Philippines - the Quality Improvement Demonstration Study (QIDS). As part of this study, we researched lead exposure in children under the age of five. METHODS: We sampled a population of children from the Visayas region in the central Philippines, covering approximately one third of the country's geographical area. From December 2003 to September 2004, the survey collected blood lead levels (BLL) together with demographic, socioeconomic and child health data points. Supplemental field-testing among a sub-sample of the most exposed children assessed the sources of environmental lead exposure. FINDINGS: Among children in this study, 21% (601 of 2861 children) had BLL greater than 10 microg/dl. BLL were associated independently with age, haemoglobin concentration, water source, roofing material, expenditures and history of breastfeeding. A follow-up assessment of possible environmental exposures among the sub-sample of children with elevated BLL revealed no single or predominant exposure source. Instead, there appear to be multiple potential sources, such as fossil-fuel combustion, lead paint (in or around 38% of homes) and household items. CONCLUSION: Elevated BLL are common among children in the Visayas, and may signify an under-recognized threat to children living in rural areas of other developing nations. This setting has varied environmental sources of lead. Observed correlates of BLL may be of clinical, environmental and public health utility to identify and mitigate the consequences of lead toxicity.