Trends in health worker performance after implementing the Integrated Management of Childhood Illness strategy in Benin.

OBJECTIVE: Training health workers to use Integrated Management of Childhood Illness (IMCI) guidelines can improve care for ill children in outpatient settings in developing countries. However, even after IMCI training, important performance gaps exist. One potential reason is that the effect of training can rapidly wane. Our aim was to determine if the performance of IMCI-trained health workers deteriorated over 3 years. METHODS: We studied two departments in Benin. First, we performed a record review of 32 IMCI-trained health workers during the first year of IMCI implementation (2001-2002). Second, we analysed data from cross-sectional health facility surveys from 2001 to 2004 that represented the entire study area. Primary outcomes were the proportion of children under 5 years old with potentially life-threatening illnesses who received either recommended or adequate treatment, and among all children, an index of overall guideline adherence. Secondary outcomes reflected the treatment of individual diseases. Outcomes were calculated monthly, and time trends were evaluated with regression modelling. RESULTS: The record review included 9393 consultations, and the surveys included 411 consultations performed by 105 health workers. For both data sources, performance trends were essentially flat for nearly all outcomes. Absolute levels of performance revealed substantial performance gaps. CONCLUSIONS: We found no evidence that performance declined over 3 years after IMCI training. However, important performance gaps found immediately after IMCI training persisted and should be addressed.

Impact of a malaria-control project in Benin that included the integrated management of childhood illness strategy.

OBJECTIVES: To estimate the impact of the Integrated Management of Childhood Illness (IMCI) strategy on early-childhood mortality, we evaluated a malaria-control project in Benin that implemented IMCI and promoted insecticide-treated nets (ITNs). METHODS: We conducted a before-and-after intervention study that included a nonrandomized comparison group. We used the preceding birth technique to measure early-childhood mortality (risk of dying before age 30 months), and we used health facility surveys and household surveys to measure process indicators. RESULTS: Most process indicators improved in the area covered by the intervention. Notably, because ITNs were also promoted in the comparison area children's ITN use increased by about 20 percentage points in both areas. Regarding early-childhood mortality, the trend from baseline (1999-2001) to follow-up (2002-2004) for the intervention area (13.0% decrease; P < .001) was 14.1% (P < .001) lower than was the trend for the comparison area (1.3% increase; P = .46). CONCLUSIONS: Mortality decreased in the intervention area after IMCI and ITN promotion. ITN use increased similarly in both study areas, so the mortality impact of ITNs in the 2 areas might have canceled each other out. Thus, the mortality reduction could have been primarily attributable to IMCI's effect on health care quality and care-seeking.

A multifaceted intervention to improve health worker adherence to integrated management of childhood illness guidelines in Benin.

OBJECTIVES: We evaluated an intervention to support health workers after training in Integrated Management of Childhood Illness (IMCI), a strategy that can improve outcomes for children in developing countries by encouraging workers' use of evidence-based guidelines for managing the leading causes of child mortality. METHODS: We conducted a randomized trial in Benin. We administered a survey in 1999 to assess health care quality before IMCI training. Health workers then received training plus either study supports (job aids, nonfinancial incentives, and supervision of workers and supervisors) or usual supports. Follow-up surveys conducted in 2001 to 2004 assessed recommended treatment, recommended or adequate treatment, and an index of overall guideline adherence. RESULTS: We analyzed 1244 consultations. Performance improved in both intervention and control groups, with no significant differences between groups. However, training proceeded slowly, and low-quality care from health workers without IMCI training diluted intervention effects. Per-protocol analyses revealed that workers with IMCI training plus study supports provided better care than did those with training plus usual supports (27.3 percentage-point difference for recommended treatment; P < .05), and both groups outperformed untrained workers. CONCLUSIONS: IMCI training was useful but insufficient. Relatively inexpensive supports can lead to additional improvements.

Improving pneumonia case-management in Benin: a randomized trial of a multi-faceted intervention to support health worker adherence to Integrated Management of Childhood Illness guidelines.

BACKGROUND: Pneumonia is a leading cause of death among children under five years of age. The Integrated Management of Childhood Illness strategy can improve the quality of care for pneumonia and other common illnesses in developing countries, but adherence to these guidelines could be improved. We evaluated an intervention in Benin to support health worker adherence to the guidelines after training, focusing on pneumonia case management. METHODS: We conducted a randomized trial. After a health facility survey in 1999 to assess health care quality before Integrated Management of Childhood Illness training, health workers received training plus either study supports (job aids, non-financial incentives and supervision of workers and supervisors) or "usual" supports. Follow-up surveys were conducted in 2001, 2002 and 2004. Outcomes were indicators of health care quality for Integrated Management-defined pneumonia. Further analyses included a graphical pathway analysis and multivariable logistic regression modelling to identify factors influencing case-management quality. RESULTS: We observed 301 consultations of children with non-severe pneumonia that were performed by 128 health workers in 88 public and private health facilities. Although outcomes improved in both intervention and control groups, we found no statistically significant difference between groups. However, training proceeded slowly, and low-quality care from untrained health workers diluted intervention effects. Per-protocol analyses suggested that health workers with training plus study supports performed better than those with training plus usual supports (20.4 and 19.2 percentage-point improvements for recommended treatment [p=0.08] and "recommended or adequate" treatment [p=0.01], respectively). Both groups tended to perform better than untrained health workers. Analyses of treatment errors revealed that incomplete assessment and difficulties processing clinical findings led to missed pneumonia diagnoses, and missed diagnoses led to inadequate treatment. Increased supervision frequency was associated with better care (odds ratio for recommended treatment=2.1 [95% confidence interval: 1.13.9] per additional supervisory visit). CONCLUSION: Integrated Management of Childhood Illness training was useful, but insufficient, to achieve high-quality pneumonia case management. Our study supports led to additional improvements, although large gaps in performance still remained. A simple graphical pathway analysis can identify specific, common errors that health workers make in the case-management process; this information could be used to target quality improvement activities, such as supervision (ClinicalTrials.gov number NCT00510679).

Predictors of treatment error for children with uncomplicated malaria seen as outpatients in Blantyre district, Malawi.

BACKGROUND: Past studies have shown that health workers in developing countries often do not follow clinical guidelines, though few studies have explored with appropriate methods why errors occur. To develop interventions that improve health worker performance, factors affecting treatment practices must be better understood. METHODS: We analysed data from a health facility survey in Blantyre District, Malawi, in which health workers were observed treating ill children, and then children were independently re-examined by 'gold-standard' study clinicians. The analysis was limited to children with uncomplicated malaria (defined according to Malawi's guidelines as fever or anaemia without signs of severe illness), and a treatment error was defined as failure to treat with an effective antimalarial. RESULTS: Twenty-eight health workers and 349 ill-child consultations were evaluated; 247 (70.8%) children were treated with an effective antimalarial, and 102 (29.2%) were subject to treatment error. Logistic regression analysis revealed that in-service malaria training was not associated with treatment quality (univariate odds ratio (OR) = 1.16, 95% confidence interval (CI): 0.46-2.93); whereas acute respiratory infections training was associated with making an error (adjusted OR (aOR) = 2.42, 95% CI: 1.23-4.76). High fever and chief complaint of fever were associated with fewer errors (aOR = 0.25, 95% CI: 0.10-0.60 and aOR = 0.25, 95% CI: 0.13-0.48, respectively). Errors were more likely to occur in consultations starting before 1 p.m. (aOR = 1.88, 95% CI: 1.07-3.31). Supervision was not associated with better treatment quality. CONCLUSIONS: These results suggest that the disease-specific training and supervision, performed before the survey, did not lead to long-term improvements in health care quality. Furthermore, case management training for one specific disease may have worsened quality of care for another disease. These results support integration of guidelines for multiple conditions. Interventions should be evaluated for unintended negative effects on overall quality of care.