The effectiveness of supervision strategies to improve health care provider practices in low- and middle-income countries: secondary analysis of a systematic review.

BACKGROUND: Although supervision is a ubiquitous approach to support health programs and improve health care provider (HCP) performance in low- and middle-income countries (LMICs), quantitative evidence of its effects is unclear. The objectives of this study are to describe the effect of supervision strategies on HCP practices in LMICs and to identify attributes associated with greater effectiveness of routine supervision. METHODS: We performed a secondary analysis of data on HCP practice outcomes (e.g., percentage of patients correctly treated) from a systematic review on improving HCP performance. The review included controlled trials and interrupted time series studies. We described distributions of effect sizes (defined as percentage-point [%-point] changes) for each supervision strategy. To identify attributes associated with supervision effectiveness, we performed random-effects linear regression modeling and examined studies that directly compared different approaches of routine supervision. RESULTS: We analyzed data from 81 studies from 36 countries. For professional HCPs, such as nurses and physicians, primarily working at health facilities, routine supervision (median improvement when compared to controls: 10.7%-points; IQR: 9.9, 27.9) had similar effects on HCP practices as audit with feedback (median improvement: 10.1%-points; IQR: 6.2, 23.7). Two attributes were associated with greater mean effectiveness of routine supervision (p < 0.10): supervisors received supervision (by 8.8-11.5%-points), and supervisors participated in problem-solving with HCPs (by 14.2-20.8%-points). Training for supervisors and use of a checklist during supervision visits were not associated with effectiveness. The effects of supervision frequency (i.e., number of visits per year) and dose (i.e., the number of supervision visits during a study) were unclear. For lay HCPs, the effect of routine supervision was difficult to characterize because few studies existed, and effectiveness in those studies varied considerably. Evidence quality for all findings was low primarily because many studies had a high risk of bias. CONCLUSIONS: Although evidence is limited, to promote more effective supervision, our study supports supervising supervisors and having supervisors engage in problem-solving with HCPs. Supervision's integral role in health systems in LMICs justifies a more deliberate research agenda to identify how to deliver supervision to optimize its effect on HCP practices.

How does the effectiveness of strategies to improve healthcare provider practices in low-income and middle-income countries change after implementation? Secondary analysis of a systematic review.

BACKGROUND: A recent systematic review evaluated the effectiveness of strategies to improve healthcare provider (HCP) performance in low-income and middle-income countries. The review identified strategies with varying effects, including in-service training, supervision and group problem-solving. However, whether their effectiveness changed over time remained unclear. In particular, understanding whether effects decay over time is crucial to improve sustainability. METHODS: We conducted a secondary analysis of data from the aforementioned review to explore associations between time and effectiveness. We calculated effect sizes (defined as percentage-point (%-point) changes) for HCP practice outcomes (eg, percentage of patients correctly treated) at each follow-up time point after the strategy was implemented. We estimated the association between time and effectiveness using random-intercept linear regression models with time-specific effect sizes clustered within studies and adjusted for baseline performance. RESULTS: The primary analysis included 37 studies, and a sensitivity analysis included 77 additional studies. For training, every additional month of follow-up was associated with a 0.19 %-point decrease in effectiveness (95% CI: -0.36 to -0.03). For training combined with supervision, every additional month was associated with a 0.40 %-point decrease in effectiveness (95% CI: -0.68 to -0.12). Time trend results for supervision were inconclusive. For group problem-solving alone, time was positively associated with effectiveness, with a 0.50 %-point increase in effect per month (95% CI: 0.37 to 0.64). Group problem-solving combined with training was associated with large improvements, and its effect was not associated with time. CONCLUSIONS: Time trends in the effectiveness of different strategies to improve HCP practices vary among strategies. Programmes relying solely on in-service training might need periodical refresher training or, better still, consider combining training with group problem-solving. Although more high-quality research is needed, these results, which are important for decision-makers as they choose which strategies to use, underscore the utility of studies with multiple post-implementation measurements so sustainability of the impact on HCP practices can be assessed.

The effectiveness of training strategies to improve healthcare provider practices in low-income and middle-income countries.

INTRODUCTION: In low/middle-income countries (LMICs), training is often used to improve healthcare provider (HCP) performance. However, important questions remain about how well training works and the best ways to design training strategies. The objective of this study is to characterise the effectiveness of training strategies to improve HCP practices in LMICs and identify attributes associated with training effectiveness. METHODS: We performed a secondary analysis of data from a systematic review on improving HCP performance. The review included controlled trials and interrupted time series, and outcomes measuring HCP practices (eg, percentage of patients correctly treated). Distributions of effect sizes (defined as percentage-point (%-point) changes) were described for each training strategy. To identify effective training attributes, we examined studies that directly compared training approaches and performed random-effects linear regression modelling. RESULTS: We analysed data from 199 studies from 51 countries. For outcomes expressed as percentages, educational outreach visits (median effect size when compared with controls: 9.9 %-points; IQR: 4.3-20.6) tended to be somewhat more effective than in-service training (median: 7.3 %-points; IQR: 3.6-17.4), which seemed more effective than peer-to-peer training (4.0 %-points) and self-study (by 2.0-9.3 %-points). Mean effectiveness was greater (by 6.0-10.4 %-points) for training that incorporated clinical practice and training at HCPs' work site. Attributes with little or no effect were: training with computers, interactive methods or over multiple sessions; training duration; number of educational methods; distance training; trainers with pedagogical training and topic complexity. For lay HCPs, in-service training had no measurable effect. Evidence quality for all findings was low. CONCLUSIONS: Although additional research is needed, by characterising the effectiveness of training strategies and identifying attributes of effective training, decision-makers in LMICs can improve how these strategies are selected and implemented.

The effectiveness of the quality improvement collaborative strategy in low- and middle-income countries: A systematic review and meta-analysis.

BACKGROUND: Quality improvement collaboratives (QICs) have been used to improve health care for decades. Evidence on QIC effectiveness has been reported, but systematic reviews to date have little information from low- and middle-income countries (LMICs). OBJECTIVE: To assess the effectiveness of QICs in LMICs. METHODS: We conducted a systematic review following Cochrane methods, the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach for quality of evidence grading, and the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement for reporting. We searched published and unpublished studies between 1969 and March 2019 from LMICs. We included papers that compared usual practice with QICs alone or combined with other interventions. Pairs of reviewers independently selected and assessed the risk of bias and extracted data of included studies. To estimate strategy effectiveness from a single study comparison, we used the median effect size (MES) in the comparison for outcomes in the same outcome group. The primary analysis evaluated each strategy group with a weighted median and interquartile range (IQR) of MES values. In secondary analyses, standard random-effects meta-analysis was used to estimate the weighted mean MES and 95% confidence interval (CI) of the mean MES of each strategy group. This review is registered with PROSPERO (International Prospective Register of Systematic Reviews): CRD42017078108. RESULTS: Twenty-nine studies were included; most (21/29, 72.4%) were interrupted time series studies. Evidence quality was generally low to very low. Among studies involving health facility-based health care providers (HCPs), for "QIC only", effectiveness varied widely across outcome groups and tended to have little effect for patient health outcomes (median MES less than 2 percentage points for percentage and continuous outcomes). For "QIC plus training", effectiveness might be very high for patient health outcomes (for continuous outcomes, median MES 111.6 percentage points, range: 96.0 to 127.1) and HCP practice outcomes (median MES 52.4 to 63.4 percentage points for continuous and percentage outcomes, respectively). The only study of lay HCPs, which used "QIC plus training", showed no effect on patient care-seeking behaviors (MES -0.9 percentage points), moderate effects on non-care-seeking patient behaviors (MES 18.7 percentage points), and very large effects on HCP practice outcomes (MES 50.4 percentage points). CONCLUSIONS: The effectiveness of QICs varied considerably in LMICs. QICs combined with other invention components, such as training, tended to be more effective than QICs alone. The low evidence quality and large effect sizes for QIC plus training justify additional high-quality studies assessing this approach in LMICs.

A systematic review of the effectiveness of strategies to improve health care provider performance in low- and middle-income countries: Methods and descriptive results.

BACKGROUND: Health care provider (HCP) performance in low- and middle-income countries (LMICs) is often inadequate. The Health Care Provider Performance Review (HCPPR) is a comprehensive systematic review of the effectiveness and cost of strategies to improve HCP performance in LMICs. We present the HCPPR's methods, describe methodological and contextual attributes of included studies, and examine time trends of study attributes. METHODS: The HCPPR includes studies from LMICs that quantitatively evaluated any strategy to improve HCP performance for any health condition, with no language restrictions. Eligible study designs were controlled trials and interrupted time series. In 2006, we searched 15 databases for published studies; in 2008 and 2010, we completed searches of 30 document inventories for unpublished studies. Data from eligible reports were double-abstracted and entered into a database, which is publicly available. The primary outcome measure was the strategy's effect size. We assessed time trends with logistic, Poisson, and negative binomial regression modeling. We were unable to register with PROSPERO (International Prospective Register of Systematic Reviews) because the protocol was developed prior to the PROSPERO launch. RESULTS: We screened 105,299 citations and included 824 reports from 499 studies of 161 intervention strategies. Most strategies had multiple components and were tested by only one study each. Studies were from 79 countries and had diverse methodologies, geographic settings, HCP types, work environments, and health conditions. Training, supervision, and patient and community supports were the most commonly evaluated strategy components. Only 33.6% of studies had a low or moderate risk of bias. From 1958-2003, the number of studies per year and study quality increased significantly over time, as did the proportion of studies from low-income countries. Only 36.3% of studies reported information on strategy cost or cost-effectiveness. CONCLUSIONS: Studies have reported on the efficacy of many strategies to improve HCP performance in LMICs. However, most studies have important methodological limitations. The HCPPR is a publicly accessible resource for decision-makers, researchers, and others interested in improving HCP performance.

Effectiveness of strategies to improve health-care provider practices in low-income and middle-income countries: a systematic review.

BACKGROUND: Inadequate health-care provider performance is a major challenge to the delivery of high-quality health care in low-income and middle-income countries (LMICs). The Health Care Provider Performance Review (HCPPR) is a comprehensive systematic review of strategies to improve health-care provider performance in LMICs. METHODS: For this systematic review we searched 52 electronic databases for published studies and 58 document inventories for unpublished studies from the 1960s to 2016. Eligible study designs were controlled trials and interrupted time series. We only included strategy-versus-control group comparisons. We present results of improving health-care provider practice outcomes expressed as percentages (eg, percentage of patients treated correctly) or as continuous measures (eg, number of medicines prescribed per patient). Effect sizes were calculated as absolute percentage-point changes. The summary measure for each comparison was the median effect size (MES) for all primary outcomes. Strategy effectiveness was described with weighted medians of MES. This study is registered with PROSPERO, number CRD42016046154. FINDINGS: We screened 216 477 citations and selected 670 reports from 337 studies of 118 strategies. Most strategies had multiple intervention components. For professional health-care providers (generally, facility-based health workers), the effects were near zero for only implementing a technology-based strategy (median MES 1·0 percentage points, IQR -2·8 to 9·9) or only providing printed information for health-care providers (1·4 percentage points, -4·8 to 6·2). For percentage outcomes, training or supervision alone typically had moderate effects (10·3-15·9 percentage points), whereas combining training and supervision had somewhat larger effects than use of either strategy alone (18·0-18·8 percentage points). Group problem solving alone showed large improvements in percentage outcomes (28·0-37·5 percentage points), but, when the strategy definition was broadened to include group problem solving alone or other strategy components, moderate effects were more typical (12·1 percentage points). Several multifaceted strategies had large effects, but multifaceted strategies were not always more effective than simpler ones. For lay health-care providers (generally, community health workers), the effect of training alone was small (2·4 percentage points). Strategies with larger effect sizes included community support plus health-care provider training (8·2-125·0 percentage points). Contextual and methodological heterogeneity made comparisons difficult, and most strategies had low quality evidence. INTERPRETATION: The impact of strategies to improve health-care provider practices varied substantially, although some approaches were more consistently effective than others. The breadth of the HCPPR makes its results valuable to decision makers for informing the selection of strategies to improve health-care provider practices in LMICs. These results also emphasise the need for researchers to use better methods to study the effectiveness of interventions. FUNDING: Bill & Melinda Gates Foundation, CDC Foundation.

Does shortening the training on Integrated Management of Childhood Illness guidelines reduce its effectiveness? A systematic review.

OBJECTIVE: Implementation of the Integrated Management of Childhood Illness (IMCI) strategy with an 11-day training course for health workers improves care for ill children in outpatient settings in developing countries. The 11-day course duration is recommended by the World Health Organization, which developed IMCI. Our aim was to determine if shortening the training (to reduce cost) reduces its effectiveness. METHODS: We conducted a systematic review to compare IMCI's effectiveness with standard training (duration ≥ 11 days) versus shortened training (5-10 days). Studies were identified from a search of MEDLINE, two existing systematic reviews, and by contacting investigators. We included published or unpublished studies that evaluated IMCI's effectiveness in developing countries and reported quantitative measures of health worker practices related to managing ill children under 5 years old in public or private health facilities. Summary measures were the median of effect sizes for all outcomes from a given study, and the percentage of patients needing oral antimicrobials or rehydration who were treated according to IMCI guidelines. FINDINGS: Twenty-nine studies were included. Direct comparisons from three studies showed little difference between standard and shortened training. Indirect comparisons from 26 studies revealed that effect sizes for standard training versus no IMCI were greater than shortened training versus no IMCI. Across all comparisons, differences ranged from -3 to +23 percentage-points, and our best estimate was a 2 to 16 percentage-point advantage for standard training. No result was statistically significant. After IMCI training (of any duration), 34% of ill children needing oral antimicrobials or rehydration were not receiving these treatments according to IMCI guidelines. CONCLUSIONS: Based on limited evidence, standard IMCI training seemed more effective than shortened training, although the difference might be small. As sizable performance gaps often existed after IMCI training, countries should consider implementing other interventions to support health workers after training, regardless of training duration.

A multifaceted intervention to improve health worker adherence to integrated management of childhood illness guidelines in Benin.

OBJECTIVES: We evaluated an intervention to support health workers after training in Integrated Management of Childhood Illness (IMCI), a strategy that can improve outcomes for children in developing countries by encouraging workers' use of evidence-based guidelines for managing the leading causes of child mortality. METHODS: We conducted a randomized trial in Benin. We administered a survey in 1999 to assess health care quality before IMCI training. Health workers then received training plus either study supports (job aids, nonfinancial incentives, and supervision of workers and supervisors) or usual supports. Follow-up surveys conducted in 2001 to 2004 assessed recommended treatment, recommended or adequate treatment, and an index of overall guideline adherence. RESULTS: We analyzed 1244 consultations. Performance improved in both intervention and control groups, with no significant differences between groups. However, training proceeded slowly, and low-quality care from health workers without IMCI training diluted intervention effects. Per-protocol analyses revealed that workers with IMCI training plus study supports provided better care than did those with training plus usual supports (27.3 percentage-point difference for recommended treatment; P < .05), and both groups outperformed untrained workers. CONCLUSIONS: IMCI training was useful but insufficient. Relatively inexpensive supports can lead to additional improvements.

Effect of multiple interventions on community health workers' adherence to clinical guidelines in Siaya district, Kenya.

Evaluation of a community health worker (CHW) programme in Siaya district, Kenya, showed CHWs commonly made errors in managing childhood illness. We assessed the effect of multiple interventions on CHW healthcare practices. A sample of 192 ill-child consultations performed by 114 CHWs in a hospital outpatient department between February and March 2001 were analysed. The mean percentage of assessment, classification and treatment procedures performed correctly for each child was 79.8% (range 13.3-100%). Of the 187 children who required at least one treatment or referral to a health facility, only 38.8% were prescribed all treatments (including referral) recommended by the guidelines. Multivariate analyses found no evidence that the intervention-related factors studied (refresher training, supervision, involvement of community women in the CHW selection process, adequacy of medicine supplies, and use of a guideline flipchart during consultations) were significantly associated with overall or treatment-specific guideline adherence. A multivariate linear regression analysis revealed that several non-intervention-related factors, such as patient characteristics, were significantly associated with overall guideline adherence. Given that our study was cross-sectional and our measurement of exposure to several interventions was based on CHW recall, the estimated effects of the interventions should be interpreted with caution. Despite these limitations, however, our results raise questions about the effectiveness, in the setting of Siaya district, of several interventions commonly used to improve the quality of care given by CHWs.

The burden of malaria mortality among African children in the year 2000.

BACKGROUND: Although malaria is a leading cause of child deaths, few well-documented estimates of its direct and indirect burden exist. Our objective was to estimate the number of deaths directly attributable to malaria among children <5 years old in sub-Saharan Africa for the year 2000. METHODS: We divided the population into six sub-populations and, using results of studies identified in a literature review, estimated a malaria mortality rate for each sub-population. Malaria deaths were estimated by multiplying each sub-population by its corresponding rate. Sensitivity analyses were performed to assess the impact of varying key assumptions. RESULTS: The literature review identified 31 studies from 14 countries in middle Africa and 17 studies and reports from four countries in southern Africa. In 2000, we estimated that approximately 100 million children lived in areas where malaria transmission occurs and that 803 620 (precision estimate: 705 821-901 418) children died from the direct effects of malaria. For all of sub-Saharan Africa, including populations not exposed to malaria, malaria accounted for 18.0% (precision estimate: 15.8-20.2%) of child deaths. These estimates were sensitive to extreme assumptions about the causes of deaths with no known cause. CONCLUSIONS: These estimates, based on the best available data and methods, clearly demonstrate malaria's enormous mortality burden. We emphasize that these estimates are an approximation with many limitations and that the estimates do not account for malaria's large indirect burden. We describe information needs that, if filled, might improve the validity of future estimates.

Epidemic diarrhea due to enterotoxigenic Escherichia coli.

BACKGROUND: In June 1998, we investigated one of the largest foodborne outbreaks of enterotoxigenic Escherichia coli gastroenteritis reported in the United States. METHODS: We conducted cohort studies of 11 catered events to determine risk factors for illness. We used stool cultures, polymerase chain reaction, and serologic tests to determine the etiologic agent, and we conducted an environmental inspection to identify predisposing conditions and practices at the implicated establishment. RESULTS: During 5-7 June, the implicated delicatessen catered 539 events attended by >16,000 people. Our epidemiological study of 11 events included a total of 612 attendees. By applying the median prevalence of illness (20%) among events with ill attendees to the total number of events with any ill attendees, we estimate that at least 3300 persons may have developed gastroenteritis during this outbreak. Multiple food items (potato salad, macaroni salad, egg salad, and watermelon) were associated with illness, all of which required extensive handling during preparation. Enterotoxigenic Escherichia coli serotype O6:H16 producing heat-labile and heat-stable toxins was isolated from the stool specimens from 11 patients. Eight patients with positive stool culture results, 11 (58%) of 19 other symptomatic attendees, and 0 (0%) of 17 control subjects had elevated serum antibody titers to E. coli O6 lipopolysaccharide. The delicatessen had inadequate hand-washing supplies, inadequate protection against back siphonage of wastewater in the potable water system, a poorly draining kitchen sink, and improper food storage and transportation practices. CONCLUSIONS: In the United States, where enterotoxigenic Escherichia coli is an emerging cause of foodborne disease, enterotoxigenic Escherichia coli should be suspected in outbreaks of gastroenteritis when common bacterial or viral enteric pathogens are not identified.

Breast-feeding decreases the risk of sporadic salmonellosis among infants in FoodNet sites.

Among the population of the Foodborne Diseases Active Surveillance Network (FoodNet) surveillance areas ("FoodNet sites") in 1996, children under 12 months of age had the highest incidence of sporadic salmonellosis. We conducted a case-control study in 5 FoodNet sites to identify risk factors for sporadic infant salmonellosis. A case patient was a child under 12 months of age with a laboratory-confirmed, nontyphoidal serogroup B or D Salmonella infection. Twenty-two case patients were matched with 39 control subjects by age and either telephone exchange or vital record birth list. In a multivariate analysis, case patients were more likely to have a liquid diet containing no breast milk than a liquid diet containing only breast milk (matched odds ratio, 44.5; P=.04). Case-patients were more likely to reside in a household where a member had diarrhea (matched odds ratio, 13.2; P=.01). To decrease their infants' risk of salmonellosis, mothers should be encouraged to breast-feed their infants. Caretakers of infants should learn about salmonellosis, hand washing, and safe preparation of formula and solid food.