Measuring health workers' motivation in rural health facilities: baseline results from three study districts in Zambia.

INTRODUCTION: Health worker motivation can potentially affect the provision of health services. Low morale among the workforce can undermine the quality of service provision and drive workers away from the profession. While the presence of high-quality, motivated staff is a key aspect of health system performance, it is also one of the most difficult factors to measure. METHODS: We assessed health worker motivation as part of the baseline assessment for a health system strengthening intervention in three rural districts in Zambia. The intervention (Better Health Outcomes Through Mentoring and Assessment (BHOMA)) aims to increase health worker motivation through training, mentoring and support. We assessed motivation by examining underlying issues grouped around relevant outcome constructs such as job satisfaction, general motivation, burnout, organization commitment, conscientiousness and timeliness that collectively measure overall levels of motivation. The tools and the concepts have been used in high-income countries and they were recently applied in African settings to measure health worker motivation. RESULTS: Female participants had the highest motivation scores (female: mean 78.5 (SD 7.8) vs male: mean (SD 7.0)). By type of worker, nurses had the highest scores while environmental health technicians had the lowest score (77.4 (SD 7.8 vs 73.2 (SD 9.3)). Health workers who had been in post longer also had higher scores (>7 months). Health workers who had received some form of training in the preceding 12 months were more likely to have a higher score; this was also true for those older than 40 years when compared to those less than 40 years of age. The highest score values were noted in conscientiousness and timeliness, with all districts scoring above 80. CONCLUSIONS: This study evaluated motivation among rural health workers using a simple adapted tool to measure the concept of motivation. Results showed variation in motivation score by sex, type of health worker, training and time in post. Further research is needed to establish why these health worker attributes were associated with motivation and whether health system interventions targeting health workers, such as the current intervention, could influence health worker motivation.

Systems thinking in practice: the current status of the six WHO building blocks for health system strengthening in three BHOMA intervention districts of Zambia: a baseline qualitative study.

BACKGROUND: The primary bottleneck to achieving the MDGs in low-income countries is health systems that are too fragile to deliver the volume and quality of services to those in need. Strong and effective health systems are increasingly considered a prerequisite to reducing the disease burden and to achieving the health MDGs. Zambia is one of the countries that are lagging behind in achieving millennium development targets. Several barriers have been identified as hindering the progress towards health related millennium development goals. Designing an intervention that addresses these barriers was crucial and so the Better Health Outcomes through Mentorship (BHOMA) project was designed to address the challenges in the Zambia's MOH using a system wide approach. We applied systems thinking approach to describe the baseline status of the Six WHO building blocks for health system strengthening. METHODS: A qualitative study was conducted looking at the status of the Six WHO building blocks for health systems strengthening in three BHOMA districts. We conducted Focus group discussions with community members and In-depth Interviews with key informants. Data was analyzed using Nvivo version 9. RESULTS: The study showed that building block specific weaknesses had cross cutting effect in other health system building blocks which is an essential element of systems thinking. Challenges noted in service delivery were linked to human resources, medical supplies, information flow, governance and finance building blocks either directly or indirectly. Several barriers were identified as hindering access to health services by the local communities. These included supply side barriers: Shortage of qualified health workers, bad staff attitude, poor relationships between community and health staff, long waiting time, confidentiality and the gender of health workers. Demand side barriers: Long distance to health facility, cost of transport and cultural practices. Participating communities seemed to lack the capacity to hold health workers accountable for the drugs and services. CONCLUSION: The study has shown that building block specific weaknesses had cross cutting effect in other health system building blocks. These linkages emphasised the need to use system wide approaches in assessing the performance of health system strengthening interventions.

Measuring governance at health facility level: developing and validation of simple governance tool in Zambia.

BACKGROUND: Governance has been cited as a key determinant of economic growth, social advancement and overall development. Achievement of millennium development goals is partly dependant on governance practices. In 2007, Health Systems 20/20 conducted an Internet-based survey on the practice of good governance. The survey posed a set of good practices related to health governance and asked respondents to indicate whether their experience confirmed or disconfirmed those practices. We applied the 17 governance statements in rural health facilities of Zambia. The aim was to establish whether the statements were reliable and valid for assessing governance practices at primary care level. METHODS: Both quantitative and qualitative methods were used. We first applied the governance statements developed by the health system 20/20 and then conducted focus group discussion and In-depth interviews to explore some elements of governance including accountability and community participation. The target respondents were the health facility management team and community members. The sample size include 42 health facilities. Data was analyzed using SPSS version 17 and Nvivo version 9. RESULTS: The 95% one-sided confidence interval for Cronbach's alpha was between 0.69 and 0.74 for the 16 items.The mean score for most of the items was above 3. Factor analysis yielded five principle components: Transparency, community participation, Intelligence & vision, Accountability and Regulation & oversight. Most of the items (6) clustered around the transparency latent factor. Chongwe district performed poorly in overall mean governance score and across the five domains of governance. The overall scores in Chongwe ranged between 51 and 94% with the mean of 80%. Kafue and Luangwa districts had similar overall mean governance scores (88%). Community participation was generally low. Generally, it was noted that community members lacked capacity to hold health workers accountable for drugs and medical supplies. CONCLUSIONS: The study successfully validated and applied the new tool for evaluating health system governance at health facility level. The results have shown that it is feasible to measure governance practices at health facility level and that the adapted tool is fairly reliable with the 95% one-sided confidence interval for Cronbach's alpha laying between 0.69 and 0.74 for the 16 items. Caution should be taken when interpreting overall scores as they tended to mask domain specific variations.

Application of systems thinking: 12-month postintervention evaluation of a complex health system intervention in Zambia: the case of the BHOMA.

RATIONALE, AIMS AND OBJECTIVES: Strong health systems are said to be paramount to achieving effective and equitable health care. The World Health Organization has been advocating for using system-wide approaches such as 'systems thinking' to guide intervention design and evaluation. In this paper we report the system-wide effects of a complex health system intervention in Zambia known as Better Health Outcome through Mentorship and Assessment (BHOMA) that aimed to improve service quality. METHODS: We conducted a qualitative study in three target districts. We used a systems thinking conceptual framework to guide the analysis focusing on intended and unintended consequences of the intervention. NVivo version 10 was used for data analysis. RESULTS: The addressed community responded positively to the BHOMA intervention. The indications were that in the short term there was increased demand for services but the health worker capacity was not severely affected. This means that the prediction that service demand would increase with implementation of BHOMA was correct and the workload also increased, but the help of clinic lay supporters meant that some of the work of clinicians was transferred to these lay workers. However, from a systems perspective, unintended consequences also occurred during the implementation of the BHOMA. CONCLUSIONS: We applied an innovative approach to evaluate a complex intervention in low-income settings, exploring empirically how systems thinking can be applied in the context of health system strengthening. Although the intervention had some positive outcomes by employing system-wide approaches, we also noted unintended consequences.

Application of system thinking concepts in health system strengthening in low-income settings: a proposed conceptual framework for the evaluation of a complex health system intervention: the case of the BHOMA intervention in Zambia.

BACKGROUND: The current drive to strengthen health systems provides an opportunity to develop new strategies that will enable countries to achieve targets for millennium development goals. In this paper, we present a proposed framework for evaluating a new health system strengthening intervention in Zambia known as Better Health Outcomes through Mentoring and Assessment. APPROACH: We briefly describe the intervention design and focus on the proposed evaluation approach through the lens of systems thinking. DISCUSSION: In this paper, we present a proposed framework to evaluate a complex health system intervention applying systems thinking concepts. We hope that lessons learnt from this process will help to adapt the intervention and limit unintended negative consequences while promoting positive effects. Emphasis will be paid to interaction and interdependence between health system building blocks, context and the community.

Application of balanced scorecard in the evaluation of a complex health system intervention: 12 months post intervention findings from the BHOMA intervention: a cluster randomised trial in Zambia.

INTRODUCTION: In many low income countries, the delivery of quality health services is hampered by health system-wide barriers which are often interlinked, however empirical evidence on how to assess the level and scope of these barriers is scarce. A balanced scorecard is a tool that allows for wider analysis of domains that are deemed important in achieving the overall vision of the health system. We present the quantitative results of the 12 months follow-up study applying the balanced scorecard approach in the BHOMA intervention with the aim of demonstrating the utility of the balanced scorecard in evaluating multiple building blocks in a trial setting. METHODS: The BHOMA is a cluster randomised trial that aims to strengthen the health system in three rural districts in Zambia. The intervention aims to improve clinical care quality by implementing practical tools that establish clear clinical care standards through intensive clinic implementations. This paper reports the findings of the follow-up health facility survey that was conducted after 12 months of intervention implementation. Comparisons were made between those facilities in the intervention and control sites. STATA version 12 was used for analysis. RESULTS: The study found significant mean differences between intervention(I) and control (C) sites in the following domains: Training domain (Mean I:C; 87.5.vs 61.1, mean difference 23.3, p = 0.031), adult clinical observation domain (mean I:C; 73.3 vs.58.0, mean difference 10.9, p = 0.02 ) and health information domain (mean I:C; 63.6 vs.56.1, mean difference 6.8, p = 0.01. There was no gender differences in adult service satisfaction. Governance and motivation scores did not differ between control and intervention sites. CONCLUSION: This study demonstrates the utility of the balanced scorecard in assessing multiple elements of the health system. Using system wide approaches and triangulating data collection methods seems to be key to successful evaluation of such complex health intervention. TRIAL NUMBER: ClinicalTrials.gov NCT01942278.

Measuring health system strengthening: application of the balanced scorecard approach to rank the baseline performance of three rural districts in Zambia.

INTRODUCTION: There is growing interest in health system performance and recently WHO launched a report on health systems strengthening emphasising the need for close monitoring using system-wide approaches. One recent method is the balanced scorecard system. There is limited application of this method in middle- and low-income countries. This paper applies the concept of balanced scorecard to describe the baseline status of three intervention districts in Zambia. METHODOLOGY: The Better Health Outcome through Mentoring and Assessment (BHOMA) project is a randomised step-wedged community intervention that aims to strengthen the health system in three districts in the Republic of Zambia. To assess the baseline status of the participating districts we used a modified balanced scorecard approach following the domains highlighted in the MOH 2011 Strategic Plan. RESULTS: Differences in performance were noted by district and residence. Finance and service delivery domains performed poorly in all study districts. The proportion of the health workers receiving training in the past 12 months was lowest in Kafue (58%) and highest in Luangwa district (77%). Under service capacity, basic equipment and laboratory capacity scores showed major variation, with Kafue and Luangwa having lower scores when compared to Chongwe. The finance domain showed that Kafue and Chongwe had lower scores (44% and 47% respectively). Regression model showed that children's clinical observation scores were negatively correlated with drug availability (coeff -0.40, p = 0.02). Adult clinical observation scores were positively association with adult service satisfaction score (coeff 0.82, p = 0.04) and service readiness (coeff 0.54, p = 0.03). CONCLUSION: The study applied the balanced scorecard to describe the baseline status of 42 health facilities in three districts of Zambia. Differences in performance were noted by district and residence in most domains with finance and service delivery performing poorly in all study districts. This tool could be valuable in monitoring and evaluation of health systems.