Strengthening monitoring and evaluation (M&E) and building sustainable health information systems in resource limited countries: lessons learned from an M&E task-shifting initiative in Botswana.

BACKGROUND: The demand for quality data and the interest in health information systems has increased due to the need for country-level progress reporting towards attainment of the United Nations Millennium Development Goals and global health initiatives. To improve monitoring and evaluation (M&E) of health programs in Botswana, 51 recent university graduates with no experience in M&E were recruited and provided with on-the-job training and mentoring to develop a new cadre of health worker: the district M&E officer. Three years after establishment of the cadre, an assessment was conducted to document achievements and lessons learnt. METHODS: This qualitative assessment included in-depth interviews at the national level (n = 12) with officers from government institutions, donor agencies, and technical organizations; and six focus group discussions separately with district M&E officers, district managers, and program officers coordinating different district health programs. RESULTS: Reported achievements of the cadre included improved health worker capacity to monitor and evaluate programs within the districts; improved data quality, management, and reporting; increased use of health data for disease surveillance, operational research, and planning purposes; and increased availability of time for nurses and other health workers to concentrate on core clinical duties. Lessons learnt from the assessment included: the importance of clarifying roles for newly established cadres, aligning resources and equipment to expectations, importance of stakeholder collaboration in implementation of sustainable programs, and ensuring retention of new cadres. CONCLUSION: The development of a dedicated M&E cadre at the district level contributed positively to health information systems in Botswana by helping build M&E capacity and improving data quality, management, and data use. This assessment has shown that such cadres can be developed sustainably if the initiative is country-led, focusing on recruitment and capacity-development of local counterparts, with a clear government retention plan.

Continuing professional development training needs of medical laboratory personnel in Botswana.

BACKGROUND: Laboratory professionals are expected to maintain their knowledge on the most recent advances in laboratory testing and continuing professional development (CPD) programs can address this expectation. In developing countries, accessing CPD programs is a major challenge for laboratory personnel, partly due to their limited availability. An assessment was conducted among clinical laboratory workforce in Botswana to identify and prioritize CPD training needs as well as preferred modes of CPD delivery. METHODS: A self-administered questionnaire was disseminated to medical laboratory scientists and technicians registered with the Botswana Health Professions Council. Questions were organized into domains of competency related to (i) quality management systems, (ii) technical competence, (iii) laboratory management, leadership, and coaching, and (iv) pathophysiology, data interpretation, and research. Participants were asked to rank their self-perceived training needs using a 3-point scale in order of importance (most, moderate, and least). Furthermore, participants were asked to select any three preferences for delivery formats for the CPD. RESULTS: Out of 350 questionnaires that were distributed, 275 were completed and returned giving an overall response rate of 79%. The most frequently selected topics for training in rank order according to key themes were (mean, range) (i) quality management systems, most important (79%, 74-84%); (ii) pathophysiology, data interpretation, and research (68%, 52-78%); (iii) technical competence (65%, 44-73%); and (iv) laboratory management, leadership, and coaching (60%, 37-77%). The top three topics selected by the participants were (i) quality systems essentials for medical laboratory, (ii) implementing a quality management system, and (iii) techniques to identify and control sources of error in laboratory procedures. The top three preferred CPD delivery modes, in rank order, were training workshops, hands-on workshops, and internet-based learning. Journal clubs at the workplace was the least preferred method of delivery of CPD credits. CONCLUSIONS: CPD programs to be developed should focus on topics that address quality management systems, case studies, competence assessment, and customer care. The findings from this survey can also inform medical laboratory pre-service education curriculum.

Improving the quality of health information: a qualitative assessment of data management and reporting systems in Botswana.

BACKGROUND: Ensuring that data collected through national health information systems are of sufficient quality for meaningful interpretation is a challenge in many resource-limited countries. An assessment was conducted to identify strengths and weaknesses of the health data management and reporting systems that capture and transfer routine monitoring and evaluation (M&E) data in Botswana. METHODS: This was a descriptive, qualitative assessment. In-depth interviews were conducted at the national (n = 27), district (n = 31), and facility/community (n = 71) levels to assess i) M&E structures, functions, and capabilities; ii) indicator definitions and reporting guidelines; iii) data collection forms and tools; iv) data management processes; and v) links with the national reporting system. A framework analysis was conducted using ATLAS.ti v6.1. RESULTS: Health programs generally had standardized data collection and reporting tools and defined personnel for M&E responsibilities at the national and district levels. Best practices unique to individual health programs were identified and included a variety of relatively low-resource initiatives such as attention to staffing patterns, making health data more accessible for evidence-based decision-making, developing a single source of information related to indicator definitions, data collection tools, and management processes, and utilization of supportive supervision visits to districts and facilities. Weakness included limited ownership of M&E-related duties within facilities, a lack of tertiary training programs to build M&E skills, few standard practices related to confidentiality and document storage, limited dissemination of indicator definitions, and limited functionality of electronic data management systems. CONCLUSIONS: Addressing fundamental M&E system issues, further standardization of M&E practices, and increasing health services management responsiveness to time-sensitive information are critical to sustain progress related to health service delivery in Botswana. In addition to high-resource initiatives, such as investments in electronic medical record systems and tertiary training programs, there are a variety of low-resource initiatives, such as regular data quality checks, that can strengthen national health information systems. Applying best practices that are effective within one health program to data management and reporting systems of other programs is a practical approach for strengthening health informatics and improving data quality.

Establishing a health information workforce: innovation for low- and middle-income countries.

BACKGROUND: To address the shortage of health information personnel within Botswana, an innovative human resources approach was taken. University graduates without training or experience in health information or health sciences were hired and provided with on-the-job training and mentoring to create a new cadre of health worker: the district Monitoring and Evaluation (M&E) Officer. This article describes the early outcomes, achievements, and challenges from this initiative. METHODS: Data were collected from the district M&E Officers over a 2-year period and included a skills assessment at baseline and 12 months, pre- and post-training tests, interviews during stakeholder site visits, a survey of achievements, focus group discussions, and an attrition assessment. RESULTS: An average of 2.7 mentoring visits were conducted for M&E Officers in each district. There were five training sessions over 18 months. Knowledge scores significantly increased (p < 0.05) during the three trainings in which pre/post tests were administered. Over 1 year, there were significant improvements (p < 0.05) in self-rated skills related to computer literacy, checking data validity, implementing data quality procedures, using data to support program planning, proposing indicators, and writing M&E reports. Out of the 34 district M&E Officers interviewed during site visits, most were conducting facility visits to review data (27/34; 79%), comparing data sets over time (31/34; 91%), backing up data (32/34; 94%), and analyzing data (32/34; 94%). Common challenges included late facility reports (28/34; 82%), lack of transportation (22/34; 65%), inaccurate facility reports (10/34; 29%), and colleagues' misunderstanding of M&E (10/34; 29%). Six posts were vacated in the first year (6/51; 12%). A total of 49 Officers completed the achievements survey; of these, common accomplishments related to improvements in data management (35/49; 71%), data quality (31/49; 63%), data use (29/49; 59%), and capacity development (26/49; 53%). CONCLUSIONS: The development of a cadre of district M&E Officers has contributed positively to the health information system in Botswana. In the absence of tertiary training related to health information, on-the-job training and mentoring of university graduates can be an effective approach for developing a new professional cadre of M&E expertise and for strengthening capacity within a national health system.