Demographic and Geographic Characterization of Excess Mortality During the COVID-19 Pandemic in Baltimore City, Maryland, March 2020 to March 2021.

Estimates of excess mortality can provide insight into direct and indirect impacts of the coronavirus disease 2019 (COVID-19) pandemic beyond deaths specifically attributed to COVID-19. We analyzed death certificate data from Baltimore City, Maryland, from March 1, 2020, to March 31, 2021, and found that 1,725 individuals (95% confidence interval: 1,495, 1,954) died in excess of what was expected from all-cause mortality trends in 2016-2019; 1,050 (61%) excess deaths were attributed to COVID-19. Observed mortality was 23%-32% higher than expected among individuals aged 50 years and older. Non-White residents of Baltimore City also experienced 2 to 3 times higher rates of excess mortality than White residents (e.g., 37.4 vs. 10.7 excess deaths per 10,000 population among Black residents vs. White residents). There was little to no observed excess mortality among residents of hospice, long-term care, and nursing home facilities, despite accounting for nearly 30% (312/1,050) of recorded COVID-19 deaths. There was significant geographic variation in excess mortality within the city, largely following racial population distributions. These results demonstrate the substantial and unequal impact of the COVID-19 pandemic on Baltimore City residents and the importance of building robust, timely surveillance systems to track disparities and inform targeted strategies to remediate the impact of future epidemics.

Behavioral Determinants of Routine Health Information System Data Use in Senegal: A Qualitative Inquiry Based on the Integrated Behavioral Model.

Routine health information system (RHIS) data are essential in driving decision making and planning in health systems as well as health programs. However, despite their importance, these data are underutilized, and the underlying individual-level facilitators and barriers to use remain understudied. In this research, we applied the Integrated Behavior Model (IBM) to examine how attitudes toward RHIS data, perceived norms concerning RHIS data use, and the ability to use RHIS data influence the demand and use of RHIS data among stakeholders in Senegal. Using data from interviews with respondents working at national levels of malaria, HIV, and TB control programs in Senegal, we used a framework analysis approach to apply the IBM behavioral constructs and identify their linkages to RHIS data use. We found that attitudes about the quality, availability, and relevance of RHIS data for decision making were important in driving data use among respondents. Institutional expectations, organizational protocols, policies, and practices around RHIS data ultimately shape social norms around the use of the data. Although we found that perceived ability and self-efficacy to use RHIS data were not barriers to RHIS data use among stakeholders at the strategic levels of their respective organizations, these were reported to be barriers at lower levels of the health system. Low perceived control of the RHIS data production process ultimately reduced RHIS data use for decision making among the strategic-level respondents. We recommend context-specific reexamination of existing RHIS interventions with a renewed emphasis on behavioral aspects of data use. The IBM can help guide practitioners, policy makers, and academics to address multiple socioecological factors that influence data use behavior when recommending RHIS and data use solutions.

From raw data to a score: comparing quantitative methods that construct multi-level composite implementation strength scores of family planning programs in Malawi.

BACKGROUND: Data that capture implementation strength can be combined in multiple ways across content and health system levels to create a summary measure that can help us to explore and compare program implementation across facility catchment areas. Summary indices can make it easier for national policymakers to understand and address variation in strength of program implementation across jurisdictions. In this paper, we describe the development of an index that we used to describe the district-level strength of implementation of Malawi's national family planning program. METHODS: To develop the index, we used data collected during a 2017 national, health facility and community health worker Implementation Strength Assessment survey in Malawi to test different methods to combine indicators within and then across domains (4 methods-simple additive, weighted additive, principal components analysis, exploratory factor analysis) and combine scores across health facility and community health worker levels (2 methods-simple average and mixed effects model) to create a catchment area-level summary score for each health facility in Malawi. We explored how well each model captures variation and predicts couple-years protection and how feasible it is to conduct each type of analysis and the resulting interpretability. RESULTS: We found little difference in how the four methods combined indicator data at the individual and combined levels of the health system. However, there were major differences when combining scores across health system levels to obtain a score at the health facility catchment area level. The scores resulting from the mixed effects model were able to better discriminate differences between catchment area scores compared to the simple average method. The scores using the mixed effects combination method also demonstrated more of a dose-response relationship with couple-years protection. CONCLUSIONS: The summary measure that was calculated from the mixed effects combination method captured the variation of strength of implementation of Malawi's national family planning program at the health facility catchment area level. However, the best method for creating an index should be based on the pros and cons listed, not least, analyst capacity and ease of interpretability of findings. Ultimately, the resulting summary measure can aid decision-makers in understanding the combined effect of multiple aspects of programs being implemented in their health system and comparing the strengths of programs across geographies.

Comparing program supervision with an external RADAR evaluation of quality of care in integrated community case management for childhood illnesses in Mali.

BACKGROUND: Many countries have adopted integrated community case management (iCCM) to reduce mortality among children under five years from common childhood illnesses. The 2016-2020 Malian Red Cross iCCM program trained 441 Community Health Workers (CHWs) to treat malaria, pneumonia, diarrhea, and malnutrition for children under five years of age in six districts. Implementation strength and quality of care (QoC) were assessed through the program's supervision function, using the Malian Ministry of Health's system. OBJECTIVE: This paper compares methods and results of program supervision data and an independent evaluation to assess the effectiveness of program implementation and supervision and inform program improvement. It also presents the benefits and limitations of each method. METHOD: An independent QoC evaluation was conducted using tools developed by the Real Accountability: Data Analysis for Results (RADAR) project, hereafter referred to as the RADAR evaluation. RADAR evaluation data collected in July and August 2018 were compared with program supervision data collected mostly between May and December 2018. RESULTS: The RADAR evaluation provided detailed findings on correct assessment, classification, and treatment per illness, medication type, and dosage. Program supervision combined the findings for all illnesses, medication type, and dosage due to limitations in the data collection process. Six indicators were comparable between both methods. Findings were similar for temperature and mid-upper arm circumference measurements but diverged between program supervision and the RADAR evaluation, respectively, on correct classification for all illnesses (87.1% vs. 65.3%), correct treatment for all illnesses (69.5% vs. 39.8%), correct respiratory rate counting (88.5% vs. 54.7%), and administering the first dose by CHW (75.4% vs. 65.0%). Findings from the RADAR evaluation guided improvements in program supervision. CONCLUSIONS: A robust program supervision system can serve as a credible method to assess QoC. However, a rigorous independent QoC evaluation provides a valuable benchmark to gauge the effectiveness of the supervisory process.

Measuring Quality of Maternal, Neonatal, Child, Reproductive Health and Nutrition Care with tools developed by the RADAR project and tested in Sub Saharan Africa.

Increasing coverage of evidence-based maternal, neonatal, child, reproductive health and nutrition (MNCRHN) programs in low- and middle-income countries has coincided with dramatic improvements in health despite variable quality of implementation. Comprehensive evaluation to inform program improvement requires standardized but adaptable tools, which the Real Accountability, Data Analysis for Results (RADAR) project has developed. To inform selection of tools and methods packages ('packages') to measure program quality of care (QoC), we documented experiences testing the packages, which were developed and adapted based on global and local expertise, and pre- and pilot-testing. We conducted cross-sectional studies in 2018-2019 on the quality of 1) integrated community case management, 2) counseling on maternal, infant, and young child feeding, 3) intrapartum care, and 4) family planning counseling in Mali, Mozambique, Tanzania, and Malawi. Herein we describe package performance and highlight experiences that inform their selection and use. Direct observation packages provided high-quality, immediately applicable results but they required specialized expertise, in-person collection, adequate patient volume, reasonable wait times, and unambiguously 'correct' provision of care. General satisfaction questions from exit interview packages produced unvaryingly positive responses despite variable observed quality of care. Variation increased when questions were more targeted, but findings on caregiver and client's recall of recommendations were more actionable. When interactive, clinical vignettes can capture knowledge of clinical care. But for conditions that can be simulated, like provision of family planning counseling, we could capture provider practice from simulated clients. Clinicians could more easily demonstrate tactile aspects of intrapartum care using observed structured clinical examinations, but this method required storage and transport of the required mannequins. Based on our findings we recommend ten questions upon which evaluators can base package selection. Findings from these packages inform programs and, in the context of comprehensive program evaluation enable us to link programs with impact.

A data quality assessment of the first four years of malaria reporting in the Senegal DHIS2, 2014-2017.

BACKGROUND: As the global burden of malaria decreases, routine health information systems (RHIS) have become invaluable for monitoring progress towards elimination. The District Health Information System, version 2 (DHIS2) has been widely adopted across countries and is expected to increase the quality of reporting of RHIS. In this study, we evaluated the quality of reporting of key indicators of childhood malaria from January 2014 through December 2017, the first 4 years of DHIS2 implementation in Senegal. METHODS: Monthly data on the number of confirmed and suspected malaria cases as well as tests done were extracted from the Senegal DHIS2. Reporting completeness was measured as the number of monthly reports received divided by the expected number of reports in a given year. Completeness of indicator data was measured as the percentage of non-missing indicator values. We used a quasi-Poisson model with natural cubic spline terms of month of reporting to impute values missing at the facility level. We used the imputed values to take into account the percentage of malaria cases that were missed due to lack of reporting. Consistency was measured as the absence of moderate and extreme outliers, internal consistency between related indicators, and consistency of indicators over time. RESULTS: In contrast to public facilities of which 92.7% reported data in the DHIS2 system during the study period, only 15.3% of the private facilities used the reporting system. At the national level, completeness of facility reporting increased from 84.5% in 2014 to 97.5% in 2017. The percentage of expected malaria cases reported increased from 76.5% in 2014 to 94.7% in 2017. Over the study period, the percentage of malaria cases reported across all districts was on average 7.5% higher (P < 0.01) during the rainy season relative to the dry season. Reporting completeness rates were lower among hospitals compared to health centers and health posts. The incidence of moderate and extreme outlier values was 5.2 and 2.3%, respectively. The number of confirmed malaria cases increased by 15% whereas the numbers of suspected cases and tests conducted more than doubled from 2014 to 2017 likely due to a policy shift towards universal testing of pediatric febrile cases. CONCLUSIONS: The quality of reporting for malaria indicators in the Senegal DHIS2 has improved over time and the data are suitable for use to monitor progress in malaria programs, with an understanding of their limitations. Senegalese health authorities should maintain the focus on broader adoption of DHIS2 reporting by private facilities, the sustainability of district-level data quality reviews, facility-level supervision and feedback mechanisms at all levels of the health system.

Assessing provider performance of intrapartum care using simulated encounters and clinical vignettes: A comparison study from Tanzania.

OBJECTIVE: To compare clinical vignettes and objective structured clinical examinations (OSCE) as methods for assessing the quality of intrapartum care among skilled providers in rural primary-level health facilities in Tanzania. METHODS: Cross-sectional study conducted at six health facilities in the Simiyu region of Tanzania. Providers were assessed using OSCE and clinical vignettes in spontaneous delivery, neonatal resuscitation, and management of postpartum hemorrhage. Trained researchers used a structured clinical checklist. The frequencies of items are presented as percentages and the agreement of the methods of assessment are reported using kappa statistics (high: kappa > 0.80, moderate: kappa = 0.60-0.80, low: kappa < 0.60). RESULTS: Most healthcare providers were female (60.7%), registered nurses by training (29.0%), and worked in a dispensary (56.1%), with an average age of 33 years and an average of 7.4 years of experience in their respective professions. Five items had high agreement between OSCE and clinical vignettes: postpartum vital signs every 15 min, oxytocin within 1 min of birth, diagnosis of postpartum hemorrhage, elevating legs of the mother, and deciding on manual compression of the uterus. CONCLUSION: OSCE and clinical vignettes should be viewed as complimentary to one another in the assessment of provider knowledge and skill, with priority given to OSCE, particularly in intrapartum care.

Challenges and Barriers to Adverse Event Reporting in Clinical Trials: A Children's Oncology Group Report.

OBJECTIVE: Adverse event (AE) reporting is crucial for determining safety of trials. Adverse events are captured manually by clinical research associates (CRAs) and research nurses (RNs), and prior studies show underreporting. It is necessary to understand AE reporting training, processes, and institution-level differences to improve AE capture. METHODS: A 26-item questionnaire regarding AE reporting training, identification, tracking, and challenges was distributed to all Children's Oncology Group (COG) CRAs and RNs from February 15 to March 11, 2019, regardless of if they report AEs based on limitations of COG rosters. Results were tabulated. Institutions were grouped by self-reported full-time equivalents and compared using χ2 tests. RESULTS: Of 1315 CRAs and 2703 RNs surveyed, 509 (12.7%) responded. Of those, 369 (64.9%) representing 71.8% of COG institutions report AEs. Only data from respondents who report AEs were collected and analyzed. There was a range in AE training; COG training modules were most common (79.7%). There was wide variability in AE ascertainment; only 51.2% use standardized approaches at their site. There was no standard AE tracking method; larger sites more commonly use spreadsheets (P = 0.002) and smaller sites more commonly use paper (P = 0.028). The greatest AE reporting challenges were differences between protocols (70%) and between AE definitions and documentation (53%). Half of the respondents endorsed 6 of 13 proposed tools for improving reporting including online AE reporting modules (75.3%), tip sheets for interpreting Common Terminology Criteria for Adverse Events definitions (67.5%), and standardized AE tracking forms (66.9%). Only half of the respondents reported that all colleagues at their site followed the same AE reporting practices, and there was no dominant AE tracking approach across the respondents. DISCUSSION: There is wide variability in AE reporting training and practices. Numerous challenges exist, including differences between trials, challenges in interpreting AE definitions, and engaging clinicians. CONCLUSIONS: Respondents are eager for additional central resources. These results provide a roadmap for areas of potential improvement.

The aggregate effect of implementation strength of family planning programs on modern contraceptive use at the health systems level in rural Malawi.

To explore the association between the strength of implementation of family planning (FP) programs on the use of modern contraceptives. Specifically, how strongly these programs are being implemented across a health facility's catchment area in Malawi and the odds of a woman in that catchment area is using modern contraceptives. This information can be used to assess whether the combined impact of multiple large-scale FP programs is leading to change in the health outcomes they aim to improve. We used data from the 2017 Implementation Strength Assessment (ISA) that quantified how much of family planning programs at the health facility and community health worker levels were being implemented across every district of Malawi. We used a summary measure developed in a previous study that employs quantitative methods to combine data across FP domains and health system levels. We tested the association of this summary measure for implementation strength with household data from the 2015 Malawi Demographic Health Survey (DHS). We found that areas with stronger implementation of FP programs had higher odds of women using modern contraceptives compared with areas with weaker implementation. The association of ISA with use of modern contraception was different by education, marital status, and geography. After controlling for these factors, we found that the adjusted odds of using a modern contraceptive was three times higher in catchment areas with high implementation strength compared to those with lower strength. Metrics that summarize how strongly FP programs are being implemented were used to show a statistically significantly positive relationship between increasing implementation strength and higher rates of modern contraceptive use. Decisionmakers at the various levels of health authority can use this type of summary measure to better understand the combined impact of their diverse FP programming and inform future programmatic and policy decisions. The findings also reinforce the idea that having a well-supported and supplied cadre of community health workers supplementing FP provision at the health facility can be an important health systems mechanism, especially in rural settings and to target youth populations.

Comparability of family planning quality of care measurement tools in low-and-middle income country settings: a systematic review.

BACKGROUND: In low-and-middle income countries (LMICs), accurate measures of the elements of quality care provided by a health worker through family planning services (also known as process quality) are required to ensure family's contraceptives needs are being met. There are many tools used to assess family planning process quality of care (QoC) but no one standardized method. Those measuring QoC in LMICs should select an appropriate tool based the program context and financial/logistical parameters, but they require data on how well each tool measures routine clinical care. We aim to synthesize the literature on validity/comparability of family planning process QoC measurement tools through a quantitative systematic review with no meta-analysis. METHODS: We searched six literature databases for studies that compared quality measurements from different tools using quantitative statistics such as sensitivity/specificity, kappa statistic or absolute difference. We extracted the comparative measure along with other relevant study information, organized by quality indicator domain (e.g. counseling and privacy), and then classified the measure by low, medium, and high agreement. RESULTS: We screened 8172 articles and identified eight for analysis. Studies comparing quality measurements from simulated clients, direct observation, client exit interview, provider knowledge quizzes, and medical record review were included. These eight studies were heterogenous in their methods and the measurements compared. There was insufficient data to estimate overall summary measures of validity for the tools. Client exit interviews compared to direct observation or simulated client protocols had the most data and they were a poor proxy of the actual quality care received for many measurements. CONCLUSION: To measure QoC consistently and accurately in LMICs, standardized tools and measures are needed along with an established method of combining them for a comprehensive picture of quality care. Data on how different tools proxy quality client care will inform these guidelines. Despite the small number of studies found during the review, we described important differences on how tools measure quality of care.

Accurate measures of process quality of care­or how well clinicians deliver services according to standards of care­are important to monitor, evaluate and improve service quality. Periodic surveys of health facilities or provider are the main source of national or regional quality of care data in many low- and middle-income countries. Many tools are used for these surveys: exit interviews with patients, observations of the visits by a clinician-assessor, simulated or mystery patients, and others. Implementers must select the appropriate and feasible tools for their program, context and setting but there is little information on how well different tools measure the same quality of care indicators.This review summarizes the current literature on the validity of measurements from different family planning quality of care tools. We found only eight studies, but we were able to see some differences important to consider when selecting the most appropriate tool. For instance, patients reported different events through an exit interview than what was documented by the assessor during the same visit. Exit interviews may be more appropriate to measure client experience or satisfaction rather than specifics of the care received. Knowing these differences will help implementers choose an appropriate tool depending on the focus of the quality assessment. This review contributes to the body of knowledge on improving quality of care measurements, resulting in better data to improve family planning services for patients.

Validity of using mobile phone surveys to evaluate community health worker program in Mali.

BACKGROUND: The monitoring and evaluation of public health programs based on traditional face-to-face interviews in hard-to-reach and unstable regions present many challenges. Mobile phone-based methods are considered to be an effective alternative, but the validity of mobile phone-based data for assessing implementation strength has not been sufficiently studied yet. Nested within an evaluation project for an integrated community case management (iCCM) and family planning program in Mali, this study aimed to assess the validity of a mobile phone-based health provider survey to measure the implementation strength of this program. METHODS: From July to August 2018, a cross-sectional survey was conducted among the community health workers (ASCs) from six rural districts working with the iCCM and family planning program. ASCs were first reached to complete the mobile phone-based survey; within a week, ASCs were visited in their communities to complete the in-person survey. Both surveys used identical implementation strength tools to collect data on program activities related to iCCM and family planning. Sensitivity and specificity were calculated for each implementation strength indicator collected from the phone-based survey, with the in-person survey as the gold standard. A threshold of ≥ 80% for sensitivity and specificity was considered adequate for evaluation purposes. RESULTS: Of the 157 ASCs interviewed by mobile phone, 115 (73.2%) were reached in person. Most of the training (2/2 indicators), supervision (2/3), treatment/modern contraceptive supply (9/9), and reporting (3/3) indicators reached the 80% threshold for sensitivity, while only one supervision indicator and one supply indicator reached 80% for specificity. In contrast, most of the stock-out indicators (8/9) reached 80% for specificity, while only two indicators reached the threshold for sensitivity. CONCLUSIONS: The validity of mobile phone-based data was adequate for general training, supervision, and supply indicators for iCCM and family planning. With sufficient mobile phone coverage and reliable mobile network connection, mobile phone-based surveys are useful as an alternative for data collection to assess the implementation strength of general activities in hard-to-reach areas.

Key informant perspectives on the challenges and opportunities for using routine health data for decision-making in Senegal.

BACKGROUND: Increasing the performance of routine health information systems (RHIS) is an important policy priority both globally and in Senegal. As RHIS data become increasingly important in driving decision-making in Senegal, it is imperative to understand the factors that determine their use. METHODS: Semi-structured interviews were conducted with 18 high- and mid-level key informants active in the malaria, tuberculosis and HIV programmatic areas in Senegal. Key informants were employed in the relevant divisions of the Senegal Ministry of Health or nongovernmental / civil society organizations. We asked respondents questions related to the flow, quality and use of RHIS data in their organizations. A framework approach was used to analyze the qualitative data. RESULTS: Although the respondents worked at the strategic levels of their respective organizations, they consistently indicated that data quality and data use issues began at the operational level of the health system before the data made its way to the central level. We classify the main identified barriers and facilitators to the use of routine data into six categories and attempt to describe their interrelated nature. We find that data quality is a central and direct determinant of RHIS data use. We report that a number of upstream factors in the Senegal context interact to influence the quality of routine data produced. We identify the sociopolitical, financial and system design determinants of RHIS data collection, dissemination and use. We also discuss the organizational and infrastructural factors that influence the use of RHIS data. CONCLUSIONS: We recommend specific prescriptive actions with potential to improve RHIS performance in Senegal, the quality of the data produced and their use. These actions include addressing sociopolitical factors that often interrupt RHIS functioning in Senegal, supporting and motivating staff that maintain RHIS data systems as well as ensuring RHIS data completeness and representativeness. We argue for improved coordination between the various stakeholders in order to streamline RHIS data processes and improve transparency. Finally, we recommend the promotion of a sustained culture of data quality assessment and use.

Disrespectful care in family planning services among youth and adult simulated clients in public sector facilities in Malawi.

BACKGROUND: Provision of high-quality family planning (FP) services improves access to contraceptives. Negative experiences in maternal health have been documented worldwide and likely occur in other services including FP. This study aims to quantify disrespectful care for adult and adolescent women accessing FP in Malawi. METHODS: We used simulated clients (SCs) to measure disrespectful care in a census of public facilities in six districts of Malawi in 2018. SCs visited one provider in each of the 112 facilities: two SCs visits (one adult and one adolescent case scenario) or 224 SC visits total. We measured disrespectful care using a quantitative tool and field notes and report the prevalence and 95% confidence intervals for the indicators and by SC case scenarios contextualized with quotes from the field notes. RESULTS: Some SCs (12%) were refused care mostly because they did not agree to receive a HIV test or vaccination, or less commonly because the clinic was closed during operating hours. Over half (59%) of the visits did not have privacy. The SCs were not asked their contraceptive preference in 57% of the visits, 28% reported they were not greeted respectfully, and 20% reported interruptions. In 18% of the visits the SCs reported humiliation such as verbal abuse. Adults SCs received poorer counseling compared to the adolescent SCs with no other differences found. CONCLUSIONS: We documented instances of refusal of care, lack of privacy, poor client centered care and humiliating treatment by providers. We recommend continued effort to improve quality of care with an emphasis on client treatment, regular quality assessments that include measurement of disrespectful care, and more research on practices to reduce it.

Linking household surveys and facility assessments: a comparison of geospatial methods using nationally representative data from Malawi.

BACKGROUND: Linking facility and household surveys through geographic methods is a popular technique to draw conclusions about the relationship between health services and population health outcomes at local levels. These methods are useful tools for measuring effective coverage and tracking progress towards Universal Health Coverage, but are understudied. This paper compares the appropriateness of several geospatial methods used for linking individuals (within displaced survey cluster locations) to their source of family planning (at undisplaced health facilities) at a national level. METHODS: In Malawi, geographic methods linked a population health survey, rural clusters from the Woman's Questionnaire of the 2015 Malawi Demographic and Health Survey (MDHS 2015), to Malawi's national health facility census to understand the service environment where women receive family planning services. Individuals from MDHS 2015 clusters were linked to health facilities through four geographic methods: (i) closest facility, (ii) buffer (5 km), (iii) administrative boundary, and (iv) a newly described theoretical catchment area method. Results were compared across metrics to assess the number of unlinked clusters (data lost), the number of linkages per cluster (precision of linkage), and the number of women linked to their last source of modern contraceptive (appropriateness of linkage). RESULTS: The closest facility and administrative boundary methods linked every cluster to at least one facility, while the 5-km buffer method left 288 clusters (35.3%) unlinked. The theoretical catchment area method linked all but one cluster to at least one facility (99.9% linked). Closest facility, 5-km buffer, administrative boundary, and catchment methods linked clusters to 1.0, 1.4, 21.1, and 3.3 facilities on average, respectively. Overall, the closest facility, 5-km buffer, administrative boundary, and catchment methods appropriately linked 64.8%, 51.9%, 97.5%, and 88.9% of women to their last source of modern contraceptive, respectively. CONCLUSIONS: Of the methods studied, the theoretical catchment area linking method loses a marginal amount of population data, links clusters to a relatively low number of facilities, and maintains a high level of appropriate linkages. This linking method is demonstrated at scale and can be used to link individuals to qualities of their service environments and better understand the pathways through which interventions impact health.

Testing the validity and feasibility of using a mobile phone-based method to assess the strength of implementation of family planning programs in Malawi.

BACKGROUND: To effectively deliver on proposed objectives, it is vital that practitioners, policymakers, and other stakeholders are able to clearly understand how strongly their large-scale program is being implemented. This study sought to test the feasibility, cost-effectiveness, and validity of a phone-based method as an innovative and cost-efficient approach to assessing program implementation strength (through an Implementation Strength Assessment - ISA), alternative to the traditional in-person field methods. METHODS: We conducted 701 mobile phone and 356 in-person interviews with facility in-Charges and two types of community health workers who provide family planning services in the Dowa and Ntcheu districts in Malawi. Responses received via the phone interview were validated through in-person review of records and inspections. Sensitivity and specificity were calculated to determine validity. RESULTS: Most indicators at the health facility and community health worker levels were above a 70% threshold for sensitivity. However, there were fewer indicators that met this threshold for specificity. The primary reason for lower specificity was due to poor recordkeeping. Collecting data via mobile phone was found to be feasible and twice as cost-efficient as collecting the same data via in-person inspections. CONCLUSIONS: The rapid increase in mobile phone ownership and network availability in lower income countries could offer an alternative, cost-effective avenue to collect data for a better understanding of program implementation. Through rigorous assessment, this study found that using mobile phones could be a low-cost alternative to collect data on health system delivery of services, especially in places where routine data quality is poor and traditional, in-person methods are costly.

How strong are Malawi's family planning programs for adolescent and adult women? Results of a national assessment of implementation strength conducted by Malawi's National Evaluation Platform.

BACKGROUND: To assess the strength of implementation of family planning programs targeting youth (15-24) in Malawi with a specific focus on youth and the Youth-Friendly Health Services program. METHODS: We conducted 9781 mobile phone interviews with facility in-Charge Nurses and health workers (health facility workers, health surveillance assistants [HSAs] and community-based distribution agent [CBDAs]) who provide family planning (FP) services across the 28 districts. Responses were entered in tablet using Open Data Kit. They were summarized and presented using R, Stata (College Station, TX, USA, StatsReport, JHU, Baltimore, MD, USA) and ArcView GIS (ESRI, Redlands, CA, USA). RESULTS: Availability of key products was a challenge across all health worker types as only 39% of health facilities, 29% of HSAs and 45% of CBDAs had all the FP methods they are supposed to provide on the day of the interview. About 50% of health workers were supervised within past 90 days preceding the study. Despite most facilities saying that they provide youth friendly health services, youth-specific FP guidelines or protocols were not available in 43% of facilities that provide these services and only 33% of facilities had special rooms and 58% have special days for youth. CONCLUSIONS: The commodity supply system needs to ensure that all facilities and workers have a consistent supply of all contraceptive methods. Government and program implementers should ensure availability of all FP guidelines and information, education, communication materials at all service delivery points and facilitate creation of special rooms or days for youth.

Youth accessing reproductive health services in Malawi: drivers, barriers, and suggestions from the perspectives of youth and parents.

BACKGROUND: Malawi has made progress in increasing its overall modern contraceptive prevalence rate since 2000, resulting in a dramatic reduction in its total fertility rate. However, youth, 15-24 years, have not had the same successes. Teenage pregnancies are on the rise and little progress has been made in reducing unmet need for family planning among youth. With two-thirds of the population under the age of 25 and with Malawi's rapid population growth, reducing unmet need for family planning among youth remains a priority for the government's reproductive health agenda. To further explore this situation, we conducted a qualitative study to explore the perspectives of youth and adults about the drivers and barriers to youth accessing family planning in Malawi and their ideas to improve services. METHODS: We conducted 34 focus group discussions with youth aged 15-24 and parents or legal guardians of female youth in 3 districts in Malawi. Focus groups were translated and transcribed. Data was input into Dedoose and analyzed using a thematic framework to identify broader patterns and themes. RESULTS: Youth participants felt motivated to use family planning to protect themselves from sexually transmitted diseases and to prevent unwanted pregnancies. Females focused on the consequences of unplanned pregnancies and believed family planning services were targeted primarily at them, while males thought family planning services targeted males and females equally. Barriers to youth accessing family planning included contraception misconceptions, the costs of family planning services, and negative attitudes. Parents had mixed views on family planning. While many parents acknowledged they could play a role in supporting youth, most said they are reluctant to support youth using family planning. Participants said improving counseling services, integrating family planning services and education within school curricula, and utilizing youth clubs could improve family planning services for youth. CONCLUSIONS: Policy makers and program implementers should consider the diverse preferences among youth and parents and continue seeking their input when designing policies and programs. Youth clubs and school-based services were among the most common suggestions. However, the effectiveness of youth clubs and school-based initiatives to increase contraceptive use among youth in Malawi is not clear.

National Assessment of Data Quality and Associated Systems-Level Factors in Malawi.

BACKGROUND: Routine health data can guide health systems improvements, but poor quality of these data hinders use. To address concerns about data quality in Malawi, the Ministry of Health and National Statistical Office conducted a data quality assessment (DQA) in July 2016 to identify systems-level factors that could be improved. METHODS: We used 2-stage stratified random sampling methods to select health centers and hospitals under Ministry of Health auspices, included those managed by faith-based entities, for this DQA. Dispensaries, village clinics, police and military facilities, tertiary-level hospitals, and private facilities were excluded. We reviewed client registers and monthly reports to verify availability, completeness, and accuracy of data in 4 service areas: antenatal care (ANC), family planning, HIV testing and counseling, and acute respiratory infection (ARI). We also conducted interviews with facility and district personnel to assess health management information system (HMIS) functioning and systems-level factors that may be associated with data quality. We compared systems and quality factors by facility characteristics using 2-sample t tests with Welch's approximation, and calculated verification ratios comparing total entries in registers to totals from summarized reports. RESULTS: We selected 16 hospitals (of 113 total in Malawi), 90 health centers (of 466), and 16 district health offices (of 28) in 16 of Malawi's 28 districts. Nearly all registers were available and complete in health centers and district hospitals, but data quality varied across service areas; median verification ratios comparing register and report totals at health centers ranged from 0.78 (interquartile range [IQR]: 0.25, 1.07) for ARI and 0.99 (IQR: 0.82, 1.36) for family planning to 1.00 (IQR: 0.96, 1.00) for HIV testing and counseling and 1.00 (IQR: 0.80, 1.23) for ANC. More than half (60%) of facilities reported receiving a documented supervisory visit for HMIS in the prior 6 months. A recent supervision visit was associated with better availability of data (P=.05), but regular district- or central-level supervision was not. Use of data by the facility to track performance toward targets was associated with both improved availability (P=.04) and completeness of data (P=.02). Half of facilities had a full-time statistical clerk, but their presence did not improve the availability or completeness of data (P=.39 and P=.69, respectively). CONCLUSION: Findings indicate both strengths and weaknesses in Malawi's HMIS performance, with key weaknesses including infrequent data quality checks and unreliable supervision. Efforts to strengthen HMIS in low- and middle-income countries should be informed by similar assessments.