Outcomes and Costs of the Transition From a Paper-Based Immunization System to a Digital Immunization System in Vietnam: Mixed Methods Study.

BACKGROUND: The electronic National Immunization Information System (NIIS) was introduced nationwide in Vietnam in 2017. Health workers were expected to use the NIIS alongside the legacy paper-based system. Starting in 2018, Hanoi and Son La provinces transitioned to paperless reporting. Interventions to support this transition included data guidelines and training, internet-based data review meetings, and additional supportive supervision visits. OBJECTIVE: This study aims to assess (1) changes in NIIS data quality and use, (2) changes in immunization program outcomes, and (3) the economic costs of using the NIIS versus the traditional paper system. METHODS: This mixed methods study took place in Hanoi and Son La provinces. It aimed to analyses pre- and postintervention data from various sources including the NIIS; household and health facility surveys; and interviews to measure NIIS data quality, data use, and immunization program outcomes. Financial data were collected at the national, provincial, district, and health facility levels through record review and interviews. An activity-based costing approach was conducted from a health system perspective. RESULTS: NIIS data timeliness significantly improved from pre- to postintervention in both provinces. For example, the mean number of days from birth date to NIIS registration before and after intervention dropped from 18.6 (SD 65.5) to 5.7 (SD 31.4) days in Hanoi (P<.001) and from 36.1 (SD 94.2) to 11.7 (40.1) days in Son La (P<.001). Data from Son La showed that the completeness and accuracy improved, while Hanoi exhibited mixed results, possibly influenced by the COVID-19 pandemic. Data use improved; at postintervention, 100% (667/667) of facilities in both provinces used NIIS data for activities beyond monthly reporting compared with 34.8% (202/580) in Hanoi and 29.4% (55/187) in Son La at preintervention. Across nearly all antigens, the percentage of children who received the vaccine on time was higher in the postintervention cohort compared with the preintervention cohort. Up-front costs associated with developing and deploying the NIIS were estimated at US $0.48 per child in the study provinces. The commune health center level showed cost savings from changing from the paper system to the NIIS, mainly driven by human resource time savings. At the administrative level, incremental costs resulted from changing from the paper system to the NIIS, as some costs increased, such as labor costs for supportive supervision and additional capital costs for equipment associated with the NIIS. CONCLUSIONS: The Hanoi and Son La provinces successfully transitioned to paperless reporting while maintaining or improving NIIS data quality and data use. However, improvements in data quality were not associated with improvements in the immunization program outcomes in both provinces. The COVID-19 pandemic likely had a negative influence on immunization program outcomes, particularly in Hanoi. These improvements entail up-front financial costs.

Changes in on-time vaccination following the introduction of an electronic immunization registry, Tanzania 2016-2018: interrupted time-series analysis.

BACKGROUND: Digital health interventions (DHI) have the potential to improve the management and utilization of health information to optimize health care worker performance and provision of care. Despite the proliferation of DHI projects in low-and middle-income countries, few have been evaluated in an effort to understand their impact on health systems and health-related outcomes. Although more evidence is needed on their impact and effectiveness, the use of DHIs among immunization programs has become more widespread and shows promise for improving vaccination uptake and adherence to immunization schedules. METHODS: Our aim was to assess the impact of an electronic immunization registry (EIR) using an interrupted time-series analysis to analyze the effect on proportion of on-time vaccinations following introduction of an EIR in Tanzania. We hypothesized that the introduction of the EIR would lead to statistically significant changes in vaccination timeliness at 3, 6, and > 6 months post-introduction. RESULTS: For our primary analysis, we observed a decrease in the proportion of on-time vaccinations following EIR introduction. In contrast, our sensitivity analysis estimated improvements in timeliness among those children with complete vaccination records. However, we must emphasize caution interpreting these findings as they are likely affected by implementation challenges. CONCLUSIONS: This study highlights the complexities of using digitized individual-level routine health information system data for evaluation and research purposes. EIRs have the potential to improve vaccination timeliness, but analyses using EIR data can be complicated by data quality issues and inconsistent data entry leading to difficulties interpreting findings.

Determinants of Scale-up From a Small Pilot to a National Electronic Immunization Registry in Vietnam: Qualitative Evaluation.

BACKGROUND: Digital health innovations can improve health system performance, yet previous experience has shown that many innovations do not advance beyond the pilot stage to achieve scale. Vietnam's National Immunization Information System (NIIS) began as a series of digital health pilots, first initiated in 2010, and was officially launched nationwide in 2017. The NIIS is one of the few examples of an electronic immunization registry (EIR) at national scale in low- and middle-income countries. OBJECTIVE: The aim of this study was to understand the determinants of scale-up of the national EIR in Vietnam. METHODS: This qualitative study explored the facilitators and barriers to national scale-up of the EIR in Vietnam. Qualitative data were collected from October to December 2019 through in-depth key informant interviews and desk review. The mHealth Assessment and Planning for Scale (MAPS) Toolkit guided the development of the study design, interview guides, and analytic framework. MAPS defines the key determinants of success, or the "axes of scale," to be groundwork, partnerships, financial health, technology and architecture, operations, and monitoring and evaluation. RESULTS: The partnership and operations axes were critical to the successful scale-up of the EIR in Vietnam, while the groundwork and monitoring and the evaluation axes were considered to be strong contributors in the success of all the other axes. The partnership model leveraged complementary strengths of the technical working group partners: the Ministry of Health General Department of Preventive Medicine, the National Expanded Program on Immunization, Viettel (the mobile network operator), and PATH. The operational approach to introducing the NIIS with lean, iterative, and integrated training and supervision was also a key facilitator to successful scale-up. The financial health, technology and architecture, and operations axes were identified as barriers to successful deployment and scale-up. Key barriers to scale-up included insufficient estimates of operational costs, unanticipated volume of data storage and transmission, lack of a national ID to support interoperability, and operational challenges among end users. Overall, the multiple phases of EIR deployment and scale-up from 2010 to 2017 allowed for continuous learning and improvement that strengthened all the axes and contributed to successful scale-up. CONCLUSIONS: The results highlight the importance of the measured, iterative approach that was taken to gradually expand a series of small pilots to nationwide scale. The findings from this study can be used to inform other countries considering, introducing, or in the process of scaling an EIR or other digital health innovations.

HDAC8 regulates long-term hematopoietic stem-cell maintenance under stress by modulating p53 activity.

The maintenance and functional integrity of long-term hematopoietic stem cells (LT-HSCs) is critical for lifelong hematopoietic regeneration. Histone deacetylases (HDACs) modulate acetylation of lysine residues, a protein modification important for regulation of numerous biological processes. Here, we show that Hdac8 is most highly expressed in the phenotypic LT-HSC population within the adult hematopoietic hierarchy. Using an Hdac8-floxed allele and a dual-fluorescence Cre reporter allele, largely normal hematopoietic differentiation capacity of Hdac8-deficient cells was observed. However, the frequency of phenotypic LT-HSC population was significantly higher shortly after Hdac8 deletion, and the expansion had shifted to the phenotypic multipotent progenitor population by 1 year. We show that Hdac8-deficient hematopoietic progenitors are compromised in colony-forming cell serial replating in vitro and long-term serial repopulating activity in vivo. Mechanistically, we demonstrate that the HDAC8 protein interacts with the p53 protein and modulates p53 activity via deacetylation. Hdac8-deficient LT-HSCs displayed hyperactivation of p53 and increased apoptosis under genotoxic and hematopoietic stress. Genetic inactivation of p53 reversed the increased apoptosis and elevated expression of proapoptotic targets Noxa and Puma seen in Hdac8-deleted LT-HSCs. Dramatically compromised hematopoietic recovery and increased lethality were seen in Hdac8-deficient mice challenged with serial 5-fluorouracil treatment. This hypersensitivity to hematopoietic ablation was completely rescued by inactivation of p53. Altogether, these results indicate that HDAC8 functions to modulate p53 activity to ensure LT-HSC maintenance and cell survival under stress.

Validation of a new predictive risk model: measuring the impact of the major modifiable risks of death for patients and populations.

BACKGROUND: Modifiable risks account for a large fraction of disease and death, but clinicians and patients lack tools to identify high risk populations or compare the possible benefit of different interventions. METHODS: We used data on the distribution of exposure to 12 major behavioral and biometric risk factors inthe US population, mortality rates by cause, and estimates of the proportional hazards of risk factor exposure from published systematic reviews to develop a risk prediction model that estimates an adult's 10 year mortality risk compared to a population with optimum risk factors. We compared predicted risk to observed mortality in 8,241 respondents in NHANES 1988-1994 and NHANES 1999-2004 with linked mortality data up to the end of 2006. RESULTS: Predicted risk showed good discrimination with an area under the receiver operating characteristic (ROC) curve of 0.84 (standard error 0.01) for women and 0.84 (SE 0.01) for men. Across deciles of predicted risk, mortality was accurately predicted in men ((Χ (2) statistic = 12.3 for men, p=0.196) but slightly overpredicted in the highest decile among women (Χ (2) statistic = 22.8, p=0.002). Mortality risk was highly concentrated; for example, among those age 30-44 years, 5.1 % (95 % CI 4.1 % - 6.0 %) of the male and 5.9 % (95 % CI 4.8 % - 6.9 %) of the female population accounted for 25 % of the risk of death. CONCLUSION: The risk model accurately predicted mortality in a representative sample of the US population and could be used to help inform patient and provider decision-making, identify high risk groups, and monitor the impact of efforts to improve population health.

The global burden of musculoskeletal conditions for 2010: an overview of methods.

The objective of this paper is to provide an overview of methods used for estimating the burden from musculoskeletal (MSK) conditions in the Global Burden of Diseases 2010 study. It should be read in conjunction with the disease-specific MSK papers published in Annals of Rheumatic Diseases. Burden estimates (disability-adjusted life years (DALYs)) were made for five specific MSK conditions: hip and/or knee osteoarthritis (OA), low back pain (LBP), rheumatoid arthritis (RA), gout and neck pain, and an 'other MSK conditions' category. For each condition, the main disabling sequelae were identified and disability weights (DW) were derived based on short lay descriptions. Mortality (years of life lost (YLLs)) was estimated for RA and the rest category of 'other MSK', which includes a wide range of conditions such as systemic lupus erythematosus, other autoimmune diseases and osteomyelitis. A series of systematic reviews were conducted to determine the prevalence, incidence, remission, duration and mortality risk of each condition. A Bayesian meta-regression method was used to pool available data and to predict prevalence values for regions with no or scarce data. The DWs were applied to prevalence values for 1990, 2005 and 2010 to derive years lived with disability. These were added to YLLs to quantify overall burden (DALYs) for each condition. To estimate the burden of MSK disease arising from risk factors, population attributable fractions were determined for bone mineral density as a risk factor for fractures, the occupational risk of LBP and elevated body mass index as a risk factor for LBP and OA. Burden of Disease studies provide pivotal guidance for governments when determining health priority areas and allocating resources. Rigorous methods were used to derive the increasing global burden of MSK conditions.

A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990-2010: a systematic analysis for the Global Burden of Disease Study 2010.

BACKGROUND: Quantification of the disease burden caused by different risks informs prevention by providing an account of health loss different to that provided by a disease-by-disease analysis. No complete revision of global disease burden caused by risk factors has been done since a comparative risk assessment in 2000, and no previous analysis has assessed changes in burden attributable to risk factors over time. METHODS: We estimated deaths and disability-adjusted life years (DALYs; sum of years lived with disability [YLD] and years of life lost [YLL]) attributable to the independent effects of 67 risk factors and clusters of risk factors for 21 regions in 1990 and 2010. We estimated exposure distributions for each year, region, sex, and age group, and relative risks per unit of exposure by systematically reviewing and synthesising published and unpublished data. We used these estimates, together with estimates of cause-specific deaths and DALYs from the Global Burden of Disease Study 2010, to calculate the burden attributable to each risk factor exposure compared with the theoretical-minimum-risk exposure. We incorporated uncertainty in disease burden, relative risks, and exposures into our estimates of attributable burden. FINDINGS: In 2010, the three leading risk factors for global disease burden were high blood pressure (7·0% [95% uncertainty interval 6·2-7·7] of global DALYs), tobacco smoking including second-hand smoke (6·3% [5·5-7·0]), and alcohol use (5·5% [5·0-5·9]). In 1990, the leading risks were childhood underweight (7·9% [6·8-9·4]), household air pollution from solid fuels (HAP; 7·0% [5·6-8·3]), and tobacco smoking including second-hand smoke (6·1% [5·4-6·8]). Dietary risk factors and physical inactivity collectively accounted for 10·0% (95% UI 9·2-10·8) of global DALYs in 2010, with the most prominent dietary risks being diets low in fruits and those high in sodium. Several risks that primarily affect childhood communicable diseases, including unimproved water and sanitation and childhood micronutrient deficiencies, fell in rank between 1990 and 2010, with unimproved water and sanitation accounting for 0·9% (0·4-1·6) of global DALYs in 2010. However, in most of sub-Saharan Africa childhood underweight, HAP, and non-exclusive and discontinued breastfeeding were the leading risks in 2010, while HAP was the leading risk in south Asia. The leading risk factor in Eastern Europe, most of Latin America, and southern sub-Saharan Africa in 2010 was alcohol use; in most of Asia, North Africa and Middle East, and central Europe it was high blood pressure. Despite declines, tobacco smoking including second-hand smoke remained the leading risk in high-income north America and western Europe. High body-mass index has increased globally and it is the leading risk in Australasia and southern Latin America, and also ranks high in other high-income regions, North Africa and Middle East, and Oceania. INTERPRETATION: Worldwide, the contribution of different risk factors to disease burden has changed substantially, with a shift away from risks for communicable diseases in children towards those for non-communicable diseases in adults. These changes are related to the ageing population, decreased mortality among children younger than 5 years, changes in cause-of-death composition, and changes in risk factor exposures. New evidence has led to changes in the magnitude of key risks including unimproved water and sanitation, vitamin A and zinc deficiencies, and ambient particulate matter pollution. The extent to which the epidemiological shift has occurred and what the leading risks currently are varies greatly across regions. In much of sub-Saharan Africa, the leading risks are still those associated with poverty and those that affect children. FUNDING: Bill & Melinda Gates Foundation.

Can breastfeeding promote child health equity? A comprehensive analysis of breastfeeding patterns across the developing world and what we can learn from them.

BACKGROUND: In 2010 more than 7.7 million children died before their fifth birthday. Over 98% of these deaths occurred in developing countries, and recent estimates have attributed hundreds of thousands of these deaths to suboptimal breastfeeding. METHODS: This study estimated prevalence of suboptimal breastfeeding for 137 developing countries from 1990 to 2010. These estimates were compared against WHO infant feeding recommendations and combined with effect sizes from existing literature to estimate associated disease burden using a standard comparative risk assessment approach. These prevalence estimates were disaggregated by wealth quintile and linked with child mortality rates to assess how improved rates of breastfeeding may affect child health inequalities. RESULTS: In 2010, the prevalence of exclusive breastfeeding ranged from 3.5% in Djibouti to 77.3% in Rwanda. The proportion of child Disability Adjusted Life Years (DALYs) attributable to suboptimal breastfeeding is 7.6% at the global level and as high as 20.2% in Swaziland. Suboptimal breastfeeding is a leading childhood risk factor in all developing countries and consistently ranks higher than water and sanitation. Within most countries, breastfeeding prevalence rates do not vary considerably across wealth quintiles. CONCLUSIONS: Breastfeeding is an effective child health intervention that does not require extensive health system infrastructure. Improvements in rates of exclusive and continued breastfeeding can contribute to the reduction of child mortality inequalities in developing countries.

Mortality attributable to smoking in Vietnamese men in 2008.

OBJECTIVE: Smoking prevalence among Vietnamese men is among the highest in the world. Our aim was to provide estimates of tobacco attributable mortality to support tobacco control policies. METHOD: We used the Peto-Lopez method using lung cancer mortality to derive a Smoking Impact Ratio (SIR) as a marker of cumulative exposure to smoking. SIRs were applied to relative risks from the Cancer Prevention Study, Phase II. Prevalence-based and hybrid methods, using the SIR for cancers and chronic obstructive pulmonary disease and smoking prevalence for all other outcomes, were used in sensitivity analyses. RESULTS: When lung cancer was used to measure cumulative smoking exposure, 28% (95% uncertainty interval 24-31%) of all adult male deaths (>35 years) in Vietnam in 2008 were attributable to smoking. Lower estimates resulted from prevalence-based methods [24% (95% uncertainty interval 21-26%)] with the hybrid method yielding intermediate estimates [26% (95% uncertainty interval 23-28%)]. CONCLUSION: Despite uncertainty in these estimates of attributable mortality, tobacco smoking is already a major risk factor for death in Vietnamese men. Given the high current prevalence of smoking, this has important implications not only for preventing the uptake of tobacco but also for immediate action to adopt and enforce stronger tobacco control measures.

Design, Development, and Deployment of an Electronic Immunization Registry: Experiences From Vietnam, Tanzania, and Zambia.

INTRODUCTION: There is growing interest among low- and middle-income countries to introduce electronic immunization registries (EIRs) that capture individual-level vaccine data. We compare the design, development, and deployment of EIRs in Vietnam, Tanzania, and Zambia. Through desk review and the authors' firsthand implementation experiences, we describe experiences related to timeline, partnerships, financial costs, and technology and infrastructure. IMPLEMENTATION EXPERIENCE: The country cases highlight the multi-year timeline required to implement an EIR at scale and the benefit of multiple iterative cycles to pilot and redesign the system before achieving scale. Of the 3 countries, only Vietnam has achieved nationwide scale of the EIR, which took 7 years. In all 3 countries, national government leadership as part of an interdisciplinary team (with experience in leadership, technology, and immunization) was important to ensure country ownership and sustainability. Where international software developers were contracted, partnering with a local software company helped improve responsiveness and sustainability. Across all 3 countries, governments contributed significant in-kind time in addition to investments from donors. Cost savings were observed in Tanzania and Zambia, largely driven by health worker time savings from using the EIR. All 3 case countries underscore the need to understand the local technology and infrastructure context and design the EIR to fit the context. In Vietnam, an initial landscape assessment was conducted to assess technology and infrastructure, whereas in Tanzania and Zambia, user advisory groups provided insights. Existing infrastructure informed EIR design decisions, such as choosing a system with offline functionality in Tanzania and Zambia. All 3 countries have a local partner to provide ongoing technical support. CONCLUSION: Comparing implementation factors across these cases highlights practical experience and recommendations that complement existing EIR guidance documents. The findings and recommendations from this study can inform other countries considering or in the process of implementing an EIR.

Added Value of Electronic Immunization Registries in Low- and Middle-Income Countries: Observational Case Study in Tanzania.

BACKGROUND: There is growing interest and investment in electronic immunization registries (EIRs) in low- and middle-income countries. EIRs provide ready access to patient- and aggregate-level service delivery data that can be used to improve patient care, identify spatiotemporal trends in vaccination coverage and dropout, inform resource allocation and program operations, and target quality improvement measures. The Government of Tanzania introduced the Tanzania Immunization Registry (TImR) in 2017, and the system has since been rolled out in 3736 facilities in 15 regions. OBJECTIVE: The aims of this study are to conceptualize the additional ways in which EIRs can add value to immunization programs (beyond measuring vaccine coverage) and assess the potential value-add using EIR data from Tanzania as a case study. METHODS: This study comprised 2 sequential phases. First, a comprehensive list of ways EIRs can potentially add value to immunization programs was developed through stakeholder interviews. Second, the added value was evaluated using descriptive and regression analyses of TImR data for a prioritized subset of program needs. RESULTS: The analysis areas prioritized through stakeholder interviews were population movement, missed opportunities for vaccination (MOVs), continuum of care, and continuous quality improvement. The included TImR data comprised 958,870 visits for 559,542 patients from 2359 health facilities. Our analyses revealed that few patients sought care outside their assigned facility (44,733/810,568, 5.52% of applicable visits); however, this varied by region; facility urbanicity, type, ownership, patient volume, and duration of TImR system use; density of facilities in the immediate area; and patient age. Analyses further showed that MOVs were highest among children aged <12 months (215,576/831,018, 25.94% of visits included an MOV and were applicable visits); however, there were few significant differences based on other individual or facility characteristics. Nearly half (133,337/294,464, 45.28%) of the children aged 12 to 35 months were fully vaccinated or had received all doses except measles-containing vaccine-1 of the 14-dose under-12-month schedule (ie, through measles-containing vaccine-1), and facility and patient characteristics associated with dropout varied by vaccine. The continuous quality improvement analysis showed that most quality issues (eg, MOVs) were concentrated in <10% of facilities, indicating the potential for EIRs to target quality improvement efforts. CONCLUSIONS: EIRs have the potential to add value to immunization stakeholders at all levels of the health system. Individual-level electronic data can enable new analyses to understand service delivery or care-seeking patterns, potential risk factors for underimmunization, and where challenges occur. However, to achieve this potential, country programs need to leverage and strengthen the capacity to collect, analyze, interpret, and act on the data. As EIRs are introduced and scaled in low- and middle-income countries, implementers and researchers should continue to share real-world examples and build an evidence base for how EIRs can add value to immunization programs, particularly for innovative uses.

Coverage and Drivers to Reaching the Last Child With Vaccination in Urban Settings: A Mixed-Methods Study in Kampala, Uganda.

BACKGROUND: Limited evidence exists regarding the drivers of vaccination coverage and equity in Kampala city, despite frequent measles outbreaks, inequities in vaccination coverage, and the decline in vaccination coverage rates. This study was designed to determine vaccine coverage among children aged 12-36 months and to understand its demand-side drivers. METHODS: We utilized a mixed-methods parallel convergent study design. A household survey was conducted to quantify the drivers of vaccine coverage among households with children aged 12-36 months. We employed a multistage sampling approach to select households, using a primary sampling unit of an enumeration area. We conducted 30 key informant interviews, 7 focus group discussions, and 6 in-depth interviews with representatives from the immunization program, health workers, and parents residing in areas with low vaccine coverage. RESULTS: Of the 590 enrolled children, 340 (57.6%) were partially vaccinated, 244 (41.4%) were fully vaccinated and had received all the recommended vaccinations, and 6 (1.0%) had never received any vaccine. Of the 244 with all recommended vaccinations, only 65 (26.6%) received their vaccines on time. Access to vaccination services was high (first dose of diphtheria, pertussis, and tetanus [DPT1] coverage of 96%), but utilization decreased over time, as shown by a dropout rate of 17.3% from the first to third dose of DPT. The main driver of complete vaccination was the parents' appreciation of the benefits of vaccination. Among partially vaccinated children, the barriers to vaccination were inadequate information about vaccination (its benefits and schedule), vaccine stock-outs, long waiting times to receive vaccination services, and hidden vaccination costs. CONCLUSION: Vaccination needs to be targeted to all children irrespective of whether they reside in slum areas or nonslum areas, as most are under-vaccinated. Social mobilization and communication efforts should be tailored to the complexities of urban settings characterized by transient and diverse populations with different cultures.

Targeting miR-126 in inv(16) acute myeloid leukemia inhibits leukemia development and leukemia stem cell maintenance.

Acute myeloid leukemia (AML) harboring inv(16)(p13q22) expresses high levels of miR-126. Here we show that the CBFB-MYH11 (CM) fusion gene upregulates miR-126 expression through aberrant miR-126 transcription and perturbed miR-126 biogenesis via the HDAC8/RAN-XPO5-RCC1 axis. Aberrant miR-126 upregulation promotes survival of leukemia-initiating progenitors and is critical for initiating and maintaining CM-driven AML. We show that miR-126 enhances MYC activity through the SPRED1/PLK2-ERK-MYC axis. Notably, genetic deletion of miR-126 significantly reduces AML rate and extends survival in CM knock-in mice. Therapeutic depletion of miR-126 with an anti-miR-126 (miRisten) inhibits AML cell survival, reduces leukemia burden and leukemia stem cell (LSC) activity in inv(16) AML murine and xenograft models. The combination of miRisten with chemotherapy further enhances the anti-leukemia and anti-LSC activity. Overall, this study provides molecular insights for the mechanism and impact of miR-126 dysregulation in leukemogenesis and highlights the potential of miR-126 depletion as a therapeutic approach for inv(16) AML.

Determinants of Facility-Level Use of Electronic Immunization Registries in Tanzania and Zambia: An Observational Analysis.

BACKGROUND: As more countries transition from paper-based to electronic immunization registries (EIRs) to collect and track individual immunization data, guidance is needed for successful adoption and use of these systems. Little research is available on the determinants of EIR use soon after introduction. This observational study assesses the determinants of facility health care workers' use of new EIRs in Tanzania and Zambia, implemented during 2016 to 2018. METHODS: We used EIR data entered between 2016 and 2018 from 3 regions in Tanzania and 1 province in Zambia to measure weekly EIR system use for a total of 50,639 facility-weeks. We joined secondary data on facility characteristics and applied the Performance of Routine Information System Management framework to categorize characteristics as organizational, technical, or behavioral. We used a generalized estimating equations logistic regression model to assess facility characteristics as potential determinants of system use. RESULTS: In both countries, the estimated odds of weekly EIR use declined weekly after EIR introduction. In Tanzania, health centers and hospitals had increased odds of system use compared to dispensaries. For each additional health care worker trained in a facility during the EIR introduction, the estimated odds of weekly EIR use increased. Tanzanian facilities that had transitioned entirely to paperless reporting had higher odds of sustained use compared to those maintaining parallel electronic and paper-based reporting systems. In Zambia, distance from the district health office was significantly associated with decreasing odds of system use. There were significant differences in EIR use by district in both countries. DISCUSSION: The results highlight the importance of organizational and behavioral factors in explaining sustained EIR use. As EIRs are introduced in new settings, we recommend indicators of engagement and use be built directly into the system so they can be routinely monitored, and course corrections can be implemented as needed.

Perceptions of factors influencing the introduction and adoption of electronic immunization registries in Tanzania and Zambia: a mixed methods study.

BACKGROUND: As technology has become cheaper and more accessible, health programs are adopting digital health interventions (DHI) to improve the provision of and demand for health services. These interventions are complex and require strong coordination and support across different health system levels and government departments, and they need significant capacities in technology and information to be properly implemented. Electronic immunization registries (EIRs) are types of DHI used to capture, store, access, and share individual-level, longitudinal health information in digitized records. The BID Initiative worked in partnership with the governments of Tanzania and Zambia to introduce an EIR at the sub-national level in both countries within 5 years as part of a multi-component complex intervention package focusing on data use capacity-building. METHODS: We aimed to gather and describe learnings from the BID experience by conducting a framework-based mixed methods study to describe perceptions of factors that influenced scale-up of the EIR. Data were collected through key informant interviews, a desk review, EIRs, and health management information systems. We described how implementation of the EIRs fulfilled domains described in our conceptual framework and used cases to illustrate the relationships and relative influence of domains for scale-up and adoption of the EIR. RESULTS: We found that there was no single factor that seemed to influence the introduction or sustained adoption of the EIR as many of the factors were interrelated. For EIR introduction, strong strategic engagement among partners was important, while EIR adoption was influenced by adequate staffing at facilities, training, use of data for supervision, internet and electricity connectivity, and community sensitization. CONCLUSIONS: Organizations deploying DHIs in the future should consider how best to adapt their intervention to the existing ecosystem, including human resources and organizational capacity, as well as the changing technological landscape during planning and implementation.

Lessons Learned From Implementing Prospective, Multicountry Mixed-Methods Evaluations for Gavi and the Global Fund.

INTRODUCTION: As global health programs have become increasingly complex, corresponding evaluations must be designed to assess the full complexity of these programs. Gavi and the Global Fund have commissioned 2 such evaluations to assess the full spectrum of their investments using a prospective mixed-methods approach. We aim to describe lessons learned from implementing these evaluations. METHODS: This article presents a synthesis of lessons learned based on the Gavi and Global Fund prospective mixed-methods evaluations, with each evaluation considered a case study. The lessons are based on the evaluation team's experience from over 7 years (2013-2020) implementing these evaluations. The Centers for Disease Control and Prevention Framework for Evaluation in Public Health was used to ground the identification of lessons learned. RESULTS: We identified 5 lessons learned that build on existing evaluation best practices and include a mix of practical and conceptual considerations. The lessons cover the importance of (1) including an inception phase to engage stakeholders and inform a relevant, useful evaluation design; (2) aligning on the degree to which the evaluation is embedded in the program implementation; (3) monitoring programmatic, organizational, or contextual changes and adapting the evaluation accordingly; (4) hiring evaluators with mixed-methods expertise and using tools and approaches that facilitate mixing methods; and (5) contextualizing recommendations and clearly communicating their underlying strength of evidence. CONCLUSION: Global health initiatives, particularly those leveraging complex interventions, should consider embedding evaluations to understand how and why the programs are working. These initiatives can learn from the lessons presented here to inform the design and implementation of such evaluations.

How to evaluate the implementation of complex health programmes in low-income settings: the approach of the Gavi Full Country Evaluations.

Vaccination, like most other public health services, relies on a complex package of intervention components, functioning systems and committed actors to achieve universal coverage. Despite significant investment in immunization programmes, national coverage trends have slowed and equity gaps have grown. This paper describes the design and implementation of the Gavi Full Country Evaluations, a multi-country, prospective, mixed-methods approach whose goal was to monitor and evaluate processes, inputs, outputs and outcomes of immunization programmes in Bangladesh, Mozambique, Uganda and Zambia. We implemented the Full Country Evaluations from 2013 to 2018 with the goal of identifying the drivers of immunization programme improvement to support programme implementation and increase equitable immunization coverage. The framework supported methodological and paradigmatic flexibility to respond to a broad range of evaluation and implementation research questions at global, national and cross-country levels, but was primarily underpinned by a focus on evaluating processes and identifying the root causes of implementation breakdowns. Process evaluation was driven by theories of change for each Gavi funding stream (e.g. Health Systems Strengthening) or activity, ranging from global policy development to district-level programme implementation. Mixing of methods increased in relevance and rigour over time as we learned to build multiple methods into increasingly tailored evaluation questions. Evaluation teams in country-based research institutes increasingly strengthened their level of embeddedness with immunization programmes as the emphasis shifted over time to focus more heavily on the use of findings for programme learning and adaptation. Based on our experiences implementing this approach, we recommend it for the evaluation of other complex interventions, health programmes or development assistance.

State-Transition Analysis of Time-Sequential Gene Expression Identifies Critical Points That Predict Development of Acute Myeloid Leukemia.

Temporal dynamics of gene expression inform cellular and molecular perturbations associated with disease development and evolution. Given the complexity of high-dimensional temporal genomic data, an analytic framework guided by a robust theory is needed to interpret time-sequential changes and to predict system dynamics. Here we model temporal dynamics of the transcriptome of peripheral blood mononuclear cells in a two-dimensional state-space representing states of health and leukemia using time-sequential bulk RNA-seq data from a murine model of acute myeloid leukemia (AML). The state-transition model identified critical points that accurately predict AML development and identifies stepwise transcriptomic perturbations that drive leukemia progression. The geometry of the transcriptome state-space provided a biological interpretation of gene dynamics, aligned gene signals that are not synchronized in time across mice, and allowed quantification of gene and pathway contributions to leukemia development. Our state-transition model synthesizes information from multiple cell types in the peripheral blood and identifies critical points in the transition from health to leukemia to guide interpretation of changes in the transcriptome as a whole to predict disease progression. SIGNIFICANCE: These findings apply the theory of state transitions to model the initiation and development of acute myeloid leukemia, identifying transcriptomic perturbations that accurately predict time to disease development.See related commentary by Kuijjer, p. 3072 GRAPHICAL ABSTRACT: http://cancerres.aacrjournals.org/content/canres/80/15/3157/F1.large.jpg.

Estimating reach of social impact products: A model to standardize the calculation of product reach in data-scarce settings.

BACKGROUND: Social impact interventions often involve the introduction of a product intended to create positive impact. Program decision makers need data to routinely review product delivery as well as predict potential outcomes and impact to optimize intervention plans and allocate resources effectively. We propose a novel model to support data-driven decision-making in data and budget-constrained settings and use of routine monitoring to ensure progress towards program outcomes and impact. METHODS: We present a complete model to estimate product reach of durable and fast-moving consumer products, which includes required inputs, potential data sources, formulas, trade-offs, and assumptions. RESULTS: We illustrate the use of the model by applying it to the case study of fortified rice introduction in Brazil and estimate that the intervention, which aimed to improve nutrition status and health outcomes reached 2.4 million consumers. CONCLUSIONS: The model can cover a broad range of social-purpose interventions that involve the introduction or scale-up of various types of consumer products. It provides a relatively simple, comprehensive, flexible, and usable framework to estimate product reach, an indicator that can be an input into impact estimates or, in many scenarios, the actual endpoint of the intervention.