Outcomes and Costs of the Transition From a Paper-Based Immunization System to a Digital Immunization System in Vietnam: Mixed Methods Study.

BACKGROUND: The electronic National Immunization Information System (NIIS) was introduced nationwide in Vietnam in 2017. Health workers were expected to use the NIIS alongside the legacy paper-based system. Starting in 2018, Hanoi and Son La provinces transitioned to paperless reporting. Interventions to support this transition included data guidelines and training, internet-based data review meetings, and additional supportive supervision visits. OBJECTIVE: This study aims to assess (1) changes in NIIS data quality and use, (2) changes in immunization program outcomes, and (3) the economic costs of using the NIIS versus the traditional paper system. METHODS: This mixed methods study took place in Hanoi and Son La provinces. It aimed to analyses pre- and postintervention data from various sources including the NIIS; household and health facility surveys; and interviews to measure NIIS data quality, data use, and immunization program outcomes. Financial data were collected at the national, provincial, district, and health facility levels through record review and interviews. An activity-based costing approach was conducted from a health system perspective. RESULTS: NIIS data timeliness significantly improved from pre- to postintervention in both provinces. For example, the mean number of days from birth date to NIIS registration before and after intervention dropped from 18.6 (SD 65.5) to 5.7 (SD 31.4) days in Hanoi (P<.001) and from 36.1 (SD 94.2) to 11.7 (40.1) days in Son La (P<.001). Data from Son La showed that the completeness and accuracy improved, while Hanoi exhibited mixed results, possibly influenced by the COVID-19 pandemic. Data use improved; at postintervention, 100% (667/667) of facilities in both provinces used NIIS data for activities beyond monthly reporting compared with 34.8% (202/580) in Hanoi and 29.4% (55/187) in Son La at preintervention. Across nearly all antigens, the percentage of children who received the vaccine on time was higher in the postintervention cohort compared with the preintervention cohort. Up-front costs associated with developing and deploying the NIIS were estimated at US $0.48 per child in the study provinces. The commune health center level showed cost savings from changing from the paper system to the NIIS, mainly driven by human resource time savings. At the administrative level, incremental costs resulted from changing from the paper system to the NIIS, as some costs increased, such as labor costs for supportive supervision and additional capital costs for equipment associated with the NIIS. CONCLUSIONS: The Hanoi and Son La provinces successfully transitioned to paperless reporting while maintaining or improving NIIS data quality and data use. However, improvements in data quality were not associated with improvements in the immunization program outcomes in both provinces. The COVID-19 pandemic likely had a negative influence on immunization program outcomes, particularly in Hanoi. These improvements entail up-front financial costs.

Determinants of Scale-up From a Small Pilot to a National Electronic Immunization Registry in Vietnam: Qualitative Evaluation.

BACKGROUND: Digital health innovations can improve health system performance, yet previous experience has shown that many innovations do not advance beyond the pilot stage to achieve scale. Vietnam's National Immunization Information System (NIIS) began as a series of digital health pilots, first initiated in 2010, and was officially launched nationwide in 2017. The NIIS is one of the few examples of an electronic immunization registry (EIR) at national scale in low- and middle-income countries. OBJECTIVE: The aim of this study was to understand the determinants of scale-up of the national EIR in Vietnam. METHODS: This qualitative study explored the facilitators and barriers to national scale-up of the EIR in Vietnam. Qualitative data were collected from October to December 2019 through in-depth key informant interviews and desk review. The mHealth Assessment and Planning for Scale (MAPS) Toolkit guided the development of the study design, interview guides, and analytic framework. MAPS defines the key determinants of success, or the "axes of scale," to be groundwork, partnerships, financial health, technology and architecture, operations, and monitoring and evaluation. RESULTS: The partnership and operations axes were critical to the successful scale-up of the EIR in Vietnam, while the groundwork and monitoring and the evaluation axes were considered to be strong contributors in the success of all the other axes. The partnership model leveraged complementary strengths of the technical working group partners: the Ministry of Health General Department of Preventive Medicine, the National Expanded Program on Immunization, Viettel (the mobile network operator), and PATH. The operational approach to introducing the NIIS with lean, iterative, and integrated training and supervision was also a key facilitator to successful scale-up. The financial health, technology and architecture, and operations axes were identified as barriers to successful deployment and scale-up. Key barriers to scale-up included insufficient estimates of operational costs, unanticipated volume of data storage and transmission, lack of a national ID to support interoperability, and operational challenges among end users. Overall, the multiple phases of EIR deployment and scale-up from 2010 to 2017 allowed for continuous learning and improvement that strengthened all the axes and contributed to successful scale-up. CONCLUSIONS: The results highlight the importance of the measured, iterative approach that was taken to gradually expand a series of small pilots to nationwide scale. The findings from this study can be used to inform other countries considering, introducing, or in the process of scaling an EIR or other digital health innovations.

Design, Development, and Deployment of an Electronic Immunization Registry: Experiences From Vietnam, Tanzania, and Zambia.

INTRODUCTION: There is growing interest among low- and middle-income countries to introduce electronic immunization registries (EIRs) that capture individual-level vaccine data. We compare the design, development, and deployment of EIRs in Vietnam, Tanzania, and Zambia. Through desk review and the authors' firsthand implementation experiences, we describe experiences related to timeline, partnerships, financial costs, and technology and infrastructure. IMPLEMENTATION EXPERIENCE: The country cases highlight the multi-year timeline required to implement an EIR at scale and the benefit of multiple iterative cycles to pilot and redesign the system before achieving scale. Of the 3 countries, only Vietnam has achieved nationwide scale of the EIR, which took 7 years. In all 3 countries, national government leadership as part of an interdisciplinary team (with experience in leadership, technology, and immunization) was important to ensure country ownership and sustainability. Where international software developers were contracted, partnering with a local software company helped improve responsiveness and sustainability. Across all 3 countries, governments contributed significant in-kind time in addition to investments from donors. Cost savings were observed in Tanzania and Zambia, largely driven by health worker time savings from using the EIR. All 3 case countries underscore the need to understand the local technology and infrastructure context and design the EIR to fit the context. In Vietnam, an initial landscape assessment was conducted to assess technology and infrastructure, whereas in Tanzania and Zambia, user advisory groups provided insights. Existing infrastructure informed EIR design decisions, such as choosing a system with offline functionality in Tanzania and Zambia. All 3 countries have a local partner to provide ongoing technical support. CONCLUSION: Comparing implementation factors across these cases highlights practical experience and recommendations that complement existing EIR guidance documents. The findings and recommendations from this study can inform other countries considering or in the process of implementing an EIR.

The Transition to an Entirely Digital Immunization Registry in Ha Noi Province and Son La Province, Vietnam: Readiness Assessment Study.

BACKGROUND: Vietnam is one of the first low- to middle-income countries to develop and implement a national-scale electronic immunization registry. This system was finalized into the National Immunization Information System (NIIS) and scaled up to a national-level system in 2017. As a result, immunization coverage and the timeliness of vaccinations have drastically improved. The time spent on planning and reporting vaccinations has drastically reduced; as a result, vaccination planning and reporting has become more accurate and effective. However, to date, end users have been tasked with managing both the NIIS and paper-based systems in parallel until a formal assessment of the readiness to fully transition to the NIIS is conducted. OBJECTIVE: This study aims to evaluate the readiness to move to an entirely digital NIIS in 2 provinces of Vietnam-Ha Noi and Son La. METHODS: All health facilities were surveyed to assess their infrastructure, capacity, and need for human resources. NIIS end users were observed and interviewed to evaluate their NIIS knowledge and skill sets. Data from immunization cards and facility paper-based logbooks were compared with data from the NIIS, and vaccine stocks at selected facilities were tallied and compared with data from the NIIS. RESULTS: Of the 990 health facilities evaluated, most used the NIIS to enter and track immunizations (987/990, 99.7%) and vaccine stocks (889/990, 90.8%). Most had stable electricity (971/990, 98.1%), at least 1 computer (986/990, 99.6%), and ≥2 trained NIIS end users (825/990, 83.3%). End users reported that the NIIS supported them in managing and reporting immunization data and saving them time (725/767, 94.5%). Although many end users were able to perform basic skills, almost half struggled with performing more complex tasks. Immunization data were compiled from the NIIS and immunization cards (338/378, 89.4%) and paper-based logbooks (254/269, 94.4%). However, only 54.5% (206/378) of immunization IDs matched, 57% (13/23) of Bacillus Calmette-Guérin vaccination records were accurate, and 70% (21/30) of the facilities had consistent physical vaccine stock balances. The feedback received from NIIS end users suggests that more supportive supervision, frequent refresher training for strengthening their skill sets, and detailed standardized guides for improving data quality are needed. CONCLUSIONS: The readiness to transition to a digital system is promising; however, additional resources are required to address the timeliness, completeness, and accuracy of the data.

Health Care Worker Preferences and Perspectives on Doses per Container for 2 Lyophilized Vaccines in Senegal, Vietnam, and Zambia.

INTRODUCTION: Limited information exists on health care workers' (HCWs) perceptions about use of multidose vaccine vials and their preferences about doses per container (DPC). We present findings from qualitative studies conducted in Senegal, Vietnam, and Zambia to explore HCWs' behavior regarding opening vials and their perceptions and preferences for the number of doses in vials of BCG and measles-containing vaccine (MCV). Zambia and Senegal currently offer MCV in 10-dose vials and BCG in 20-dose vials; 10-dose vials are used for both vaccines in Vietnam. Unused doses in vials of these reconstituted vaccines must be discarded within 6 hours. METHODS: Key informant interviews (KIIs) were conducted with frontline HCWs in Senegal, Vietnam, and Zambia. In Senegal and Vietnam, the KIIs were conducted as part of broader formative research; in Zambia, KIIs were conducted in control districts using 10-dose MCV vials only and in intervention districts that switched from 10- to 5-dose vials during the study. During analysis, themes common to all 3 countries were synthesized. Critical themes relevant to country contexts were also examined. RESULTS: HCWs in all 3 countries preferred containers with fewer doses for BCG and MCV to reduce wastage and increase the likelihood of vaccinating every eligible child. HCWs in Senegal and HCWs using 10-dose vials in Zambia reported sending unvaccinated children away because not enough children were present to warrant opening a new vial. In Vietnam, where sessions are typically held monthly, and in Zambia when the 5-dose vials were used, almost all HCWs reported opening a vial of MCV for even 1 child. DISCUSSION: HCWs prefer vials with fewer DPC. Their concerns about balancing coverage and wastage influence their decisions to vaccinate every eligible child; and their perspectives are crucial to ensuring that all target populations are reached with vaccines in a timely manner.

Digital immunization registry: evidence for the impact of mHealth on enhancing the immunization system and improving immunization coverage for children under one year old in Vietnam.

BACKGROUND: The Vietnam National Expanded Program on Immunization (NEPI) has been successfully implementing a nationwide immunization system since 1985. From the start, the program has increased the immunization coverage rate; however, data on immunization coverage in Vietnam are gathered and aggregated from commune health centers in routine, paper-based reports, which have shortcomings. Also, calculations of coverage are inconsistent at subnational levels, which lead to uncertainty about the size of the target population used as the denominator in coverage calculations. The growth of mobile networks in Vietnam provides an opportunity to apply mHealth to improve the immunization program. In 2012, PATH and the Vietnam NEPI developed and piloted a digital immunization registry, ImmReg, to overcome the challenges of the paper system. A final evaluation was conducted in 2015 to assess the impact of ImmReg, including its use of SMS reminders, on improving the immunization program. METHODS: The study population comprised all children born in Ben Tre province in September and October of 2013, 2014, and 2015, representing pre-intervention, post-intervention, and one year post-intervention, respectively. Data exported from ImmReg were used to compare the immunization rate, dropout rate, and timeliness of vaccination before and after the intervention. Additionally, a rapid survey was conducted to understand the willingness of parents with children due for vaccination to pay for SMS reminder messages on the immunization schedule. RESULTS: Timely administration of oral polio vaccine, Quinvaxem, and measles 1 vaccine significantly increased over time from baseline to post-intervention to one year post-intervention. In particular, the timeliness of vaccination with the third dose of Quinvaxem increased from 53.6% to 65.8% to 77.2%. For measles 1 vaccine, the rate increased from 70.4% to 76.2% to 92.3%. In addition, the dropout rate from Quinvaxem 1 to Quinvaxem 3 declined from 4.2% in 2013 to 0% in 2015, and the dropout rate from Bacillus Calmette-Guérin (BCG) to measles 1 fell from 12.8% in 2013 to 0% in 2015. Full immunization coverage of children under one year old increased significantly from 75.4% in 2013 to 81.7% in 2014 to 99.2% in 2015. Also, survey results indicated that 93.3% of interviewees were willing to pay for SMS reminders for immunization. CONCLUSIONS: A digital immunization registry that includes SMS reminders can improve immunization coverage and timeliness of vaccination, thereby strengthening the quality and effectiveness of immunization programs. Integrating this system into the national health information system and leveraging it for other health programs, such as maternal and child health and nutrition as well as infectious disease control, can bring more benefits to the health care system in Vietnam.

A reduction in chronic hepatitis B virus infection prevalence among children in Vietnam demonstrates the importance of vaccination.

BACKGROUND: Vietnam has high endemic hepatitis B virus infection with >8% of adults estimated to have chronic infection. Hepatitis B vaccine was first introduced in the national childhood immunization program in 1997 in high-risk areas, expanded nationwide in 2002, and included birth dose vaccination in 2003. This survey aimed to assess the impact of Vietnam's vaccination programme by estimating the prevalence of hepatitis B surface antigen (HBsAg) among children born during 2000-2008. METHODS: This nationally representative cross-sectional survey sampled children based on a stratified three-stage cluster design. Demographic and vaccination data were collected along with a whole blood specimen that was collected and interpreted in the field with a point-of-care HBsAg test. RESULTS: A total of 6,949 children were included in the survey analyses. The overall HBsAg prevalence among surveyed children was 2.70% (95% confidence interval (CI): 2.20-3.30). However, HBsAg prevalence was significantly higher among children born in 2000-2003 (3.64%) compared to children born 2007-2008 (1.64%) (prevalence ratio (PR: 2.22, CI 1.55-3.18)). Among all children included in the survey, unadjusted HBsAg prevalence among children with ≥3 doses of hepatitis B vaccine including a birth dose (1.75%) was significantly lower than among children with ≥3 doses of hepatitis B vaccine but lacked a birth dose (2.98%) (PR: 1.71, CI: 1.00-2.91) and significantly lower than among unvaccinated children (3.47%) (PR: 1.99, CI: 1.15-3.45). Infants receiving hepatitis B vaccine >7 days after birth had significantly higher HBsAg prevalence (3.20%) than those vaccinated 0-1 day after birth (1.52%) (PR: 2.09, CI: 1.27-3.46). CONCLUSION: Childhood chronic HBV infection prevalence has been markedly reduced in Vietnam due to vaccination. Further strengthening of timely birth dose vaccination will be important for reducing chronic HBV infection prevalence of under 5 children to <1%, a national and Western Pacific regional hepatitis B control goal.