Usability and Acceptability of Electronic Immunization Registry Data Entry Workflows From the Health Care Worker Perspective in Siaya, Kenya (Part 3): Pre-Post Study.

BACKGROUND: Digital health tools such as electronic immunization registries (EIRs) have the potential to improve patient care and alleviate the challenges that arise from the use of paper-based clinic records for reporting. To address some of these challenges, the Kenya Ministry of Health and the International Training and Education Center for Health Kenya implemented an EIR system in 161 immunizing clinics in Siaya County between 2018 and 2019. The successful implementation of digital health tools depends on many factors, one of which is alignment between the technology and the context in which it is used. One important aspect of that implementation context is the perceptions of the health care workers (HCWs) using the EIR. OBJECTIVE: This study aimed to evaluate HCWs' perceptions of the usability and acceptability of multiple clinic workflows using the new EIR. METHODS: We performed a mixed methods pre-post study using semistructured interviews with HCWs at 6 facilities in Siaya County, Kenya. We interviewed HCWs at each facility 4 times: at baseline and once after the implementation of 3 different workflow modifications (n=24 interviews). The baseline state was dual data entry with paper records and the EIR. We then implemented 3 workflow modifications for 1 full day each: fully paperless data entry, preparation of an appointment diary before patient visits for the day, and a combination of the 2 workflows. We compared ratings and themes across interviews after each of the 4 workflows to understand the changes in the usability and acceptability of the EIR. RESULTS: HCWs considered the EIR clinic workflows to be usable and acceptable. Of the modified workflows, HCWs perceived the fully paperless workflow most favorably. In all workflows, HCWs' perceived benefits included ease of clinical decision-making using the EIR, reduced mental burden of data entry when using the EIR, and ease of identification of errors. Perceived barriers to the workflow included contextual challenges such as staffing shortages and lack of network connectivity, EIR platform challenges such as errors in saving records and missing fields, and workflow challenges such as the dual data entry burden of using paper and digital tools simultaneously. CONCLUSIONS: Fully paperless EIR implementation shows great promise from a workflow acceptability standpoint, contingent upon the presence of supporting contextual clinic factors and the resolution of system performance and design challenges. Rather than trying to identify a singular best workflow, future efforts should provide adequate flexibility for HCWs to implement the new system in their unique clinic context. Future EIR implementation stands to benefit from continued monitoring of EIR adoption acceptability during implementation, both for Siaya's program and for other efforts around the globe, as digital health interventions become more widely used.

Integration of a Digital Health Intervention Into Immunization Clinic Workflows in Kenya: Qualitative, Realist Evaluation of Technology Usability.

BACKGROUND: In an effort to increase vaccination coverage in low-resource settings, digital tools have been introduced to better track immunization records, improve data management practices, and provide improved access to vaccination coverage data for decision-making. Despite the potential of these electronic systems to improve the provision of health services, few digital health interventions have been institutionalized at scale in low- and middle-income countries. OBJECTIVE: In this paper, we aimed to describe how health care workers in Kenya had integrated an electronic immunization registry into their immunization clinic workflows and to use these findings to inform the development of a refined program theory on the registry's usability. METHODS: Informed by realist methodology, we developed a program theory to explain usability of the electronic immunization registry. We designed a qualitative study based on our theory to describe the barriers and facilitators influencing data entry and use. Qualitative data were collected through semistructured interviews with users and workflow observations of immunization clinic sessions. Our findings were summarized by context-mechanism-outcome relationships formed after analyzing our key themes across interviews and workflow observations. Using these relationships, we were able to identify common rules for future implementers. RESULTS: Across the 12 facilities included in our study, 19 health care workers were interviewed, and 58 workflow sessions were observed. The common rules developed from our qualitative findings are as follows: rule 1-ensure that the users complete training to build familiarity with the system, understand the value of the system and data, and know where to find support; rule 2-confirm that the system captures all data needed for users to provide routine health care services and is easy to navigate; rule 3-identify work-arounds for poor network, system performance, and too few staff or resources; and rule 4-make users aware of expected changes to their workflow, and how these changes might differ over time and by facility size or number of patients. Upon study completion, we revised the program theory to reflect the importance of the goals and workflows of electronic immunization registries aligning with reality. CONCLUSIONS: We created a deeper understanding of the underlying mechanisms for usability of the registry. We found that the electronic immunization registry had high acceptability among users; however, there were numerous barriers to using the system, even under ideal conditions, causing a misalignment between the system and the reality of the users' workflows and their environment. Human-centered design and human-factors methods can assist during pilot stages to better align systems with users' needs and again after scale-up to ensure that interventions are suitable for all user settings.

Time Utilization Among Immunization Clinics Using an Electronic Immunization Registry (Part 2): Time and Motion Study of Modified User Workflows.

BACKGROUND: Digital health interventions have the potential to improve the provision of health care services through digitized data collection and management. Low- and middle-income countries are beginning to introduce electronic immunization registries (EIRs) into their routine immunization services to better capture and store childhood vaccination information. Especially in Africa, where 25% of children remain unimmunized or underimmunized, technologies that can help identify children due for a vaccination are particularly important for improving vaccination coverage. However, an improved understanding of the effectiveness of these systems is needed to develop and deploy sustainable EIRs in low- and middle-income countries. OBJECTIVE: We conducted an interventional pretest-posttest design study that sought to improve time efficiency through workflow modifications in Kenyan immunization clinics. Our aim was to describe how activity times differed after introducing workflow modifications that could potentially reduce the time needed to perform routine data entry activities. Our intent was to demonstrate changes in efficiency when moving from the existing dual-data entry workflow to a future paperless workflow by health facility size and experience length of health care workers (HCWs). METHODS: We tested how 3 workflow modifications would affect time utilization among HCWs using the EIR at the point of care compared with baseline immunization clinic workflows. Our outcome of interest was the time taken to complete individual activities and a patient's total time in the clinic where we compared the time spent during the baseline workflow with that during the modified workflow. We used a standardized tool to observe and document the immunization clinic workflow. To estimate differences in time utilization, we used bivariate analyses and fit multivariate linear mixed-effects models. RESULTS: Our study found that for HCWs using an EIR, the introduction of modified workflows decreased the amount of time needed to provide services to children seen in the immunization clinic. With a baseline mean time of 10 minutes spent per child, this decreased by about 3 minutes when the preparation modification was introduced and almost 5 minutes for the paperless and combined modifications. Results pertaining to the EIR's performance and ability to connect to the internet were particularly insightful about potential causes of delays. CONCLUSIONS: We were able to conduct a concise clinical simulation exercise by introducing modified workflows and estimating their impact on time utilization in immunization clinics using an EIR. We found that the paperless workflow provided the largest time savings when delivering services, although this was threatened by poor EIR performance and internet connectivity. This study demonstrated that not only should digital health interventions be built and adapted for particular use cases but existing user workflows also need to adapt to new technology.