Conversational AI and Vaccine Communication: Systematic Review of the Evidence.

BACKGROUND: Since the mid-2010s, use of conversational artificial intelligence (AI; chatbots) in health care has expanded significantly, especially in the context of increased burdens on health systems and restrictions on in-person consultations with health care providers during the COVID-19 pandemic. One emerging use for conversational AI is to capture evolving questions and communicate information about vaccines and vaccination. OBJECTIVE: The objective of this systematic review was to examine documented uses and evidence on the effectiveness of conversational AI for vaccine communication. METHODS: This systematic review was conducted following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. PubMed, Web of Science, PsycINFO, MEDLINE, Scopus, CINAHL Complete, Cochrane Library, Embase, Epistemonikos, Global Health, Global Index Medicus, Academic Search Complete, and the University of London library database were searched for papers on the use of conversational AI for vaccine communication. The inclusion criteria were studies that included (1) documented instances of conversational AI being used for the purpose of vaccine communication and (2) evaluation data on the impact and effectiveness of the intervention. RESULTS: After duplicates were removed, the review identified 496 unique records, which were then screened by title and abstract, of which 38 were identified for full-text review. Seven fit the inclusion criteria and were assessed and summarized in the findings of this review. Overall, vaccine chatbots deployed to date have been relatively simple in their design and have mainly been used to provide factual information to users in response to their questions about vaccines. Additionally, chatbots have been used for vaccination scheduling, appointment reminders, debunking misinformation, and, in some cases, for vaccine counseling and persuasion. Available evidence suggests that chatbots can have a positive effect on vaccine attitudes; however, studies were typically exploratory in nature, and some lacked a control group or had very small sample sizes. CONCLUSIONS: The review found evidence of potential benefits from conversational AI for vaccine communication. Factors that may contribute to the effectiveness of vaccine chatbots include their ability to provide credible and personalized information in real time, the familiarity and accessibility of the chatbot platform, and the extent to which interactions with the chatbot feel "natural" to users. However, evaluations have focused on the short-term, direct effects of chatbots on their users. The potential longer-term and societal impacts of conversational AI have yet to be analyzed. In addition, existing studies do not adequately address how ethics apply in the field of conversational AI around vaccines. In a context where further digitalization of vaccine communication can be anticipated, additional high-quality research will be required across all these areas.

Effectiveness of chatbots on COVID vaccine confidence and acceptance in Thailand, Hong Kong, and Singapore.

Chatbots have become an increasingly popular tool in the field of health services and communications. Despite chatbots' significance amid the COVID-19 pandemic, few studies have performed a rigorous evaluation of the effectiveness of chatbots in improving vaccine confidence and acceptance. In Thailand, Hong Kong, and Singapore, from February 11th to June 30th, 2022, we conducted multisite randomised controlled trials (RCT) on 2,045 adult guardians of children and seniors who were unvaccinated or had delayed vaccinations. After a week of using COVID-19 vaccine chatbots, the differences in vaccine confidence and acceptance were compared between the intervention and control groups. Compared to non-users, fewer chatbot users reported decreased confidence in vaccine effectiveness in the Thailand child group [Intervention: 4.3 % vs. Control: 17%, P = 0.023]. However, more chatbot users reported decreased vaccine acceptance [26% vs. 12%, P = 0.028] in Hong Kong child group and decreased vaccine confidence in safety [29% vs. 10%, P = 0.041] in Singapore child group. There was no statistically significant change in vaccine confidence or acceptance in the Hong Kong senior group. Employing the RE-AIM framework, process evaluation indicated strong acceptance and implementation support for vaccine chatbots from stakeholders, with high levels of sustainability and scalability. This multisite, parallel RCT study on vaccine chatbots found mixed success in improving vaccine confidence and acceptance among unvaccinated Asian subpopulations. Further studies that link chatbot usage and real-world vaccine uptake are needed to augment evidence for employing vaccine chatbots to advance vaccine confidence and acceptance.

Effect of a mobile app chatbot and an interactive small-group webinar on COVID-19 vaccine intention and confidence in Japan: a randomised controlled trial.

INTRODUCTION: We investigated the effect of social media-based interventions on COVID-19 vaccine intention (VI) and confidence in Japan. METHODS: We conducted a three-arm randomised controlled trial between 5 November 2021 and 9 January 2022 during a low incidence (<1000/day) of COVID-19 in Japan in the midst of the second and the third waves. Japanese citizens aged ≥20 who had not received any COVID-19 vaccine and did not intend to be vaccinated were randomly assigned to one of the following three groups: (1) a control group, (2) a group using a mobile app chatbot providing information on COVID-19 vaccines and (3) a group using interactive webinars with health professionals. VI and predefined Vaccine Confidence Index (VCI) measuring confidence in the importance, safety and effectiveness were compared before and after the interventions under intention-to-treat principle. Logistic regression models were used to investigate the effect of each intervention on postintervention VI and changes of VCI compared with control. RESULTS: Among 386 participants in each group, 359 (93.0%), 231 (59.8%) and 207 (53.6%) completed the postsurvey for the control, chatbot and webinar groups, respectively. The average duration between the intervention and the postsurvey was 32 days in chatbot group and 27 days in webinar group. VI increased from 0% to 18.5% (95% CI 14.5%, 22.5%) in control group, 15.4% (95% CI 10.8%, 20.1%) in chatbot group and 19.7% (95% CI 14.5%, 24.9%) in webinar group without significant difference (OR for improvement=0.8 (95% CI 0.5, 1.3), p=0.33 between chatbot and control, OR=1.1 (95% CI 0.7, 1.6), p=0.73 between webinar and control). VCI change tended to be larger in chatbot group compared with control group without significant difference (3.3% vs -2.5% in importance, OR for improvement=1.3 (95% CI 0.9, 2.0), p=0.18; 2.5% vs 1.9% in safety, OR=1.1 (95% CI 0.7, 1.9), p=0.62; -2.4% vs -7.6% in effectiveness, OR=1.4 (95% CI 0.9, 2.1), p=0.09). Improvement in VCI was larger in webinar group compared with control group for importance (7.8% vs -2.5%, OR=1.8 (95% CI 1.2, 2.8), p<0.01), effectiveness (6.4% vs -7.6%, OR=2.2 (95% CI 1.4, 3.4), p<0.01) and safety (6.0% vs 1.9%, OR=1.6 (95% CI 1.0, 2.6), p=0.08). CONCLUSION: This study demonstrated that neither the chatbot nor the webinar changed VI importantly compared with control. Interactive webinars could be an effective tool to change vaccine confidence. Further study is needed to identify risk factors associated with decreased vaccine confidence and investigate what intervention can increase VI and vaccine confidence for COVID-19 vaccines. TRIAL REGISTRATION NUMBER: UMIN000045747.