Real-Time Electronic Patient Portal Use Among Emergency Department Patients.

Importance: Patients with inequitable access to patient portals frequently present to emergency departments (EDs) for care. Little is known about portal use patterns among ED patients. Objectives: To describe real-time patient portal usage trends among ED patients and compare demographic and clinical characteristics between portal users and nonusers. Design, Setting, and Participants: In this cross-sectional study of 12 teaching and 24 academic-affiliated EDs from 8 health systems in California, Connecticut, Massachusetts, Ohio, Tennessee, Texas, and Washington, patient portal access and usage data were evaluated for all ED patients 18 years or older between April 5, 2021, and April 4, 2022. Exposure: Use of the patient portal during ED visit. Main Outcomes and Measures: The primary outcomes were the weekly proportions of ED patients who logged into the portal, viewed test results, and viewed clinical notes in real time. Pooled random-effects models were used to evaluate temporal trends and demographic and clinical characteristics associated with real-time portal use. Results: The study included 1 280 924 unique patient encounters (53.5% female; 0.6% American Indian or Alaska Native, 3.7% Asian, 18.0% Black, 10.7% Hispanic, 0.4% Native Hawaiian or Pacific Islander, 66.5% White, 10.0% other race, and 4.0% with missing race or ethnicity; 91.2% English-speaking patients; mean [SD] age, 51.9 [19.2] years). During the study, 17.4% of patients logged into the portal while in the ED, whereas 14.1% viewed test results and 2.5% viewed clinical notes. The odds of accessing the portal (odds ratio [OR], 1.36; 95% CI, 1.19-1.56), viewing test results (OR, 1.63; 95% CI, 1.30-2.04), and viewing clinical notes (OR, 1.60; 95% CI, 1.19-2.15) were higher at the end of the study vs the beginning. Patients with active portal accounts at ED arrival had a higher odds of logging into the portal (OR, 17.73; 95% CI, 9.37-33.56), viewing test results (OR, 18.50; 95% CI, 9.62-35.57), and viewing clinical notes (OR, 18.40; 95% CI, 10.31-32.86). Patients who were male, Black, or without commercial insurance had lower odds of logging into the portal, viewing results, and viewing clinical notes. Conclusions and Relevance: These findings suggest that real-time patient portal use during ED encounters has increased over time, but disparities exist in portal access that mirror trends in portal usage more generally. Given emergency medicine's role in caring for medically underserved patients, there are opportunities for EDs to enroll and train patients in using patient portals to promote engagement during and after their visits.

Risk factors for healthcare-associated candidemia in adults hospitalized with SARS-CoV-2 infection.

Analyzing data from a national deidentified electronic health record-based data set using a matched case-control study design, we found that antibiotic use and severity of illness were independent risk factors for healthcare-associated candidemia in adult patients hospitalized with SARS-CoV-2 infection. Interleukin-6 inhibitor and corticosteroid use were not independent risk factors.

Safe and Equitable Pediatric Clinical Use of AI.

This Viewpoint provides recommendations and stakeholder actions to support safe and equitable use of artificial intelligence (AI) in pediatric clinical settings.

Early identification of patients at risk for iron-deficiency anemia using deep learning techniques.

OBJECTIVES: Iron-deficiency anemia (IDA) is a common health problem worldwide, and up to 10% of adult patients with incidental IDA may have gastrointestinal cancer. A diagnosis of IDA can be established through a combination of laboratory tests, but it is often underrecognized until a patient becomes symptomatic. Based on advances in machine learning, we hypothesized that we could reduce the time to diagnosis by developing an IDA prediction model. Our goal was to develop 3 neural networks by using retrospective longitudinal outpatient laboratory data to predict the risk of IDA 3 to 6 months before traditional diagnosis. METHODS: We analyzed retrospective outpatient electronic health record data between 2009 and 2020 from an academic medical center in northern Texas. We included laboratory features from 30,603 patients to develop 3 types of neural networks: artificial neural networks, long short-term memory cells, and gated recurrent units. The classifiers were trained using the Adam Optimizer across 200 random training-validation splits. We calculated accuracy, area under the receiving operating characteristic curve, sensitivity, and specificity in the testing split. RESULTS: Although all models demonstrated comparable performance, the gated recurrent unit model outperformed the other 2, achieving an accuracy of 0.83, an area under the receiving operating characteristic curve of 0.89, a sensitivity of 0.75, and a specificity of 0.85 across 200 epochs. CONCLUSIONS: Our results showcase the feasibility of employing deep learning techniques for early prediction of IDA in the outpatient setting based on sequences of laboratory data, offering a substantial lead time for clinical intervention.

Electronic Health Record Usability When Caring for Children With Medical Complexity.

OBJECTIVES: To evaluate usability of and clinician satisfaction with the electronic health record (EHR) in the context of caring for children with medical complexity (CMC) at a large academic pediatric hospital and to identify key areas for targeted improvements. METHODS: Cross-sectional study of pediatric faculty and advanced practice providers across several pediatric specialties using an online Research Electronic Data Capture survey. EHR usability was measured with 6 validated questions from the National Usability-Focused Health Information System Scale, and satisfaction with common EHR functionalities was measured with 6 original Likert-scale questions and 3 free-text questions. Results were analyzed with bivariate testing. RESULTS: More than half of providers completed the survey (n = 81, response rate 56%). When asked 6 positively-phrased, validated usability questions, a majority of respondents either agreed or were neutral. Respondents were most dissatisfied with information quality of the summary view and most satisfied with physician communication. Older respondents were less satisfied with EHR usability (P < .01). Focusing on functionalities needed for CMC, the majority of respondents were dissatisfied with the chart review process. More respondents were dissatisfied with order entry (P = .002) and documentation (P = .017) when caring for CMC compared with caring for other patients. The most cited challenges were locating recent patient data, performing an accurate medication reconciliation, and lack of specialized documentation templates. CONCLUSIONS: Clinicians are less satisfied with common EHR functionalities when caring for CMC compared with caring for all other patients. Targeted interventions to improve usability and common EHR functionalities for CMC are necessary to improve the user experience.

Structure and Funding of Clinical Informatics Fellowships: A National Survey of Program Directors.

BACKGROUND: In 2011, the American Board of Medical Specialties established clinical informatics (CI) as a subspecialty in medicine, jointly administered by the American Board of Pathology and the American Board of Preventive Medicine. Subsequently, many institutions created CI fellowship training programs to meet the growing need for informaticists. Although many programs share similar features, there is considerable variation in program funding and administrative structures. OBJECTIVES: The aim of our study was to characterize CI fellowship program features, including governance structures, funding sources, and expenses. METHODS: We created a cross-sectional online REDCap survey with 44 items requesting information on program administration, fellows, administrative support, funding sources, and expenses. We surveyed program directors of programs accredited by the Accreditation Council for Graduate Medical Education between 2014 and 2021. RESULTS: We invited 54 program directors, of which 41 (76%) completed the survey. The average administrative support received was $27,732/year. Most programs (85.4%) were accredited to have two or more fellows per year. Programs were administratively housed under six departments: Internal Medicine (17; 41.5%), Pediatrics (7; 17.1%), Pathology (6; 14.6%), Family Medicine (6; 14.6%), Emergency Medicine (4; 9.8%), and Anesthesiology (1; 2.4%). Funding sources for CI fellowship program directors included: hospital or health systems (28.3%), clinical departments (28.3%), graduate medical education office (13.2%), biomedical informatics department (9.4%), hospital information technology (9.4%), research and grants (7.5%), and other sources (3.8%) that included philanthropy and external entities. CONCLUSION: CI fellowships have been established in leading academic and community health care systems across the country. Due to their unique training requirements, these programs require significant resources for education, administration, and recruitment. There continues to be considerable heterogeneity in funding models between programs. Our survey findings reinforce the need for reformed federal funding models for informatics practice and training.

Direct Secure Messaging in Practice: Addressing Workflow Challenges.

Direct Secure Messaging (DSM) is a sender-initiated communication technology for exchanging patient-specific information among clinicians and disparate healthcare organizations. As DSM adoption increases it becomes more difficult for clinicians and staff to manage the volume and variety of external data received. This can lead to information hazards that can produce cognitive overload and decrease the ability of clinicians to process patient data when reviewing multiple sources. While DSM is one of many options for electronically exchanging health information, we have found that poor user awareness of DSM features and variable EHR capabilities for sending, receiving, and managing messages and their contents demonstrate that additional work is needed to achieve DSM's potential as a low-barrier, ubiquitous option for clinical interoperability. This paper reviews these problems from end-user perspective and offers best-practices for both senders and recipients of DSM.

Next-generation study databases require FAIR, EHR-integrated, and scalable Electronic Data Capture for medical documentation and decision support.

Structured patient data play a key role in all types of clinical research. They are often collected in study databases for research purposes. In order to describe characteristics of a next-generation study database and assess the feasibility of its implementation a proof-of-concept study in a German university hospital was performed. Key characteristics identified include FAIR access to electronic case report forms (eCRF), regulatory compliant Electronic Data Capture (EDC), an EDC with electronic health record (EHR) integration, scalable EDC for medical documentation, patient generated data, and clinical decision support. In a local case study, we then successfully implemented a next-generation study database for 19 EDC systems (n = 2217 patients) that linked to i.s.h.med (Oracle Cerner) with the local EDC system called OpenEDC. Desiderata of next-generation study databases for patient data were identified from ongoing local clinical study projects in 11 clinical departments at Heidelberg University Hospital, Germany, a major tertiary referral hospital. We compiled and analyzed feature and functionality requests submitted to the OpenEDC team between May 2021 and July 2023. Next-generation study databases are technically and clinically feasible. Further research is needed to evaluate if our approach is feasible in a multi-center setting as well.

Vanderbilt Electronic Health Record Voice Assistant Supports Clinicians.

BACKGROUND: Electronic health records (EHRs) present navigation challenges due to time-consuming searches across segmented data. Voice assistants can improve clinical workflows by allowing natural language queries and contextually aware navigation of the EHR. OBJECTIVES: To develop a voice-mediated EHR assistant and interview providers to inform its future refinement. METHODS: The Vanderbilt EHR Voice Assistant (VEVA) was developed as a responsive web application and designed to accept voice inputs and execute the appropriate EHR commands. Fourteen providers from Vanderbilt Medical Center were recruited to participate in interactions with VEVA and to share their experience with the technology. The purpose was to evaluate VEVA's overall usability, gather qualitative feedback, and detail suggestions for enhancing its performance. RESULTS: VEVA's mean system usability scale score was 81 based on the 14 providers' evaluations, which was above the standard 50th percentile score of 68. For all five summaries evaluated (overview summary, A1C results, blood pressure, weight, and health maintenance), most providers offered a positive review of VEVA. Several providers suggested modifications to make the technology more useful in their practice, ranging from summarizing current medications to changing VEVA's speech rate. Eight of the providers (64%) reported they would be willing to use VEVA in its current form. CONCLUSION: Our EHR voice assistant technology was deemed usable by most providers. With further improvements, voice assistant tools such as VEVA have the potential to improve workflows and serve as a useful adjunct tool in health care.

Creation and Evaluation of a Clinical Informatics Match: Initial Findings.

BACKGROUND: Clinical Informatics (CI) fellowship programs utilize the Electronic Residency Application Service (ERAS) to gather applications but until recently used an American Medical Informatics Association (AMIA) member-developed, simultaneous offer-acceptance process to match fellowship applicants to programs. In 2021, program directors collaborated with the AMIA to develop a new match to improve the process. OBJECTIVE: Describe the results of the first 2 years of the match and address opportunities for improvement. METHODS: We obtained applicant data for fellowship applicants in 2021 and 2022 from the ERAS and match data for the same years from the AMIA. We analyzed our data using descriptive statistics. RESULTS: There were 159 unique applicants over the 2-year period. Applicants submitted 2,178 applications with a median of 10 per applicant (interquartile range [IQR] 3-20). One hundred and four applicants (65.4%) participated in the match and ranked a median of seven programs (2-12). Forty-two programs in 2021 and 47 programs in 2022 offered a combined total 153 positions in the match. Participating programs ranked a median of eight applicants per year (IQR 5-11). Of participating applicants, 95 (91.3%) successfully matched and of those 66 (69.5%) received their top choice. Thirty-two programs (76.2%) matched at least one candidate in 2021 and 33 programs (70.2%) matched at least one candidate in 2022. In both years, 24 programs filled all available slots (57.1% in 2021 and 51.1% in 2022). CONCLUSION: Applicants were extremely successful in the new match, which successfully addressed most of the challenges of the simultaneous offer-acceptance process identified by program directors. However, applicant attrition resulted in a quarter of programs going unmatched. Although many programs still filled slots outside the match, fellowship slots may remain unfilled while the CI practice pathway remains open.

Advances in Clinical Decision Support Systems: Contributions from the 2022 Literature.

OBJECTIVE: To summarize significant research contributions published in 2022 in the field of clinical decision support (CDS) systems and select the best papers for the Decision Support section of the International Medical Informatics Association (IMIA) Yearbook 2023. METHODS: A renewed search query for identifying CDS scholarship was developed using Medical Subject Headings (MeSH) terms and related keywords. The query was executed in PubMed in January 2023. The search results were reviewed in three stages by two reviewers: title-based triaging, followed by abstract screening, and then full text review. The resulting articles were sent for external review to identity best paper candidates. RESULTS: A total of 1,939 articles related to CDS were retrieved. Of these, 11 articles were selected as candidates for best papers. The general themes of the final three best papers are (1) reducing documentation burden through in-line guidance for clinical notes, (2) clinician engagement for continuous improvement of CDS, and (3) mitigating healthcare-related carbon emissions using scalable and accessible CDS, respectively. CONCLUSION: The field of clinical decision support remains highly active and dynamic, with innovative contributions to a range of clinical domains from primary to acute care. Interoperability issues, documentation burden, clinician acceptance, and the need for effective integration into existing healthcare workflows are among the prominent challenges and areas of interest faced by CDS implementation efforts.

Twitter discussions on breastfeeding during the COVID-19 pandemic.

BACKGROUND: Breastfeeding is a critical health intervention in infants. Recent literature reported that the COVID-19 pandemic resulted in significant mental health issues in pregnant and breastfeeding women due to social isolation and lack of direct professional support. These maternal mental health issues affected infant nutrition and decreased breastfeeding rates during COVID-19. Twitter, a popular social media platform, can provide insight into public perceptions and sentiment about various health-related topics. With evidence of significant mental health issues among women during the COVID-19 pandemic, the perception of infant nutrition, specifically breastfeeding, remains unknown. METHODS: We aimed to understand public perceptions and sentiment regarding breastfeeding during the COVID-19 pandemic through Twitter analysis using natural language processing techniques. We collected and analyzed tweets related to breastfeeding and COVID-19 during the pandemic from January 2020 to May 2022. We used Python software (v3.9.0) for all data processing and analyses. We performed sentiment and emotion analysis of the tweets using natural language processing libraries and topic modeling using an unsupervised machine-learning algorithm. RESULTS: We analyzed 40,628 tweets related to breastfeeding and COVID-19 generated by 28,216 users. Emotion analysis revealed predominantly "Positive emotions" regarding breastfeeding, comprising 72% of tweets. The overall tweet sentiment was positive, with a mean weekly sentiment of 0.25 throughout, and was affected by external events. Topic modeling revealed six significant themes related to breastfeeding and COVID-19. Passive immunity through breastfeeding after maternal vaccination had the highest mean positive sentiment score of 0.32. CONCLUSIONS: Our study provides insight into public perceptions and sentiment regarding breastfeeding during the COVID-19 pandemic. Contrary to other topics we explored in the context of COVID (e.g., ivermectin, disinformation), we found that breastfeeding had an overall positive sentiment during the pandemic despite the documented rise in mental health challenges in pregnant and breastfeeding mothers. The wide range of topics on Twitter related to breastfeeding provides an opportunity for active engagement by the medical community and timely dissemination of advice, support, and guidance. Future studies should leverage social media analysis to gain real-time insight into public health topics of importance in child health and apply targeted interventions.

Assessing Racial and Ethnic Bias in Text Generation for Healthcare-Related Tasks by ChatGPT1.

Large Language Models (LLM) are AI tools that can respond human-like to voice or free-text commands without training on specific tasks. However, concerns have been raised about their potential racial bias in healthcare tasks. In this study, ChatGPT was used to generate healthcare-related text for patients with HIV, analyzing data from 100 deidentified electronic health record encounters. Each patient's data were fed four times with all information remaining the same except for race/ethnicity (African American, Asian, Hispanic White, Non-Hispanic White). The text output was analyzed for sentiment, subjectivity, reading ease, and most used words by race/ethnicity and insurance type. Results showed that instructions for African American, Asian, Hispanic White, and Non-Hispanic White patients had an average polarity of 0.14, 0.14, 0.15, and 0.14, respectively, with an average subjectivity of 0.46 for all races/ethnicities. The differences in polarity and subjectivity across races/ethnicities were not statistically significant. However, there was a statistically significant difference in word frequency across races/ethnicities and a statistically significant difference in subjectivity across insurance types with commercial insurance eliciting the most subjective responses and Medicare and other payer types the lowest. The study suggests that ChatGPT is relatively invariant to race/ethnicity and insurance type in terms of linguistic and readability measures. Further studies are needed to validate these results and assess their implications.

Is there an association between peri-diagnostic vaccination and clinical outcomes in COVID-19 patients?

Background: Peri-diagnostic vaccination contemporaneous with SARS-CoV-2 infection might boost antiviral immunity and improve patient outcomes. We investigated, among previously unvaccinated patients, whether vaccination (with the Pfizer, Moderna, or J&J vaccines) during the week before or after a positive COVID-19 test was associated with altered 30-day patient outcomes. Methods: Using a deidentified longitudinal EHR repository, we selected all previously unvaccinated adults who initially tested positive for SARS-CoV-2 between December 11, 2020 (the date of vaccine emergency use approval) and December 19, 2021. We assessed whether vaccination between days -7 and +7 of a positive test affected outcomes. The primary measure was progression to a more severe disease outcome within 30 days of diagnosis using the following hierarchy: hospitalization, intensive care, or death. Results: Among 60,031 hospitalized patients, 543 (0.91%) were initially vaccinated at the time of diagnosis and 59,488 (99.09%) remained unvaccinated during the period of interest. Among 316,337 nonhospitalized patients, 2,844 (0.90%) were initially vaccinated and 313,493 (99.1%) remained unvaccinated. In both analyses, individuals receiving vaccines were older, more often located in the northeast, more commonly insured by Medicare, and more burdened by comorbidities. Among previously unvaccinated patients, there was no association between receiving an initial vaccine dose between days -7 and +7 of diagnosis and progression to more severe disease within 30 days compared to patients who did not receive vaccines. Conclusions: Immunization during acute SARS-CoV-2 infection does not appear associated with clinical progression during the acute infectious period.

Risk Factors for SARS-CoV-2 Infection and Severe Outcomes Among People With Human Immunodeficiency Virus: Cohort Study.

Background: Studies on COVID-19 in people with HIV (PWH) have had limitations. Further investigations on risk factors and outcomes of SARS-CoV-2 infection among PWH are needed. Methods: This retrospective cohort study leveraged the national OPTUM COVID-19 data set to investigate factors associated with SARS-CoV-2 positivity among PWH and risk factors for severe outcomes, including hospitalization, intensive care unit stays, and death. A subset analysis was conducted to examine HIV-specific variables. Multiple variable logistic regression was used to adjust for covariates. Results: Of 43 173 PWH included in this study, 6472 had a positive SARS-CoV-2 result based on a polymerase chain reaction test or antigen test. For PWH with SARS-CoV-2 positivity, higher odds were found for those who were younger (18-49 years), Hispanic White, African American, from the US South, uninsured, and a noncurrent smoker and had a higher body mass index and higher Charlson Comorbidity Index. For PWH with severe outcomes, higher odds were identified for those who were SARS-CoV-2 positive, older, from the US South, receiving Medicaid/Medicare or uninsured, a current smoker, and underweight and had a higher Charlson Comorbidity Index. In a subset analysis including PWH with HIV care variables (n = 5098), those with unsuppressed HIV viral load, a low CD4 count, and no antiretroviral therapy had higher odds of severe outcomes. Conclusions: This large US study found significant ethnic, racial, and geographic differences in SARS-CoV-2 infection among PWH. Chronic comorbidities, older age, lower body mass index, and smoking were associated with severe outcomes among PWH during the COVID-19 pandemic. SARS-CoV-2 infection was associated with severe outcomes, but once we adjusted for HIV care variables, SARS-CoV-2 was no longer significant; however, low CD4 count, high viral load, and lack of antiretroviral therapy had higher odds of severe outcomes.

ChatGPT and the clinical informatics board examination: the end of unproctored maintenance of certification?

We aimed to assess ChatGPT's performance on the Clinical Informatics Board Examination and to discuss the implications of large language models (LLMs) for board certification and maintenance. We tested ChatGPT using 260 multiple-choice questions from Mankowitz's Clinical Informatics Board Review book, omitting 6 image-dependent questions. ChatGPT answered 190 (74%) of 254 eligible questions correctly. While performance varied across the Clinical Informatics Core Content Areas, differences were not statistically significant. ChatGPT's performance raises concerns about the potential misuse in medical certification and the validity of knowledge assessment exams. Since ChatGPT is able to answer multiple-choice questions accurately, permitting candidates to use artificial intelligence (AI) systems for exams will compromise the credibility and validity of at-home assessments and undermine public trust. The advent of AI and LLMs threatens to upend existing processes of board certification and maintenance and necessitates new approaches to the evaluation of proficiency in medical education.

The lived experience of people with disabilities during the COVID-19 pandemic on Twitter: Content analysis.

Objective: People with disabilities (PWDs) are at greater risk of COVID-19 infection, complications, and death, and experience more difficulty accessing care. We analyzed Twitter tweets to identify important topics and investigate health policies' effects on PWDs. Methods: Twitter's application programming interface was used to access its public COVID-19 stream. English-language tweets from January 2020 to January 2022 containing a combination of keywords related to COVID-19, disability, discrimination, and inequity were collected and refined to exclude duplicates, replies, and retweets. The remaining tweets were analyzed for user demographics, content, and long-term availability. Results: The collection yielded 94,814 tweets from 43,296 accounts. During the observation period, 1068 (2.5%) accounts were suspended and 1088 (2.5%) accounts were deleted. Account suspension and deletion among verified users tweeting about COVID-19 and disability were 0.13% and 0.3%, respectively. Emotions were similar among active, suspended, and deleted users, with general negative and positive emotions most common followed by sadness, trust, anticipation, and anger. The overall average sentiment for the tweets was negative. Ten of the 12 topics identified (96.8%) related to pandemic effects on PWDs; "politics that rejects and leaves the disabled, elderly, and children behind" (48.3%) and "efforts to support PWDs in the COVID crisis" (31.8%) were most common. The sample of tweets by organizations (43.9%) was higher for this topic than for other COVID-19-related topics the authors have investigated. Conclusions: The primary discussion addressed how pandemic politics and policies disadvantage PWDs, older adults, and children, and secondarily expressed support for these populations. The increased level of Twitter use by organizations suggests a higher level of organization and advocacy within the disability community than in other groups. Twitter may facilitate recognition of increased harm to or discrimination against specific populations such as people living with disability during national health events.

Public perception of COVID-19 vaccines through analysis of Twitter content and users.

BACKGROUND: With the global continuation of the COVID-19 pandemic, the large-scale administration of a SARS-CoV-2 vaccine is crucial to achieve herd immunity and curtail further spread of the virus, but success is contingent on public understanding and vaccine uptake. We aim to understand public perception about vaccines for COVID-19 through the wide-scale, organic discussion on Twitter. METHODS: This cross-sectional observational study included Twitter posts matching the search criteria (('covid*' OR 'coronavirus') AND 'vaccine') posted during vaccine development from February 1st through December 11th, 2020. These COVID-19 vaccine related posts were analyzed with topic modeling, sentiment and emotion analysis, and demographic inference of users to provide insight into the evolution of public attitudes throughout the study period. FINDINGS: We evaluated 2,287,344 English tweets from 948,666 user accounts. Individuals represented 87.9 % (n = 834,224) of user accounts. Of individuals, men (n = 560,824) outnumbered women (n = 273,400) by 2:1 and 39.5 % (n = 329,776) of individuals were ≥40 years old. Daily mean sentiment fluctuated congruent with news events, but overall trended positively. Trust, anticipation, and fear were the three most predominant emotions; while fear was the most predominant emotion early in the study period, trust outpaced fear from April 2020 onward. Fear was more prevalent in tweets by individuals (26.3 % vs. organizations 19.4 %; p < 0.001), specifically among women (28.4 % vs. males 25.4 %; p < 0.001). Multiple topics had a monthly trend towards more positive sentiment. Tweets comparing COVID-19 to the influenza vaccine had strongly negative early sentiment but improved over time. INTERPRETATION: This study successfully explores sentiment, emotion, topics, and user demographics to elucidate important trends in public perception about COVID-19 vaccines. While public perception trended positively over the study period, some trends, especially within certain topic and demographic clusters, are concerning for COVID-19 vaccine hesitancy. These insights can provide targets for educational interventions and opportunity for continued real-time monitoring.