The 2-week systematic review (2weekSR) method was successfully blind-replicated by another team: a case study.

OBJECTIVE: To assess the replicability of a 2-week systematic review (index 2weekSR) created with the assistance of automation tools using the fidelity method. METHODS: A Preferred Reporting Items for Systematic reviews and Meta-Analyses compliant SR protocol was developed based on the published information of the index 2weekSR study. The replication team consisted of three reviewers. Two reviewers blocked off time during the replication. The total time to complete tasks and the meta-analysis results were compared with the index 2weekSR study. Review process fidelity scores (FSs) were calculated for review methods and outcomes. Barriers to completing the replication were identified. RESULTS: The review was completed over 63 person-hours (11 workdays/15 calendar days). A FS of 0.95 was achieved for the methods, with 3 (of 8) tasks only partially replicated, and an FS of 0.63 for the outcomes, with 6 (of 7) only partially replicated and one task was not replicated. Nonreplication was mainly caused by missing information in the index 2weekSR study that was not required in standard reporting guidelines. The replication arrived at the same conclusions as the original study. CONCLUSION: A 2weekSR study was replicated by a small team of three reviewers supported by automation tools. Including additional information when reporting SRs should improve their replicability.

Using automated methods to detect safety problems with health information technology: a scoping review.

OBJECTIVE: To summarize the research literature evaluating automated methods for early detection of safety problems with health information technology (HIT). MATERIALS AND METHODS: We searched bibliographic databases including MEDLINE, ACM Digital, Embase, CINAHL Complete, PsycINFO, and Web of Science from January 2010 to June 2021 for studies evaluating the performance of automated methods to detect HIT problems. HIT problems were reviewed using an existing classification for safety concerns. Automated methods were categorized into rule-based, statistical, and machine learning methods, and their performance in detecting HIT problems was assessed. The review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta Analyses extension for Scoping Reviews statement. RESULTS: Of the 45 studies identified, the majority (n = 27, 60%) focused on detecting use errors involving electronic health records and order entry systems. Machine learning (n = 22) and statistical modeling (n = 17) were the most common methods. Unsupervised learning was used to detect use errors in laboratory test results, prescriptions, and patient records while supervised learning was used to detect technical errors arising from hardware or software issues. Statistical modeling was used to detect use errors, unauthorized access, and clinical decision support system malfunctions while rule-based methods primarily focused on use errors. CONCLUSIONS: A wide variety of rule-based, statistical, and machine learning methods have been applied to automate the detection of safety problems with HIT. Many opportunities remain to systematically study their application and effectiveness in real-world settings.

Social interactions and quality of life of residents in aged care facilities: A multi-methods study.

BACKGROUND: The relationship between social contact and quality of life is well-established within the general population. However, limited data exist about the extent of social interactions in residential aged care facilities (RACFs) providing long-term accommodation and care. We aimed to record the frequency and duration of interpersonal interactions among residents in RACFs and identify the association between residents' interpersonal interactions and quality of life (QoL). MATERIALS AND METHODS: A multi-methods study, including time and motion observations and a QoL survey, was conducted between September 2019 to January 2020. Thirty-nine residents from six Australian RACFs were observed between 09:30-17:30 on weekdays. Observations included residents' actions, location of the action, and who the resident was with during the action. At the end of the observation period, residents completed a QoL survey. The proportion of time residents spent on different actions, in which location, and with whom were calculated, and correlations between these factors and QoL were analysed. RESULTS: A total of 312 hours of observations were conducted. Residents spent the greatest proportion of time in their own room (45.2%, 95%CI 40.7-49.8), alone (47.9%, 95%CI 43.0-52.7) and being inactive (25.6%, 95%CI 22.5-28.7). Residents were also largely engaged in interpersonal communication (20.2%, 95%CI 17.9-22.5) and self-initiated or scheduled events (20.5%, 95%CI 18.0-23.0). Residents' interpersonal communication was most likely to occur in the common area (29.3%, 95%CI 22.9-35.7), residents' own room (26.7%, 95%CI 21.0-32.4) or the dining room (24.6%, 95%CI 18.9-30.2), and was most likely with another resident (54.8%, 95%CI 45.7-64.2). Quality of life scores were low (median = 0.68, IQR = 0.54-0.76). Amount of time spent with other residents was positively correlated with QoL (r = 0.39, p = 0.02), whilst amount of time spent with facility staff was negatively correlated with QoL (r = -0.45, p = 0.008). DISCUSSION AND CONCLUSIONS: Our findings confirm an established association between social interactions and improved QoL. Opportunities and activities which encourage residents to engage throughout the day in common facility areas can support resident wellbeing.

The automation of relevant trial registration screening for systematic review updates: an evaluation study on a large dataset of ClinicalTrials.gov registrations.

BACKGROUND: Clinical trial registries can be used as sources of clinical evidence for systematic review synthesis and updating. Our aim was to evaluate methods for identifying clinical trial registrations that should be screened for inclusion in updates of published systematic reviews. METHODS: A set of 4644 clinical trial registrations (ClinicalTrials.gov) included in 1089 systematic reviews (PubMed) were used to evaluate two methods (document similarity and hierarchical clustering) and representations (L2-normalised TF-IDF, Latent Dirichlet Allocation, and Doc2Vec) for ranking 163,501 completed clinical trials by relevance. Clinical trial registrations were ranked for each systematic review using seeding clinical trials, simulating how new relevant clinical trials could be automatically identified for an update. Performance was measured by the number of clinical trials that need to be screened to identify all relevant clinical trials. RESULTS: Using the document similarity method with TF-IDF feature representation and Euclidean distance metric, all relevant clinical trials for half of the systematic reviews were identified after screening 99 trials (IQR 19 to 491). The best-performing hierarchical clustering was using Ward agglomerative clustering (with TF-IDF representation and Euclidean distance) and needed to screen 501 clinical trials (IQR 43 to 4363) to achieve the same result. CONCLUSION: An evaluation using a large set of mined links between published systematic reviews and clinical trial registrations showed that document similarity outperformed hierarchical clustering for identifying relevant clinical trials to include in systematic review updates.

A rule-based approach for automatically extracting data from systematic reviews and their updates to model the risk of conclusion change.

Few data-driven approaches are available to estimate the risk of conclusion change in systematic review updates. We developed a rule-based approach to automatically extract information from reviews and updates to be used as features for modelling conclusion change risk. Rules were developed to extract relevant information from published Cochrane reviews and used to construct four features: the number of included trials and participants in the reviews, a measure based on the number of participants, and the time elapsed between the search dates. We compared the performance of random forest, decision tree, and logistic regression to predict the conclusion change risk. The performance was measured by accuracy, precision, recall, F1 -score, and area under ROC (AU-ROC). One rule was developed to extract the conclusion change information (96% accuracy, 100 reviews), one for the search date (100% accuracy, 100 reviews), one for the number of included clinical trials (100% accuracy, 100 reviews), and 22 for the number of participants (97.3% accuracy, 200 reviews). For unseen reviews, the random forest classifier showed the highest accuracy (80.8%) and AU-ROC (0.80). All classifiers showed relatively similar performance with overlapping 95% confidence interval (CI). The coverage score was shown to be the most useful feature for predicting the conclusion change risk. Features mined from Cochrane reviews and updates can estimate conclusion change risk. If data from more published reviews and updates were made accessible, data-driven methods to predict the conclusion change risk may be a feasible way to support decisions about updating reviews.

Limited Role of Bots in Spreading Vaccine-Critical Information Among Active Twitter Users in the United States: 2017-2019.

Objectives. To examine the role that bots play in spreading vaccine information on Twitter by measuring exposure and engagement among active users from the United States.Methods. We sampled 53 188 US Twitter users and examined who they follow and retweet across 21 million vaccine-related tweets (January 12, 2017-December 3, 2019). Our analyses compared bots to human-operated accounts and vaccine-critical tweets to other vaccine-related tweets.Results. The median number of potential exposures to vaccine-related tweets per user was 757 (interquartile range [IQR] = 168-4435), of which 27 (IQR = 6-169) were vaccine critical, and 0 (IQR = 0-12) originated from bots. We found that 36.7% of users retweeted vaccine-related content, 4.5% retweeted vaccine-critical content, and 2.1% retweeted vaccine content from bots. Compared with other users, the 5.8% for whom vaccine-critical tweets made up most exposures more often retweeted vaccine content (62.9%; odds ratio [OR] = 2.9; 95% confidence interval [CI] = 2.7, 3.1), vaccine-critical content (35.0%; OR = 19.0; 95% CI = 17.3, 20.9), and bots (8.8%; OR = 5.4; 95% CI = 4.7, 6.3).Conclusions. A small proportion of vaccine-critical information that reaches active US Twitter users comes from bots.

Automatically Appraising the Credibility of Vaccine-Related Web Pages Shared on Social Media: A Twitter Surveillance Study.

BACKGROUND: Tools used to appraise the credibility of health information are time-consuming to apply and require context-specific expertise, limiting their use for quickly identifying and mitigating the spread of misinformation as it emerges. OBJECTIVE: The aim of this study was to estimate the proportion of vaccine-related Twitter posts linked to Web pages of low credibility and measure the potential reach of those posts. METHODS: Sampling from 143,003 unique vaccine-related Web pages shared on Twitter between January 2017 and March 2018, we used a 7-point checklist adapted from validated tools and guidelines to manually appraise the credibility of 474 Web pages. These were used to train several classifiers (random forests, support vector machines, and recurrent neural networks) using the text from a Web page to predict whether the information satisfies each of the 7 criteria. Estimating the credibility of all other Web pages, we used the follower network to estimate potential exposures relative to a credibility score defined by the 7-point checklist. RESULTS: The best-performing classifiers were able to distinguish between low, medium, and high credibility with an accuracy of 78% and labeled low-credibility Web pages with a precision of over 96%. Across the set of unique Web pages, 11.86% (16,961 of 143,003) were estimated as low credibility and they generated 9.34% (1.64 billion of 17.6 billion) of potential exposures. The 100 most popular links to low credibility Web pages were each potentially seen by an estimated 2 million to 80 million Twitter users globally. CONCLUSIONS: The results indicate that although a small minority of low-credibility Web pages reach a large audience, low-credibility Web pages tend to reach fewer users than other Web pages overall and are more commonly shared within certain subpopulations. An automatic credibility appraisal tool may be useful for finding communities of users at higher risk of exposure to low-credibility vaccine communications.

Tracking a moving user in indoor environments using Bluetooth low energy beacons.

BACKGROUND: Bluetooth low energy (BLE) beacons have been used to track the locations of individuals in indoor environments for clinical applications such as workflow analysis and infectious disease modelling. Most current approaches use the received signal strength indicator (RSSI) to track locations. When using the RSSI to track indoor locations, devices need to be calibrated to account for complex interference patterns, which is a laborious process. Our aim was to investigate an alternative method for indoor location tracking of a moving user using BLE beacons in dynamic indoor environments. METHODS AND MATERIALS: We developed a new method based on the received number of signals indicator (RNSI) and compared it to a standard RSSI-based method for predicting a user's location. Experiments were performed in an office environment and a tertiary hospital. Both RNSI and RSSI were compared at various distances from BLE beacons. In moving user experiments, a user wearing a beacon walked from one location to another based on a pre-defined route. Performance in predicting user locations was measured based on accuracy. RESULTS: RNSI values decreased substantially with distance from the BLE beacon than RSSI values. Moving user experiments in the office environment demonstrated that the RNSI-based method produced higher accuracy (80.0%) than the RSSI-based method (76.2%). In the hospital, where the environment may introduce signal quality problems due to increased signal interference, the RNSI-based method still outperformed (83.3%) the RSSI-based method (51.9%). CONCLUSIONS: Our results suggest that the RNSI-based method could be useful to track the locations of a moving user without involving complex calibration, especially when deploying within a new environment. RNSI has the potential to be used together with other methods in more robust indoor positioning systems.

HPV vaccine coverage in Australia and associations with HPV vaccine information exposure among Australian Twitter users.

Introduction: Human papillomavirus (HPV) vaccine coverage in Australia is 80% for females and 76% for males. Attitudes may influence coverage but surveys measuring attitudes are resource-intensive. The aim of this study was to determine whether Twitter-derived estimates of HPV vaccine information exposure were associated with differences in coverage across regions in Australia. Methods: Regional differences in information exposure were estimated from 1,103,448 Australian Twitter users and 655,690 HPV vaccine related tweets posted between 6 September 2013 and 1 September 2017. Tweets about HPV vaccines were grouped using topic modelling; an algorithm for clustering text-based data. Proportional exposure to topics across 25 regions in Australia were used as factors to model HPV vaccine coverage in females and males, and compared to models using employment and education as factors. Results: Models using topic exposure measures were more closely correlated with HPV vaccine coverage (female: Pearson's R = 0.75 [0.49 to 0.88]; male: R = 0.76 [0.51 to 0.89]) than models using employment and education as factors (female: 0.39 [-0.02 to 0.68]; male: 0.36 [-0.04 to 0.66]). In Australia, positively-framed news tended to reach more Twitter users overall, but vaccine-critical information made up higher proportions of exposures among Twitter users in low coverage regions, where distorted characterisations of safety research and vaccine-critical blogs were popular. Conclusions: Twitter-derived models of information exposure were correlated with HPV vaccine coverage in Australia. Topic exposure measures may be useful for providing timely and localised reports of the information people access and share to inform the design of targeted vaccine promotion interventions.

The risk of conclusion change in systematic review updates can be estimated by learning from a database of published examples.

OBJECTIVES: To determine which systematic review characteristics are needed to estimate the risk of conclusion change in systematic review updates. STUDY DESIGN AND SETTING: We applied classification trees (a machine learning method) to model the risk of conclusion change in systematic review updates, using pairs of systematic reviews and their updates as samples. The classifiers were constructed using a set of features extracted from systematic reviews and the relevant trials added in published updates. Model performance was measured by recall, precision, and area under the receiver operating characteristic curve (AUC). RESULTS: We identified 63 pairs of systematic reviews and updates, of which 20 (32%) exhibited a change in conclusion in their updates. A classifier using information about new trials exhibited the highest performance (AUC: 0.71; recall: 0.75; precision: 0.43) compared to a classifier that used fewer features (AUC: 0.65; recall: 0.75; precision: 0.39). CONCLUSION: When estimating the risk of conclusion change in systematic review updates, information about the sizes of trials that will be added in an update are most useful. Future tools aimed at signaling conclusion change risks would benefit from complementary tools that automate screening of relevant trials.

Trial2rev: Combining machine learning and crowd-sourcing to create a shared space for updating systematic reviews.

OBJECTIVES: Systematic reviews of clinical trials could be updated faster by automatically monitoring relevant trials as they are registered, completed, and reported. Our aim was to provide a public interface to a database of curated links between systematic reviews and trial registrations. MATERIALS AND METHODS: We developed the server-side system components in Python, connected them to a PostgreSQL database, and implemented the web-based user interface using Javascript, HTML, and CSS. All code is available on GitHub under an open source MIT license and registered users can access and download all available data. RESULTS: The trial2rev system is a web-based interface to a database that collates and augments information from multiple sources including bibliographic databases, the ClinicalTrials.gov registry, and the actions of registered users. Users interact with the system by browsing, searching, or adding systematic reviews, verifying links to trials included in the review, and adding or voting on trials that they would expect to include in an update of the systematic review. The system can trigger the actions of software agents that add or vote on included and relevant trials, in response to user interactions or by scheduling updates from external resources. DISCUSSION AND CONCLUSION: We designed a publicly-accessible resource to help systematic reviewers make decisions about systematic review updates. Where previous approaches have sought to reactively filter published reports of trials for inclusion in systematic reviews, our approach is to proactively monitor for relevant trials as they are registered and completed.

Time-to-update of systematic reviews relative to the availability of new evidence.

BACKGROUND: A number of methods for deciding when a systematic review should be updated have been proposed, yet little is known about whether systematic reviews are updated more quickly when new evidence becomes available. Our aim was to examine the timing of systematic review updates relative to the availability of new evidence. METHODS: We performed a retrospective analysis of the update timing of systematic reviews published in the Cochrane Database of Systematic Reviews in 2010 relative to the availability of new trial evidence. We compared the update timing of systematic reviews with and without signals defined by the completion or publication of studies that were included in the updates. RESULTS: We found 43% (293/682) systematic reviews were updated before June 2017, of which 204 included an updated primary outcome meta-analysis (median update time 35.4 months; IQR 25.5-54.0), 38% (77/204) added new trials, and 4% (8/204) reported a change in conclusion. In the 171 systematic reviews with reconcilable trial reporting information, we did not find a clear difference in update timing (p = 0.05) between the 15 systematic reviews with a publication signal (median 25.3 months; IQR 15.3-43.5) and the 156 systematic reviews without a publication signal (median 34.4 months; IQR 25.1-52.2). In the 145 systematic reviews with reconcilable trial completion information, we did not find a difference in update timing (p = 0.33) between the 15 systematic reviews with a trial completion signal (median 26.0 months; IQR 19.3-49.5) and the 130 systematic reviews without a trial completion signal (median 32.4 months; IQR 24.1 to 46.0). CONCLUSION: A minority of 2010 Cochrane reviews were updated before June 2017 to incorporate evidence from new primary studies, and very few updates led to a change in conclusion. We did not find clear evidence that updates were undertaken faster when new evidence was made available. New approaches for finding early signals that a systematic review conclusion is at risk of change may be useful in allocated resources to the updating of systematic reviews.

Conversational agents in healthcare: a systematic review.

Objective: Our objective was to review the characteristics, current applications, and evaluation measures of conversational agents with unconstrained natural language input capabilities used for health-related purposes. Methods: We searched PubMed, Embase, CINAHL, PsycInfo, and ACM Digital using a predefined search strategy. Studies were included if they focused on consumers or healthcare professionals; involved a conversational agent using any unconstrained natural language input; and reported evaluation measures resulting from user interaction with the system. Studies were screened by independent reviewers and Cohen's kappa measured inter-coder agreement. Results: The database search retrieved 1513 citations; 17 articles (14 different conversational agents) met the inclusion criteria. Dialogue management strategies were mostly finite-state and frame-based (6 and 7 conversational agents, respectively); agent-based strategies were present in one type of system. Two studies were randomized controlled trials (RCTs), 1 was cross-sectional, and the remaining were quasi-experimental. Half of the conversational agents supported consumers with health tasks such as self-care. The only RCT evaluating the efficacy of a conversational agent found a significant effect in reducing depression symptoms (effect size d = 0.44, p = .04). Patient safety was rarely evaluated in the included studies. Conclusions: The use of conversational agents with unconstrained natural language input capabilities for health-related purposes is an emerging field of research, where the few published studies were mainly quasi-experimental, and rarely evaluated efficacy or safety. Future studies would benefit from more robust experimental designs and standardized reporting. Protocol Registration: The protocol for this systematic review is registered at PROSPERO with the number CRD42017065917.

A shared latent space matrix factorisation method for recommending new trial evidence for systematic review updates.

BACKGROUND: Clinical trial registries can be used to monitor the production of trial evidence and signal when systematic reviews become out of date. However, this use has been limited to date due to the extensive manual review required to search for and screen relevant trial registrations. Our aim was to evaluate a new method that could partially automate the identification of trial registrations that may be relevant for systematic review updates. MATERIALS AND METHODS: We identified 179 systematic reviews of drug interventions for type 2 diabetes, which included 537 clinical trials that had registrations in ClinicalTrials.gov. Text from the trial registrations were used as features directly, or transformed using Latent Dirichlet Allocation (LDA) or Principal Component Analysis (PCA). We tested a novel matrix factorisation approach that uses a shared latent space to learn how to rank relevant trial registrations for each systematic review, comparing the performance to document similarity to rank relevant trial registrations. The two approaches were tested on a holdout set of the newest trials from the set of type 2 diabetes systematic reviews and an unseen set of 141 clinical trial registrations from 17 updated systematic reviews published in the Cochrane Database of Systematic Reviews. The performance was measured by the number of relevant registrations found after examining 100 candidates (recall@100) and the median rank of relevant registrations in the ranked candidate lists. RESULTS: The matrix factorisation approach outperformed the document similarity approach with a median rank of 59 (of 128,392 candidate registrations in ClinicalTrials.gov) and recall@100 of 60.9% using LDA feature representation, compared to a median rank of 138 and recall@100 of 42.8% in the document similarity baseline. In the second set of systematic reviews and their updates, the highest performing approach used document similarity and gave a median rank of 67 (recall@100 of 62.9%). CONCLUSIONS: A shared latent space matrix factorisation method was useful for ranking trial registrations to reduce the manual workload associated with finding relevant trials for systematic review updates. The results suggest that the approach could be used as part of a semi-automated pipeline for monitoring potentially new evidence for inclusion in a review update.

Comparing human papillomavirus vaccine concerns on Twitter: a cross-sectional study of users in Australia, Canada and the UK.

OBJECTIVE: Opposition to human papillomavirus (HPV) vaccination is common on social media and has the potential to impact vaccine coverage. This study aims to conduct an international comparison of the proportions of tweets about HPV vaccines that express concerns, the types of concerns expressed and the social connections among users posting about HPV vaccines in Australia, Canada and the UK. DESIGN: Using a cross-sectional design, an international comparison of English language tweets about HPV vaccines and social connections among Twitter users posting about HPV vaccines between January 2014 and April 2016 was conducted. The Health Belief Model, one of the most widely used theories in health psychology, was used as the basis for coding the types of HPV vaccine concerns expressed on Twitter. SETTING: The content of tweets and the social connections between users who posted tweets about HPV vaccines from Australia, Canada and the UK. POPULATION: 16 789 Twitter users who posted 43 852 tweets about HPV vaccines. MAIN OUTCOME MEASURES: The proportions of tweets expressing concern, the type of concern expressed and the proportions of local and international social connections between users. RESULTS: Tweets expressing concerns about HPV vaccines made up 14.9% of tweets in Canada, 19.4% in Australia and 22.6% in the UK. The types of concerns expressed were similar across the three countries, with concerns related to 'perceived barriers' being the most common. Users expressing concerns about HPV vaccines in each of the three countries had a relatively high proportion of international followers also expressing concerns. CONCLUSIONS: The proportions and types of HPV vaccine concerns expressed on Twitter were similar across the three countries. Twitter users who mostly expressed concerns about HPV vaccines were better connected to international users who shared their concerns compared with users who did not express concerns about HPV vaccines.

Mapping information exposure on social media to explain differences in HPV vaccine coverage in the United States.

BACKGROUND: Together with access, acceptance of vaccines affects human papillomavirus (HPV) vaccine coverage, yet little is known about media's role. Our aim was to determine whether measures of information exposure derived from Twitter could be used to explain differences in coverage in the United States. METHODS: We conducted an analysis of exposure to information about HPV vaccines on Twitter, derived from 273.8 million exposures to 258,418 tweets posted between 1 October 2013 and 30 October 2015. Tweets were classified by topic using machine learning methods. Proportional exposure to each topic was used to construct multivariable models for predicting state-level HPV vaccine coverage, and compared to multivariable models constructed using socioeconomic factors: poverty, education, and insurance. Outcome measures included correlations between coverage and the individual topics and socioeconomic factors; and differences in the predictive performance of the multivariable models. RESULTS: Topics corresponding to media controversies were most closely correlated with coverage (both positively and negatively); education and insurance were highest among socioeconomic indicators. Measures of information exposure explained 68% of the variance in one dose 2015 HPV vaccine coverage in females (males: 63%). In comparison, models based on socioeconomic factors explained 42% of the variance in females (males: 40%). CONCLUSIONS: Measures of information exposure derived from Twitter explained differences in coverage that were not explained by socioeconomic factors. Vaccine coverage was lower in states where safety concerns, misinformation, and conspiracies made up higher proportions of exposures, suggesting that negative representations of vaccines in the media may reflect or influence vaccine acceptance.

Characterizing Twitter Discussions About HPV Vaccines Using Topic Modeling and Community Detection.

BACKGROUND: In public health surveillance, measuring how information enters and spreads through online communities may help us understand geographical variation in decision making associated with poor health outcomes. OBJECTIVE: Our aim was to evaluate the use of community structure and topic modeling methods as a process for characterizing the clustering of opinions about human papillomavirus (HPV) vaccines on Twitter. METHODS: The study examined Twitter posts (tweets) collected between October 2013 and October 2015 about HPV vaccines. We tested Latent Dirichlet Allocation and Dirichlet Multinomial Mixture (DMM) models for inferring topics associated with tweets, and community agglomeration (Louvain) and the encoding of random walks (Infomap) methods to detect community structure of the users from their social connections. We examined the alignment between community structure and topics using several common clustering alignment measures and introduced a statistical measure of alignment based on the concentration of specific topics within a small number of communities. Visualizations of the topics and the alignment between topics and communities are presented to support the interpretation of the results in context of public health communication and identification of communities at risk of rejecting the safety and efficacy of HPV vaccines. RESULTS: We analyzed 285,417 Twitter posts (tweets) about HPV vaccines from 101,519 users connected by 4,387,524 social connections. Examining the alignment between the community structure and the topics of tweets, the results indicated that the Louvain community detection algorithm together with DMM produced consistently higher alignment values and that alignments were generally higher when the number of topics was lower. After applying the Louvain method and DMM with 30 topics and grouping semantically similar topics in a hierarchy, we characterized 163,148 (57.16%) tweets as evidence and advocacy, and 6244 (2.19%) tweets describing personal experiences. Among the 4548 users who posted experiential tweets, 3449 users (75.84%) were found in communities where the majority of tweets were about evidence and advocacy. CONCLUSIONS: The use of community detection in concert with topic modeling appears to be a useful way to characterize Twitter communities for the purpose of opinion surveillance in public health applications. Our approach may help identify online communities at risk of being influenced by negative opinions about public health interventions such as HPV vaccines.