Developing public health surveillance dashboards: a scoping review on the design principles.

BACKGROUND: Public Health Dashboards (PHDs) facilitate the monitoring and prediction of disease outbreaks by continuously monitoring the health status of the community. This study aimed to identify design principles and determinants for developing public health surveillance dashboards. METHODOLOGY: This scoping review is based on Arksey and O'Malley's framework as included in JBI guidance. Four databases were used to review and present the proposed principles of designing PHDs: IEEE, PubMed, Web of Science, and Scopus. We considered articles published between January 1, 2010 and November 30, 2022. The final search of articles was done on November 30, 2022. Only articles in the English language were included. Qualitative synthesis and trend analysis were conducted. RESULTS: Findings from sixty-seven articles out of 543 retrieved articles, which were eligible for analysis, indicate that most of the dashboards designed from 2020 onwards were at the national level for managing and monitoring COVID-19. Design principles for the public health dashboard were presented in five groups, i.e., considering aim and target users, appropriate content, interface, data analysis and presentation types, and infrastructure. CONCLUSION: Effective and efficient use of dashboards in public health surveillance requires implementing design principles to improve the functionality of these systems in monitoring and decision-making. Considering user requirements, developing a robust infrastructure for improving data accessibility, developing, and applying Key Performance Indicators (KPIs) for data processing and reporting purposes, and designing interactive and intuitive interfaces are key for successful design and development.

Marketing yourself in social media as a global health organization: lessons to be learned.

PURPOSE: To evaluate whether X, formerly known as Twitter, is being used effectively to advance the goals of International Volunteers in Urology (IVUmed). How is X activity associated with end-user engagement? METHODS: Monthly analytics of the X account @IVUmed were reviewed between September 2014 and November 2022 using https://analytics.twitter.com/ . Outcomes included tweets, mentions, impressions, engagements, interactions, followers, and profile visits. Statistical analysis using Mann-Whitney U test and Spearman's rank-order correlation was performed. Top tweet content between December 2020 and November 2022 was also analyzed and assigned one of seven different categories: research, workshops, mission statement, educational materials, fundraising, individual spotlight, and other. RESULTS: Of @IVUmed's 1668 followers, 1334 (80.0%) were individuals. One thousand one hundred twenty-six (84.4%) individuals listed their locations with the majority (79.8%) residing in high-income countries. Tweet impressions have increased over time; they were significantly higher (p < 0.01) on average after the onset of COVID-19 in March 2020. From December 2020 to November 2022, new followers were positively correlated with tweet impressions (p < 0.01), total mentions (p < 0.01), and profile visits (p < 0.01). Profile visits were positively correlated with total tweets (p < 0.01). The content categories for monthly top tweets that proportionally garnered the most engagements were workshops (50%) and individual spotlight (29%), despite not being the most tweeted about content categories. CONCLUSION: Non-profit organizations wishing to increase their web-based outreach can benefit from increased primary X activity. While not evaluated in this study, it may also improve fundraising capabilities. Nevertheless, periodic review of account activity is important to ensure engagement of the targeted audience.

Predictors of COVID-19 From a Statewide Digital Symptom and Risk Assessment Tool: Cross-Sectional Study.

BACKGROUND: Some of the most vexing issues with the COVID-19 pandemic were the inability of facilities and events, such as schools and work areas, to track symptoms to mitigate the spread of the disease. To combat these challenges, many turned to the implementation of technology. Technology solutions to mitigate repercussions of the COVID-19 pandemic include tools that provide guidelines and interfaces to influence behavior, reduce exposure to the disease, and enable policy-driven avenues to return to a sense of normalcy. This paper presents the implementation and early evaluation of a return-to-work COVID-19 symptom and risk assessment tool. The system was implemented across 34 institutions of health and education in Alabama, including more than 174,000 users with over 4 million total uses and more than 86,000 reports of exposure risk between July 2020 and April 2021. OBJECTIVE: This study aimed to explore the usage of technology, specifically a COVID-19 symptom and risk assessment tool, to mitigate exposure to COVID-19 within public spaces. More specifically, the objective was to assess the relationship between user-reported symptoms and exposure via a mobile health app, with confirmed COVID-19 cases reported by the Alabama Department of Public Health (ADPH). METHODS: This cross-sectional study evaluated the relationship between confirmed COVID-19 cases and user-reported COVID-19 symptoms and exposure reported through the Healthcheck web-based mobile application. A dependent variable for confirmed COVID-19 cases in Alabama was obtained from ADPH. Independent variables (ie, health symptoms and exposure) were collected through Healthcheck survey data and included measures assessing COVID-19-related risk levels and symptoms. Multiple linear regression was used to examine the relationship between ADPH-confirmed diagnosis of COVID-19 and self-reported health symptoms and exposure via Healthcheck that were analyzed across the state population but not connected at the individual patient level. RESULTS: Regression analysis showed that the self-reported information collected by Healthcheck significantly affects the number of COVID-19-confirmed cases. The results demonstrate that the average number of confirmed COVID-19 cases increased by 5 (high risk: ß=5.10; P=.001), decreased by 24 (sore throat: ß=-24.03; P=.001), and increased by 21 (nausea or vomiting: ß=21.67; P=.02) per day for every additional self-report of symptoms by Healthcheck survey respondents. Congestion or runny nose was the most frequently reported symptom. Sore throat, low risk, high risk, nausea, or vomiting were all statistically significant factors. CONCLUSIONS: The use of technology allowed organizations to remotely track a population as it is related to COVID-19. Healthcheck was a platform that aided in symptom tracking, risk assessment, and evaluation of status for admitting individuals into public spaces for people in the Alabama area. The confirmed relationship between symptom and exposure self-reporting using an app and population-wide confirmed cases suggests that further investigation is needed to determine the opportunity for such apps to mitigate disease spread at a community and individual level.

An anti-infodemic virtual center for the Americas.

The Pan American Health Organization/World Health Organization (PAHO/WHO) Anti-Infodemic Virtual Center for the Americas (AIVCA) is a project led by the Department of Evidence and Intelligence for Action in Health, PAHO and the Center for Health Informatics, PAHO/WHO Collaborating Center on Information Systems for Health, at the University of Illinois, with the participation of PAHO staff and consultants across the region. Its goal is to develop a set of tools-pairing AI with human judgment-to help ministries of health and related health institutions respond to infodemics. Public health officials will learn about emerging threats detected by the center and get recommendations on how to respond. The virtual center is structured with three parallel teams: detection, evidence, and response. The detection team will employ a mixture of advanced search queries, machine learning, and other AI techniques to sift through more than 800 million new public social media posts per day to identify emerging infodemic threats in both English and Spanish. The evidence team will use the EasySearch federated search engine backed by AI, PAHO's knowledge management team, and the Librarian Reserve Corps to identify the most relevant authoritative sources. The response team will use a design approach to communicate recommended response strategies based on behavioural science, storytelling, and information design approaches.

El centro virtual contra la infodemia para la Región de las Américas de la Organización Panamericana de la Salud/Organización Mundial de la Salud (OPS/OMS) es un proyecto liderado por el Departamento de Evidencia e Inteligencia para la Acción en la Salud de la OPS y el Center for Health Informatics de la Universidad de Illinois, centro colaborador de la OPS/OMS en sistemas de información para la salud, con la participación de personal y consultores de la OPS en toda la Región. Su objetivo es crear un conjunto de herramientas que combinen inteligencia artificial (IA) y los criterios humanos para apoyar a los ministerios de salud y las instituciones relacionadas con la salud en la respuesta a la infodemia. Los funcionarios de salud pública recibirán formación sobre las amenazas emergentes detectadas por el centro y recomendaciones sobre cómo abordarlas. El centro virtual está estructurado en tres equipos paralelos: detección, evidencia y respuesta. El equipo de detección empleará una combinación de consultas mediante búsqueda avanzada, aprendizaje automático y otras técnicas de IA para evaluar más de 800 millones de publicaciones nuevas en las redes sociales al día con el fin de detectar amenazas emergentes en el ámbito de la infodemia tanto en inglés como en español. El equipo de evidencia hará uso del motor de búsqueda federado EasySearch y, con el apoyo de la IA, el equipo de gestión del conocimiento de la OPS y la red Librarian Reserve Corps, determinará cuáles son las fuentes autorizadas más pertinentes. El equipo de respuesta utilizará un enfoque vinculado al diseño para difundir las estrategias recomendadas sobre la base de las ciencias del comportamiento, la narración de historias y el diseño de la información.

O Centro Virtual Anti-Infodemia para as Américas (AIVCA, na sigla em inglês) da Organização Pan-Americana da Saúde/Organização Mundial da Saúde (OPAS/OMS) é um projeto liderado pelo Departamento de Evidência e Inteligência para a Ação em Saúde da OPAS e pelo Centro de Informática em Saúde da Universidade de Illinois, EUA (Centro Colaborador da OPAS/OMS para Sistemas de Informação para a Saúde), com a participação de funcionários e consultores da OPAS de toda a região. Seu objetivo é desenvolver um conjunto de ferramentas ­ combinando a inteligência artificial (IA) com o discernimento humano ­ para ajudar os ministérios e instituições de saúde a responder às infodemias. As autoridades de saúde pública aprenderão sobre as ameaças emergentes detectadas pelo centro e obterão recomendações sobre como responder. O centro virtual está estruturado com três equipes paralelas: detecção, evidência e resposta. A equipe de detecção utilizará consultas de pesquisa avançada, machine learning (aprendizagem de máquina) e outras técnicas de IA para filtrar mais de 800 milhões de novas postagens públicas nas redes sociais por dia, a fim de identificar ameaças infodêmicas emergentes em inglês e espanhol. A equipe de evidência usará o mecanismo de busca federada EasySearch, com apoio de IA, da equipe de gestão de conhecimento da OPAS e do Librarian Reserve Corps (LRC), para identificar as fontes abalizadas mais relevantes. A equipe de resposta usará uma abordagem de design para comunicar estratégias de resposta recomendadas com base em abordagens de ciência comportamental, narração de histórias e design da informação.

Supporting rural public health practice to address local-level social determinants of health across Northwest states: Development of an interactive visualization dashboard.

BACKGROUND: Rural local health departments (LHDs) lack adequate capacity and funding to effectively make data-driven decisions to support their communities that face greater health disparities compared to urban counterparts. The need, therefore, exists for informatics solutions to support rural LHDs. PURPOSE: We describe the user-centered design (UCD) of SHARE-NW: Solutions in Health Analytics for Rural Equity across the Northwest, a website (sharenw.nwcphp.org) with data visualization dashboards for rural LHD practitioners in Alaska, Idaho, Oregon, and Washington to help them identify health disparities in their jurisdictions. METHODS: In this UCD study guided by Munzner's Nested Model for Visualization Design and Validation, we (1) completed a needs assessment, (2) created and evaluated mockups, and (3) conducted usability testing of a functional alpha testing website. Potential end-users (rural LHD practitioners) and Equity Advisory Committee members (public health experts from state, rural local, and tribal public health agencies) across our four-state catchment area were engaged throughout the website development and testing. We adapted traditional in-person UCD methods to be remote to reach participants across a large geographic area and in rural/frontier areas of Alaska, Idaho, Oregon, and Washington. RESULTS: We recruited participants from all four states to engage in each stage of the project. Needs assessment findings informed the mockup development, and findings from the mockup evaluations informed the development of the functional website. Usability testing of the website overall was positive, with priority usability issues identified. CONCLUSIONS: By applying Munzner's Nested Model and UCD, we could purposefully and intentionally design evidence-based solutions, specifically for rural LHD practitioners. Adaptations of traditional UCD methods were successful and allowed us to reach end-users across a large geographic area. Future work on SHARE-NW will involve the evaluation of the website. We provide insights on our lessons learned to support future public health informatics solution development.

Data sources for precision public health of obesity: a scoping review, evidence map and use case in Queensland, Australia.

BACKGROUND: Global action to reduce obesity prevalence requires digital transformation of the public health sector to enable precision public health (PPH). Useable data for PPH of obesity is yet to be identified, collated and appraised and there is currently no accepted approach to creating this single source of truth. This scoping review aims to address this globally generic problem by using the State of Queensland (Australia) (population > 5 million) as a use case to determine (1) availability of primary data sources usable for PPH for obesity (2) quality of identified sources (3) general implications for public health policymakers. METHODS: The Preferred Reporting Items for Systematic Review and Meta-Analyses extension for scoping reviews (PRISMA-ScR) was followed. Unique search strategies were implemented for 'designed' (e.g. surveys) and 'organic' (e.g. electronic health records) data sources. Only primary sources of data (with stratification to Queensland) with evidence-based determinants of obesity were included. Primary data source type, availability, sample size, frequency of collection and coverage of determinants of obesity were extracted and curated into an evidence map. Data source quality was qualitatively assessed. RESULTS: We identified 38 primary sources of preventive data for obesity: 33 designed and 5 organic. Most designed sources were survey (n 20) or administrative (n 10) sources and publicly available but generally were not contemporaneous (> 2 years old) and had small sample sizes (10-100 k) relative to organic sources (> 1 M). Organic sources were identified as the electronic medical record (ieMR), wearables, environmental (Google Maps, Crime Map) and billing/claims. Data on social, biomedical and behavioural determinants of obesity typically co-occurred across sources. Environmental and commercial data was sparse and interpreted as low quality. One organic source (ieMR) was highly contemporaneous (routinely updated), had a large sample size (5 M) and represented all determinants of obesity but is not currently used for public health decision-making in Queensland. CONCLUSIONS: This review provides a (1) comprehensive data map for PPH for obesity in Queensland and (2) globally translatable framework to identify, collate and appraise primary data sources to advance PPH for obesity and other noncommunicable diseases. Significant challenges must be addressed to achieve PPH, including: using designed and organic data harmoniously, digital infrastructure for high-quality organic data, and the ethical and social implications of using consumer-centred health data to improve public health.

Developing a framework for understanding health information behavior change from avoidance to acquisition: a grounded theory exploration.

BACKGROUND: Health information avoidance is common in real life, but because it is not always conducive to health promotion and maintenance, people often actively switch to health information acquisition. Understanding this process of active change can facilitate intervention in unreasonable avoidance behaviors. However, studies so far have mostly focused on why and how avoidance takes place, little is known about the process of active change from avoidance to acquisition. We thus use a grounded theory approach (GT) to explore how the active change takes place, and to generate a grounded theoretical framework capable of illustrating stages and influencing factors involved in the active change process. METHODS: Straussian grounded theory (Corbin & Strauss, 2015) was used to analyze data collected through semi-structured interviews with 30 adults (14 in good health, 11 with disease, 5 in other health status) who had experienced health information behavior change from avoidance to acquisition. These interviews focused on how the change occurred and what effected the change. RESULTS: The core category of Health Information Avoidance Change and 12 categories were identified and integrated to form a theoretical framework termed the Health Information Avoidance Change Model (HIACM). This model describes the process using five non-linear stage variables (initiation, preparation, action, maintenance, and abandonment) and seven moderating factor variables (cognitive change, social stimulus, beliefs and attitudes, intrapsychic literacy, social resources, information source, time and material resources). CONCLUSIONS: HIACM can be used to explain the process of active change from health information avoidance to health information acquisition. HIAC is a non-linear and holistic process, and it is necessary to dynamically analyze the impact of relevant factors and take targeted intervention measures in stages. HIAC is usually not only an individual behavior, but also a socialized behavior requiring the collaboration of individuals, families, health information providers, healthcare providers, and governments.

Real-world data for precision public health of noncommunicable diseases: a scoping review.

BACKGROUND: Global public health action to address noncommunicable diseases (NCDs) requires new approaches. NCDs are primarily prevented and managed in the community where there is little investment in digital health systems and analytics; this has created a data chasm and relatively silent burden of disease. The nascent but rapidly emerging area of precision public health offers exciting new opportunities to transform our approach to NCD prevention. Precision public health uses routinely collected real-world data on determinants of health (social, environmental, behavioural, biomedical and commercial) to inform precision decision-making, interventions and policy based on social position, equity and disease risk, and continuously monitors outcomes - the right intervention for the right population at the right time. This scoping review aims to identify global exemplars of precision public health and the data sources and methods of their aggregation/application to NCD prevention. METHODS: The Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews (PRISMA-ScR) was followed. Six databases were systematically searched for articles published until February 2021. Articles were included if they described digital aggregation of real-world data and 'traditional' data for applied community, population or public health management of NCDs. Real-world data was defined as routinely collected (1) Clinical, Medication and Family History (2) Claims/Billing (3) Mobile Health (4) Environmental (5) Social media (6) Molecular profiling (7) Patient-centred (e.g., personal health record). Results were analysed descriptively and mapped according to the three horizons framework for digital health transformation. RESULTS: Six studies were included. Studies developed population health surveillance methods and tools using diverse real-world data (e.g., electronic health records and health insurance providers) and traditional data (e.g., Census and administrative databases) for precision surveillance of 28 NCDs. Population health analytics were applied consistently with descriptive, geospatial and temporal functions. Evidence of using surveillance tools to create precision public health models of care or improve policy and practice decisions was unclear. CONCLUSIONS: Applications of real-world data and designed data to address NCDs are emerging with greater precision. Digital transformation of the public health sector must be accelerated to create an efficient and sustainable predict-prevent healthcare system.

Co-establishing an infrastructure for routine data collection to address disparities in infant mortality: planning and implementation.

BACKGROUND: Efforts to address infant mortality disparities in Ohio have historically been adversely affected by the lack of consistent data collection and infrastructure across the community-based organizations performing front-line work with expectant mothers, and there is no established template for implementing such systems in the context of diverse technological capacities and varying data collection magnitude among participating organizations. METHODS: Taking into account both the needs and limitations of participating community-based organizations, we created a data collection infrastructure that was refined by feedback from sponsors and the organizations to serve as both a solution to their existing needs and a template for future efforts in other settings. RESULTS: By standardizing the collected data elements across participating organizations, integration on a scale large enough to detect changes in a rare outcome such as infant mortality was made possible. Datasets generated through the use of the established infrastructure were robust enough to be matched with other records, such as Medicaid and birth records, to allow more extensive analysis. CONCLUSION: While a consistent data collection infrastructure across multiple organizations does require buy-in at the organizational level, especially among participants with little to no existing data collection experience, an approach that relies on an understanding of existing barriers, iterative development, and feedback from sponsors and participants can lead to better coordination and sharing of information when addressing health concerns that individual organizations may struggle to quantify alone.

A Study of Publicly Available Resources Addressing Legal Data-Sharing Barriers: Systematic Assessment.

BACKGROUND: United States data protection laws vary depending on the data type and its context. Data projects involving social determinants of health often concern different data protection laws, making them difficult to navigate. OBJECTIVE: We systematically aggregated and assessed useful online resources to help navigate the data-sharing landscape. METHODS: We included publicly available resources that discussed legal data-sharing issues with some health relevance and published between 2010 and 2019. We conducted an iterative search with a common string pattern using a general-purpose search engine that targeted 24 different sectors identified by Data Across Sectors for Health. We scored each online resource for its depth of legal and data-sharing discussions and value for addressing legal barriers. RESULTS: Out of 3710 total search hits, 2721 unique URLs were reviewed for scope, 322 received full-text review, and 154 were selected for final coding. Legal agreements, consent, and agency guidance were the most widely covered legal topics, with HIPAA (The Health Insurance Portability and Accountability Act), Family Educational Rights and Privacy Act (FERPA), Title 42 of the Code of Federal Regulations Part 2 being the top 3 federal laws discussed. Clinical health care was the most prominent sector with a mention in 73 resources. CONCLUSIONS: This is the first systematic study of publicly available resources on legal data-sharing issues. We found existing gaps where resources covering certain laws or applications may be needed. The volume of resources we found is an indicator that real and perceived legal issues are a substantial barrier to efforts in leveraging data from different sectors to promote health.

Exploring COVID-19-Related Stressors: Topic Modeling Study.

BACKGROUND: The COVID-19 pandemic has affected the lives of people globally for over 2 years. Changes in lifestyles due to the pandemic may cause psychosocial stressors for individuals and could lead to mental health problems. To provide high-quality mental health support, health care organizations need to identify COVID-19-specific stressors and monitor the trends in the prevalence of those stressors. OBJECTIVE: This study aims to apply natural language processing (NLP) techniques to social media data to identify the psychosocial stressors during the COVID-19 pandemic and to analyze the trend in the prevalence of these stressors at different stages of the pandemic. METHODS: We obtained a data set of 9266 Reddit posts from the subreddit \rCOVID19_support, from February 14, 2020, to July 19, 2021. We used the latent Dirichlet allocation (LDA) topic model to identify the topics that were mentioned on the subreddit and analyzed the trends in the prevalence of the topics. Lexicons were created for each of the topics and were used to identify the topics of each post. The prevalences of topics identified by the LDA and lexicon approaches were compared. RESULTS: The LDA model identified 6 topics from the data set: (1) "fear of coronavirus," (2) "problems related to social relationships," (3) "mental health symptoms," (4) "family problems," (5) "educational and occupational problems," and (6) "uncertainty on the development of pandemic." According to the results, there was a significant decline in the number of posts about the "fear of coronavirus" after vaccine distribution started. This suggests that the distribution of vaccines may have reduced the perceived risks of coronavirus. The prevalence of discussions on the uncertainty about the pandemic did not decline with the increase in the vaccinated population. In April 2021, when the Delta variant became prevalent in the United States, there was a significant increase in the number of posts about the uncertainty of pandemic development but no obvious effects on the topic of fear of the coronavirus. CONCLUSIONS: We created a dashboard to visualize the trend in the prevalence of topics about COVID-19-related stressors being discussed on a social media platform (Reddit). Our results provide insights into the prevalence of pandemic-related stressors during different stages of the COVID-19 pandemic. The NLP techniques leveraged in this study could also be applied to analyze event-specific stressors in the future.

Interactive Visualization Applications in Population Health and Health Services Research: Systematic Scoping Review.

BACKGROUND: Simple visualizations in health research data, such as scatter plots, heat maps, and bar charts, typically present relationships between 2 variables. Interactive visualization methods allow for multiple related facets such as numerous risk factors to be studied simultaneously, leading to data insights through exploring trends and patterns from complex big health care data. The technique presents a powerful tool that can be used in combination with statistical analysis for knowledge discovery, hypothesis generation and testing, and decision support. OBJECTIVE: The primary objective of this scoping review is to describe and summarize the evidence of interactive visualization applications, methods, and tools being used in population health and health services research (HSR) and their subdomains in the last 15 years, from January 1, 2005, to March 30, 2019. Our secondary objective is to describe the use cases, metrics, frameworks used, settings, target audience, goals, and co-design of applications. METHODS: We adapted standard scoping review guidelines with a peer-reviewed search strategy: 2 independent researchers at each stage of screening and abstraction, with a third independent researcher to arbitrate conflicts and validate findings. A comprehensive abstraction platform was built to capture the data from diverse bodies of literature, primarily from the computer science and health care sectors. After screening 11,310 articles, we present findings from 56 applications from interrelated areas of population health and HSR, as well as their subdomains such as epidemiologic surveillance, health resource planning, access, and use and costs among diverse clinical and demographic populations. RESULTS: In this companion review to our earlier systematic synthesis of the literature on visual analytics applications, we present findings in 6 major themes of interactive visualization applications developed for 8 major problem categories. We found a wide application of interactive visualization methods, the major ones being epidemiologic surveillance for infectious disease, resource planning, health service monitoring and quality, and studying medication use patterns. The data sources included mostly secondary administrative and electronic medical record data. In addition, at least two-thirds of the applications involved participatory co-design approaches while introducing a distinct category, embedded research, within co-design initiatives. These applications were in response to an identified need for data-driven insights into knowledge generation and decision support. We further discuss the opportunities stemming from the use of interactive visualization methods in studying global health; inequities, including social determinants of health; and other related areas. We also allude to the challenges in the uptake of these methods. CONCLUSIONS: Visualization in health has strong historical roots, with an upward trend in the use of these methods in population health and HSR. Such applications are being fast used by academic and health care agencies for knowledge discovery, hypotheses generation, and decision support. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/14019.

The Benefits of Crowdsourcing to Seed and Align an Algorithm in an mHealth Intervention for African American and Hispanic Adults: Survey Study.

BACKGROUND: The lack of publicly available and culturally relevant data sets on African American and bilingual/Spanish-speaking Hispanic adults' disease prevention and health promotion priorities presents a major challenge for researchers and developers who want to create and test personalized tools built on and aligned with those priorities. Personalization depends on prediction and performance data. A recommender system (RecSys) could predict the most culturally and personally relevant preventative health information and serve it to African American and Hispanic users via a novel smartphone app. However, early in a user's experience, a RecSys can face the "cold start problem" of serving untailored and irrelevant content before it learns user preferences. For underserved African American and Hispanic populations, who are consistently being served health content targeted toward the White majority, the cold start problem can become an example of algorithmic bias. To avoid this, a RecSys needs population-appropriate seed data aligned with the app's purposes. Crowdsourcing provides a means to generate population-appropriate seed data. OBJECTIVE: Our objective was to identify and test a method to address the lack of culturally specific preventative personal health data and sidestep the type of algorithmic bias inherent in a RecSys not trained in the population of focus. We did this by collecting a large amount of data quickly and at low cost from members of the population of focus, thereby generating a novel data set based on prevention-focused, population-relevant health goals. We seeded our RecSys with data collected anonymously from self-identified Hispanic and self-identified non-Hispanic African American/Black adult respondents, using Amazon Mechanical Turk (MTurk). METHODS: MTurk provided the crowdsourcing platform for a web-based survey in which respondents completed a personal profile and a health information-seeking assessment, and provided data on family health history and personal health history. Respondents then selected their top 3 health goals related to preventable health conditions, and for each goal, reviewed and rated the top 3 information returns by importance, personal utility, whether the item should be added to their personal health library, and their satisfaction with the quality of the information returned. This paper reports the article ratings because our intent was to assess the benefits of crowdsourcing to seed a RecSys. The analysis of the data from health goals will be reported in future papers. RESULTS: The MTurk crowdsourcing approach generated 985 valid responses from 485 (49%) self-identified Hispanic and 500 (51%) self-identified non-Hispanic African American adults over the course of only 64 days at a cost of US $6.74 per respondent. Respondents rated 92 unique articles to inform the RecSys. CONCLUSIONS: Researchers have options such as MTurk as a quick, low-cost means to avoid the cold start problem for algorithms and to sidestep bias and low relevance for an intended population of app users. Seeding a RecSys with responses from people like the intended users allows for the development of a digital health tool that can recommend information to users based on similar demography, health goals, and health history. This approach minimizes the potential, initial gaps in algorithm performance; allows for quicker algorithm refinement in use; and may deliver a better user experience to individuals seeking preventative health information to improve health and achieve health goals.

The use of electronic health records to inform cancer surveillance efforts: a scoping review and test of indicators for public health surveillance of cancer prevention and control.

INTRODUCTION: State cancer prevention and control programs rely on public health surveillance data to set objectives to improve cancer prevention and control, plan interventions, and evaluate state-level progress towards achieving those objectives. The goal of this project was to evaluate the validity of using electronic health records (EHRs) based on common data model variables to generate indicators for surveillance of cancer prevention and control for these public health programs. METHODS: Following the methodological guidance from the PRISMA Extension for Scoping Reviews, we conducted a literature scoping review to assess how EHRs are used to inform cancer surveillance. We then developed 26 indicators along the continuum of the cascade of care, including cancer risk factors, immunizations to prevent cancer, cancer screenings, quality of initial care after abnormal screening results, and cancer burden. Indicators were calculated within a sample of patients from the New York City (NYC) INSIGHT Clinical Research Network using common data model EHR data and were weighted to the NYC population using post-stratification. We used prevalence ratios to compare these estimates to estimates from the raw EHR of NYU Langone Health to assess quality of information within INSIGHT, and we compared estimates to results from existing surveillance sources to assess validity. RESULTS: Of the 401 identified articles, 15% had a study purpose related to surveillance. Our indicator comparisons found that INSIGHT EHR-based measures for risk factor indicators were similar to estimates from external sources. In contrast, cancer screening and vaccination indicators were substantially underestimated as compared to estimates from external sources. Cancer screenings and vaccinations were often recorded in sections of the EHR that were not captured by the common data model. INSIGHT estimates for many quality-of-care indicators were higher than those calculated using a raw EHR. CONCLUSION: Common data model EHR data can provide rich information for certain indicators related to the cascade of care but may have substantial biases for others that limit their use in informing surveillance efforts for cancer prevention and control programs.

Validation of 4D Components for Measuring Quality of the Public Health Data Collection Process: Elicitation Study.

BACKGROUND: Identification of the essential components of the quality of the data collection process is the starting point for designing effective data quality management strategies for public health information systems. An inductive analysis of the global literature on the quality of the public health data collection process has led to the formation of a preliminary 4D component framework, that is, data collection management, data collection personnel, data collection system, and data collection environment. It is necessary to empirically validate the framework for its use in future research and practice. OBJECTIVE: This study aims to obtain empirical evidence to confirm the components of the framework and, if needed, to further develop this framework. METHODS: Expert elicitation was used to evaluate the preliminary framework in the context of the Chinese National HIV/AIDS Comprehensive Response Information Management System. The research processes included the development of an interview guide and data collection form, data collection, and analysis. A total of 3 public health administrators, 15 public health workers, and 10 health care practitioners participated in the elicitation session. A framework qualitative data analysis approach and a quantitative comparative analysis were followed to elicit themes from the interview transcripts and to map them to the elements of the preliminary 4D framework. RESULTS: A total of 302 codes were extracted from interview transcripts. After iterative and recursive comparison, classification, and mapping, 46 new indicators emerged; 24.8% (37/149) of the original indicators were deleted because of a lack of evidence support and another 28.2% (42/149) were merged. The validated 4D component framework consists of 116 indicators (82 facilitators and 34 barriers). The first component, data collection management, includes data collection protocols and quality assurance. It was measured by 41 indicators, decreased from the original 49% (73/149) to 35.3% (41/116). The second component, data collection environment, was measured by 37 indicators, increased from the original 13.4% (20/149) to 31.9% (37/116). It comprised leadership, training, funding, organizational policy, high-level management support, and collaboration among parallel organizations. The third component, data collection personnel, includes the perception of data collection, skills and competence, communication, and staffing patterns. There was no change in the proportion for data collection personnel (19.5% vs 19.0%), although the number of its indicators was reduced from 29 to 22. The fourth component, the data collection system, was measured using 16 indicators, with a slight decrease in percentage points from 18.1% (27/149) to 13.8% (16/116). It comprised functions, system integration, technical support, and data collection devices. CONCLUSIONS: This expert elicitation study validated and improved the 4D framework. The framework can be useful in developing a questionnaire survey instrument for measuring the quality of the public health data collection process after validation of psychometric properties and item reduction.

[Administrative data linkage and its usefulness in public health: the case of EcuadorVinculação de dados administrativos e sua utilização em saúde pública: o caso do Equador]. / Vinculación de datos administrativos y su utilidad en salud pública: el caso de Ecuador.

The objective of this article is to describe the characteristics of addressing the linkage of administrative databases and the uses of such linkages in public health research, and also to discuss the opportunities and challenges for implementation in Ecuador. The linkage of databases makes it possible to integrate a person's data that may be scattered across different subsectors such as health, education, justice, immigration, and social programs. It also facilitates research that can inform more efficient management of social and health programs and policies. The main advantages of using linked databases are: diversity of data, population coverage, stability over time, and lower cost in comparison to primary data collection. Despite the availability of tools to process, link, and analyze large data sets, there has been minimal use of this approach in Latin American countries. Ecuador is well positioned to implement this approach, due to compulsory use of a unique ID in health services delivery, which permits linkages with other national information systems. However, the country faces several cultural, technical, ethical, legal, and political challenges. To take advantage of its potential, Ecuador needs to develop a data governance strategy that includes standards for data access and data use, as well as mechanisms for data control and quality, greater investment in professional training in data use both within and beyond the health sector, and collaborations between government entities, universities, and civil society organizations.

Os objetivos deste artigo são descrever as características do método de vinculação de bancos de dados administrativos e sua utilização em pesquisa em saúde pública e examinar o potencial e os desafios para sua implementação no Equador. A vinculação de bancos de dados possibilita integrar dados de uma mesma pessoa dispersos em subsetores diversos como saúde, educação, justiça, imigração e programas sociais e realizar pesquisas para subsidiar a gestão mais eficiente de programas e políticas sociais e de saúde. Entre as principais vantagens de utilizar bancos de dados vinculados estão diversidade dos dados, cobertura populacional, estabilidade temporal e custo menor em comparação à coleta de dados primários. Apesar de existirem ferramentas para processar, vincular e analisar grandes conjuntos de dados, a utilização deste método é mínima nos países da América Latina. O Equador possui um grande potencial para beneficiar-se com este método devido à obrigatoriedade do uso de um identificador único na prestação de serviços de saúde, o que permite a vinculação com outros sistemas de informação nacionais, mas enfrenta uma série de desafios técnicos, éticos-legais, culturais e políticos. Para aproveitá-lo, o país precisa elaborar uma estratégia de governança de dados contendo um conjunto de normas para o acesso e a utilização simultâneos com mecanismos de controle e qualidade dos dados, maior investimento em formação profissional no uso dos dados dentro e fora da área da saúde e colaboração entre entidades governamentais, universidades e organizações da sociedade civil.

Health informatics publication trends in Saudi Arabia: a bibliometric analysis over the last twenty-four years.

OBJECTIVE: Understanding health informatics (HI) publication trends in Saudi Arabia may serve as a framework for future research efforts and contribute toward meeting national "e-Health" goals. The authors' intention was to understand the state of the HI field in Saudi Arabia by exploring publication trends and their alignment with national goals. METHODS: A scoping review was performed to identify HI publications from Saudi Arabia in PubMed, Embase, and Web of Science. We analyzed publication trends based on topics, keywords, and how they align with the Ministry of Health's (MOH's) "digital health journey" framework. RESULTS: The total number of publications included was 242. We found 1 (0.4%) publication in 1995-1999, 11 (4.5%) publications in 2000-2009, and 230 (95.0%) publications in 2010-2019. We categorized publications into 3 main HI fields and 4 subfields: 73.1% (n=177) of publications were in clinical informatics (85.1%, n=151 medical informatics; 5.6%, n=10 pharmacy informatics; 6.8%, n=12 nursing informatics; 2.3%, n=4 dental informatics); 22.3% (n=54) were in consumer health informatics; and 4.5% (n=11) were in public health informatics. The most common keyword was "medical informatics" (21.5%, n=52). MOH framework-based analysis showed that most publications were categorized as "digitally enabled care" and "digital health foundations." CONCLUSIONS: The years of 2000-2009 may be seen as an infancy stage of the HI field in Saudi Arabia. Exploring how the Saudi Arabian MOH's e-Health initiatives may influence research is valuable for advancing the field. Data exchange and interoperability, artificial intelligence, and intelligent health enterprises might be future research directions in Saudi Arabia.

Paper Versus Digital Data Collection Methods for Road Safety Observations: Comparative Efficiency Analysis of Cost, Timeliness, Reliability, and Results.

BACKGROUND: Roadside observational studies play a fundamental role in designing evidence-informed strategies to address the pressing global health problem of road traffic injuries. Paper-based data collection has been the standard method for such studies, although digital methods are gaining popularity in all types of primary data collection. OBJECTIVE: This study aims to understand the reliability, productivity, and efficiency of paper vs digital data collection based on three different road user behaviors: helmet use, seatbelt use, and speeding. It also aims to understand the cost and time efficiency of each method and to evaluate potential trade-offs among reliability, productivity, and efficiency. METHODS: A total of 150 observational sessions were conducted simultaneously for each risk factor in Mumbai, India, across two rounds of data collection. We matched the simultaneous digital and paper observation periods by date, time, and location, and compared the reliability by subgroups and the productivity using Pearson correlations (r). We also conducted logistic regressions separately by method to understand how similar results of inferential analyses would be. The time to complete an observation and the time to obtain a complete dataset were also compared, as were the total costs in US dollars for fieldwork, data entry, management, and cleaning. RESULTS: Productivity was higher in paper than digital methods in each round for each risk factor. However, the sample sizes across both methods provided a precision of 0.7 percentage points or smaller. The gap between digital and paper data collection productivity narrowed across rounds, with correlations improving from r=0.27-0.49 to 0.89-0.96. Reliability in risk factor proportions was between 0.61 and 0.99, improving between the two rounds for each risk factor. The results of the logistic regressions were also largely comparable between the two methods. Differences in regression results were largely attributable to small sample sizes in some variable levels or random error in variables where the prevalence of the outcome was similar among variable levels. Although data collectors were able to complete an observation using paper more quickly, the digital dataset was available approximately 9 days sooner. Although fixed costs were higher for digital data collection, variable costs were much lower, resulting in a 7.73% (US $3011/38,947) lower overall cost. CONCLUSIONS: Our study did not face trade-offs among time efficiency, cost efficiency, statistical reliability, and descriptive comparability when deciding between digital and paper, as digital data collection proved equivalent or superior on these domains in the context of our project. As trade-offs among cost, timeliness, and comparability-and the relative importance of each-could be unique to every data collection project, researchers should carefully consider the questionnaire complexity, target sample size, implementation plan, cost and logistical constraints, and geographical contexts when making the decision between digital and paper.

The Application of Internet-Based Sources for Public Health Surveillance (Infoveillance): Systematic Review.

BACKGROUND: Public health surveillance is based on the continuous and systematic collection, analysis, and interpretation of data. This informs the development of early warning systems to monitor epidemics and documents the impact of intervention measures. The introduction of digital data sources, and specifically sources available on the internet, has impacted the field of public health surveillance. New opportunities enabled by the underlying availability and scale of internet-based sources (IBSs) have paved the way for novel approaches for disease surveillance, exploration of health communities, and the study of epidemic dynamics. This field and approach is also known as infodemiology or infoveillance. OBJECTIVE: This review aimed to assess research findings regarding the application of IBSs for public health surveillance (infodemiology or infoveillance). To achieve this, we have presented a comprehensive systematic literature review with a focus on these sources and their limitations, the diseases targeted, and commonly applied methods. METHODS: A systematic literature review was conducted targeting publications between 2012 and 2018 that leveraged IBSs for public health surveillance, outbreak forecasting, disease characterization, diagnosis prediction, content analysis, and health-topic identification. The search results were filtered according to previously defined inclusion and exclusion criteria. RESULTS: Spanning a total of 162 publications, we determined infectious diseases to be the preferred case study (108/162, 66.7%). Of the eight categories of IBSs (search queries, social media, news, discussion forums, websites, web encyclopedia, and online obituaries), search queries and social media were applied in 95.1% (154/162) of the reviewed publications. We also identified limitations in representativeness and biased user age groups, as well as high susceptibility to media events by search queries, social media, and web encyclopedias. CONCLUSIONS: IBSs are a valuable proxy to study illnesses affecting the general population; however, it is important to characterize which diseases are best suited for the available sources; the literature shows that the level of engagement among online platforms can be a potential indicator. There is a necessity to understand the population's online behavior; in addition, the exploration of health information dissemination and its content is significantly unexplored. With this information, we can understand how the population communicates about illnesses online and, in the process, benefit public health.

The Adaptive Behavioral Components (ABC) Model for Planning Longitudinal Behavioral Technology-Based Health Interventions: A Theoretical Framework.

A growing number of interventions incorporate digital and social technologies (eg, social media, mobile phone apps, and wearable devices) into their design for behavior change. However, because of a number of factors, including changing trends in the use of technology over time, results on the efficacy of these interventions have been mixed. An updated framework is needed to help researchers better plan behavioral technology interventions by anticipating the needed resources and potential changes in trends that may affect interventions over time. Focusing on the domain of health interventions as a use case, we present the Adaptive Behavioral Components (ABC) model for technology-based behavioral interventions. ABC is composed of five components: basic behavior change; intervention, or problem-focused characteristics; population, social, and behavioral characteristics; individual-level and personality characteristics; and technology characteristics. ABC was designed with the goals of (1) guiding high-level development for digital technology-based interventions; (2) helping interventionists consider, plan for, and adapt to potential barriers that may arise during longitudinal interventions; and (3) providing a framework to potentially help increase the consistency of findings among digital technology intervention studies. We describe the planning of an HIV prevention intervention as a case study for how to implement ABC into intervention design. Using the ABC model to plan future interventions might help to improve the design of and adherence to longitudinal behavior change intervention protocols; allow these interventions to adapt, anticipate, and prepare for changes that may arise over time; and help to potentially improve intervention behavior change outcomes. Additional research is needed on the influence of each of ABC's components to help improve intervention design and implementation.