Evaluating the Role of Data Enrichment Approaches towards Rare Event Analysis in Manufacturing.

Rare events are occurrences that take place with a significantly lower frequency than more common, regular events. These events can be categorized into distinct categories, from frequently rare to extremely rare, based on factors like the distribution of data and significant differences in rarity levels. In manufacturing domains, predicting such events is particularly important, as they lead to unplanned downtime, a shortening of equipment lifespans, and high energy consumption. Usually, the rarity of events is inversely correlated with the maturity of a manufacturing industry. Typically, the rarity of events affects the multivariate data generated within a manufacturing process to be highly imbalanced, which leads to bias in predictive models. This paper evaluates the role of data enrichment techniques combined with supervised machine learning techniques for rare event detection and prediction. We use time series data augmentation and sampling to address the data scarcity, maintaining its patterns, and imputation techniques to handle null values. Evaluating 15 learning models, we find that data enrichment improves the F1 measure by up to 48% in rare event detection and prediction. Our empirical and ablation experiments provide novel insights, and we also investigate model interpretability.

RI2AP: Robust and Interpretable 2D Anomaly Prediction in Assembly Pipelines.

Predicting anomalies in manufacturing assembly lines is crucial for reducing time and labor costs and improving processes. For instance, in rocket assembly, premature part failures can lead to significant financial losses and labor inefficiencies. With the abundance of sensor data in the Industry 4.0 era, machine learning (ML) offers potential for early anomaly detection. However, current ML methods for anomaly prediction have limitations, with F1 measure scores of only 50% and 66% for prediction and detection, respectively. This is due to challenges like the rarity of anomalous events, scarcity of high-fidelity simulation data (actual data are expensive), and the complex relationships between anomalies not easily captured using traditional ML approaches. Specifically, these challenges relate to two dimensions of anomaly prediction: predicting when anomalies will occur and understanding the dependencies between them. This paper introduces a new method called Robust and Interpretable 2D Anomaly Prediction (RI2AP) designed to address both dimensions effectively. RI2AP is demonstrated on a rocket assembly simulation, showing up to a 30-point improvement in F1 measure compared to current ML methods. This highlights its potential to enhance automated anomaly prediction in manufacturing. Additionally, RI2AP includes a novel interpretation mechanism inspired by a causal-influence framework, providing domain experts with valuable insights into sensor readings and their impact on predictions. Finally, the RI2AP model was deployed in a real manufacturing setting for assembling rocket parts. Results and insights from this deployment demonstrate the promise of RI2AP for anomaly prediction in manufacturing assembly pipelines.

Towards Semantic Integration of Machine Vision Systems to Aid Manufacturing Event Understanding.

A manufacturing paradigm shift from conventional control pyramids to decentralized, service-oriented, and cyber-physical systems (CPSs) is taking place in today's 4th industrial revolution. Generally accepted roles and implementation recipes of cyber systems are expected to be standardized in the future of manufacturing industry. The authors intend to develop a novel CPS-enabled control architecture that accommodates: (1) intelligent information systems involving domain knowledge, empirical model, and simulation; (2) fast and secured industrial communication networks; (3) cognitive automation by rapid signal analytics and machine learning (ML) based feature extraction; (4) interoperability between machine and human. Semantic integration of process indicators is fundamental to the success of such implementation. This work proposes an automated semantic integration of data-intensive process signals that is deployable to industrial signal-based control loops. The proposed system rapidly infers manufacturing events from image-based data feeds, and hence triggers process control signals. Two image inference approaches are implemented: cloud-based ML model query and edge-end object shape detection. Depending on use cases and task requirements, these two approaches can be designated with different event detection tasks to provide a comprehensive system self-awareness. Coupled with conventional industrial sensor signals, machine vision system can rapidly understand manufacturing scenes, and feed extracted semantic information to a manufacturing ontology developed by either expert or ML-enabled cyber systems. Moreover, extracted signals are interpreted by Programmable Logical Controllers (PLCs) and field devices for cognitive automation towards fully autonomous industrial systems.

Global trends, local harms: availability of fentanyl-type drugs on the dark web and accidental overdoses in Ohio.

As America's opioid crisis has become an "epidemic of epidemics," Ohio has been identified as one of the high burden states regarding fentanyl-related overdose mortality. This study aims to examine changes in the availability of fentanyl, fentanyl analogs, and other non-pharmaceutical opioids on cryptomarkets and assess relationship with the trends in unintentional overdoses in Ohio to provide timely information for epidemiologic surveillance. Cryptomarket data were collected at two distinct periods of time: (1) Agora data covered June 2014-September 2015 and were obtained from Grams archive; (2) Dream Market data from March-April 2018 were extracted using a dedicated crawler. A Named Entity Recognition algorithm was developed to identify and categorize the type of fentanyl and other synthetic opioids advertised on cryptomarkets. Time-lagged correlations were used to assess the relationship between the fentanyl, fentanyl analog and other synthetic opioid-related ads from cryptomarkets and overdose data from the Cincinnati Fire Department Emergency Responses and Montgomery County Coroner's Office. Analysis from the cryptomarket data reveals increases in fentanyl-like drugs and changes in the types of fentanyl analogues and other synthetic opioids advertised in 2015 and 2018 with potent substances like carfentanil available during the second period. The time-lagged correlation was the largest when comparing Agora data to Cincinnati Emergency Responses 1 month later 0.84 (95% CI 0.45, 0.96). The time-lagged correlation between Agora data and Montgomery County drug overdoses was the largest when comparing synthetic opioid-related Agora ads to Montgomery County overdose deaths 7 months later 0.78 (95% CI 0.47, 0.92). Further investigations are required to establish the relationship between cryptomarket availability and unintentional overdose trends related to specific fentanyl analogs and/or other illicit synthetic opioids.

How Will the Internet of Things Enable Augmented Personalized Health?

The Internet of Things refers to network-enabled technologies, including mobile and wearable devices, which are capable of sensing and actuation as well as interaction and communication with other similar devices over the Internet. The IoT is profoundly redefining the way we create, consume, and share information. Ordinary citizens increasingly use these technologies to track their sleep, food intake, activity, vital signs, and other physiological statuses. This activity is complemented by IoT systems that continuously collect and process environment-related data that has a bearing on human health. This synergy has created an opportunity for a new generation of healthcare solutions.

Context-driven automatic subgraph creation for literature-based discovery.

BACKGROUND: Literature-based discovery (LBD) is characterized by uncovering hidden associations in non-interacting scientific literature. Prior approaches to LBD include use of: (1) domain expertise and structured background knowledge to manually filter and explore the literature, (2) distributional statistics and graph-theoretic measures to rank interesting connections, and (3) heuristics to help eliminate spurious connections. However, manual approaches to LBD are not scalable and purely distributional approaches may not be sufficient to obtain insights into the meaning of poorly understood associations. While several graph-based approaches have the potential to elucidate associations, their effectiveness has not been fully demonstrated. A considerable degree of a priori knowledge, heuristics, and manual filtering is still required. OBJECTIVES: In this paper we implement and evaluate a context-driven, automatic subgraph creation method that captures multifaceted complex associations between biomedical concepts to facilitate LBD. Given a pair of concepts, our method automatically generates a ranked list of subgraphs, which provide informative and potentially unknown associations between such concepts. METHODS: To generate subgraphs, the set of all MEDLINE articles that contain either of the two specified concepts (A, C) are first collected. Then binary relationships or assertions, which are automatically extracted from the MEDLINE articles, called semantic predications, are used to create a labeled directed predications graph. In this predications graph, a path is represented as a sequence of semantic predications. The hierarchical agglomerative clustering (HAC) algorithm is then applied to cluster paths that are bounded by the two concepts (A, C). HAC relies on implicit semantics captured through Medical Subject Heading (MeSH) descriptors, and explicit semantics from the MeSH hierarchy, for clustering. Paths that exceed a threshold of semantic relatedness are clustered into subgraphs based on their shared context. Finally, the automatically generated clusters are provided as a ranked list of subgraphs. RESULTS: The subgraphs generated using this approach facilitated the rediscovery of 8 out of 9 existing scientific discoveries. In particular, they directly (or indirectly) led to the recovery of several intermediates (or B-concepts) between A- and C-terms, while also providing insights into the meaning of the associations. Such meaning is derived from predicates between the concepts, as well as the provenance of the semantic predications in MEDLINE. Additionally, by generating subgraphs on different thematic dimensions (such as Cellular Activity, Pharmaceutical Treatment and Tissue Function), the approach may enable a broader understanding of the nature of complex associations between concepts. Finally, in a statistical evaluation to determine the interestingness of the subgraphs, it was observed that an arbitrary association is mentioned in only approximately 4 articles in MEDLINE on average. CONCLUSION: These results suggest that leveraging the implicit and explicit semantics provided by manually assigned MeSH descriptors is an effective representation for capturing the underlying context of complex associations, along multiple thematic dimensions in LBD situations.

"Sub is a weird drug:" A web-based study of lay attitudes about use of buprenorphine to self-treat opioid withdrawal symptoms.

BACKGROUND: Illicit use of buprenorphine has increased in the U.S., but our understanding of its use remains limited. This study aims to explore Web-forum discussions about the use of buprenorphine to self-treat opioid withdrawal symptoms. METHODS: PREDOSE, a novel Semantic Web platform, was used to extract relevant posts from a Web-forum that allows free discussions on illicit drugs. First, we extract information about the total number of buprenorphine-related posts per year between 2005 and 2013. Second, PREDOSE was used to identify all posts that potentially contained discussions about buprenorphine and opioid withdrawal. A total number of 1,217 posts that contained these terms were extracted and entered into NVivo data base. A random sample of 404 (33%) posts was selected and content analyzed. RESULTS: Buprenorphine-related posts increased over time, peaking in 2011. The posts were about equally divided between those that expressed positive and negative views about the effectiveness of buprenorphine in relieving withdrawal symptoms. Web-forum participants emphasized that buprenorphine's effectiveness may become compromised because of the "size of a person habit," and/or when users repeatedly switch back and forth between buprenorphine and other illicit opioids. Most posts reported use of significantly lower amounts of buprenorphine (≤2 mg) than doses used in standard treatment. Concomitant use of other psychoactive substances was also commonly reported, which may present significant health risks. CONCLUSIONS: Our findings highlight the usefulness of Web-based data in drug abuse research and add new information about lay beliefs about buprenorphine that may help inform prevention and policy measures.

Comparative analysis of online health queries originating from personal computers and smart devices on a consumer health information portal.

BACKGROUND: The number of people using the Internet and mobile/smart devices for health information seeking is increasing rapidly. Although the user experience for online health information seeking varies with the device used, for example, smart devices (SDs) like smartphones/tablets versus personal computers (PCs) like desktops/laptops, very few studies have investigated how online health information seeking behavior (OHISB) may differ by device. OBJECTIVE: The objective of this study is to examine differences in OHISB between PCs and SDs through a comparative analysis of large-scale health search queries submitted through Web search engines from both types of devices. METHODS: Using the Web analytics tool, IBM NetInsight OnDemand, and based on the type of devices used (PCs or SDs), we obtained the most frequent health search queries between June 2011 and May 2013 that were submitted on Web search engines and directed users to the Mayo Clinic's consumer health information website. We performed analyses on "Queries with considering repetition counts (QwR)" and "Queries without considering repetition counts (QwoR)". The dataset contains (1) 2.74 million and 3.94 million QwoR, respectively for PCs and SDs, and (2) more than 100 million QwR for both PCs and SDs. We analyzed structural properties of the queries (length of the search queries, usage of query operators and special characters in health queries), types of search queries (keyword-based, wh-questions, yes/no questions), categorization of the queries based on health categories and information mentioned in the queries (gender, age-groups, temporal references), misspellings in the health queries, and the linguistic structure of the health queries. RESULTS: Query strings used for health information searching via PCs and SDs differ by almost 50%. The most searched health categories are "Symptoms" (1 in 3 search queries), "Causes", and "Treatments & Drugs". The distribution of search queries for different health categories differs with the device used for the search. Health queries tend to be longer and more specific than general search queries. Health queries from SDs are longer and have slightly fewer spelling mistakes than those from PCs. Users specify words related to women and children more often than that of men and any other age group. Most of the health queries are formulated using keywords; the second-most common are wh- and yes/no questions. Users ask more health questions using SDs than PCs. Almost all health queries have at least one noun and health queries from SDs are more descriptive than those from PCs. CONCLUSIONS: This study is a large-scale comparative analysis of health search queries to understand the effects of device type (PCs vs. SDs) used on OHISB. The study indicates that the device used for online health information search plays an important role in shaping how health information searches by consumers and patients are executed.

A Hybrid Approach to Finding Relevant Social Media Content for Complex Domain Specific Information Needs.

While contemporary semantic search systems offer to improve classical keyword-based search, they are not always adequate for complex domain specific information needs. The domain of prescription drug abuse, for example, requires knowledge of both ontological concepts and "intelligible constructs" not typically modeled in ontologies. These intelligible constructs convey essential information that include notions of intensity, frequency, interval, dosage and sentiments, which could be important to the holistic needs of the information seeker. In this paper, we present a hybrid approach to domain specific information retrieval that integrates ontology-driven query interpretation with synonym-based query expansion and domain specific rules, to facilitate search in social media on prescription drug abuse. Our framework is based on a context-free grammar (CFG) that defines the query language of constructs interpretable by the search system. The grammar provides two levels of semantic interpretation: 1) a top-level CFG that facilitates retrieval of diverse textual patterns, which belong to broad templates and 2) a low-level CFG that enables interpretation of specific expressions belonging to such textual patterns. These low-level expressions occur as concepts from four different categories of data: 1) ontological concepts, 2) concepts in lexicons (such as emotions and sentiments), 3) concepts in lexicons with only partial ontology representation, called lexico-ontology concepts (such as side effects and routes of administration (ROA)), and 4) domain specific expressions (such as date, time, interval, frequency and dosage) derived solely through rules. Our approach is embodied in a novel Semantic Web platform called PREDOSE, which provides search support for complex domain specific information needs in prescription drug abuse epidemiology. When applied to a corpus of over 1 million drug abuse-related web forum posts, our search framework proved effective in retrieving relevant documents when compared with three existing search systems.

Detecting Substance Use Disorder Using Social Media Data and the Dark Web: Time- and Knowledge-Aware Study.

Background: Opioid and substance misuse has become a widespread problem in the United States, leading to the "opioid crisis." The relationship between substance misuse and mental health has been extensively studied, with one possible relationship being that substance misuse causes poor mental health. However, the lack of evidence on the relationship has resulted in opioids being largely inaccessible through legal means. objectives: This study aims to analyze social media posts related to substance use and opioids being sold through cryptomarket listings. The study aims to use state-of-the-art deep learning models to generate sentiment and emotion from social media posts to understand users' perceptions of social media. The study also aims to investigate questions such as which synthetic opioids people are optimistic, neutral, or negative about; what kind of drugs induced fear and sorrow; what kind of drugs people love or are thankful about; which drugs people think negatively about; and which opioids cause little to no sentimental reaction. Methods: The study used the drug abuse ontology and state-of-the-art deep learning models, including knowledge-aware Bidirectional Encoder Representations From Transformers-based models, to generate sentiment and emotion from social media posts related to substance use and opioids being sold through cryptomarket listings. The study crawled cryptomarket data and extracted posts for fentanyl, fentanyl analogs, and other novel synthetic opioids. The study performed topic analysis associated with the generated sentiments and emotions to understand which topics correlate with people's responses to various drugs. Additionally, the study analyzed time-aware neural models built on these features while considering historical sentiment and emotional activity of posts related to a drug. Results: The study found that the most effective model performed well (statistically significant, with a macro-F1-score of 82.12 and recall of 83.58) in identifying substance use disorder. The study also found that there were varying levels of sentiment and emotion associated with different synthetic opioids, with some drugs eliciting more positive or negative responses than others. The study identified topics that correlated with people's responses to various drugs, such as pain relief, addiction, and withdrawal symptoms. Conclusions: The study provides insight into users' perceptions of synthetic opioids based on sentiment and emotion expressed in social media posts. The study's findings can be used to inform interventions and policies aimed at reducing substance misuse and addressing the opioid crisis. The study demonstrates the potential of deep learning models for analyzing social media data to gain insights into public health issues.

A Cross Attention Approach to Diagnostic Explainability Using Clinical Practice Guidelines for Depression.

The lack of explainability in using relevant clinical knowledge hinders the adoption of artificial intelligence-powered analysis of unstructured clinical dialogue. A wealth of relevant, untapped Mental Health (MH) data is available in online communities, providing the opportunity to address the explainability problem with substantial potential impact as a screening tool for both online and offline applications. Inspired by how clinicians rely on their expertise when interacting with patients, we leverage relevant clinical knowledge to classify and explain depression-related data, reducing manual review time and engendering trust. We developed a method to enhance attention in contemporary transformer models and generate explanations for classifications that are understandable by mental health practitioners (MHPs) by incorporating external clinical knowledge. We propose a domain-general architecture called ProcesS knowledgeinfused cross ATtention (PSAT) that incorporates clinical practice guidelines (CPG) when computing attention. We transform a CPG resource focused on depression, such as the Patient Health Questionnaire (e.g. PHQ-9) and related questions, into a machine-readable ontology using SNOMED-CT. With this resource, PSAT enhances the ability of models like GPT-3.5 to generate application-relevant explanations. Evaluation of four expert-curated datasets related to depression demonstrates PSAT's applicationrelevant explanations. PSAT surpasses the performance of twelve baseline models and can provide explanations where other baselines fall short.

PREDOSE: a semantic web platform for drug abuse epidemiology using social media.

OBJECTIVES: The role of social media in biomedical knowledge mining, including clinical, medical and healthcare informatics, prescription drug abuse epidemiology and drug pharmacology, has become increasingly significant in recent years. Social media offers opportunities for people to share opinions and experiences freely in online communities, which may contribute information beyond the knowledge of domain professionals. This paper describes the development of a novel semantic web platform called PREDOSE (PREscription Drug abuse Online Surveillance and Epidemiology), which is designed to facilitate the epidemiologic study of prescription (and related) drug abuse practices using social media. PREDOSE uses web forum posts and domain knowledge, modeled in a manually created Drug Abuse Ontology (DAO--pronounced dow), to facilitate the extraction of semantic information from User Generated Content (UGC), through combination of lexical, pattern-based and semantics-based techniques. In a previous study, PREDOSE was used to obtain the datasets from which new knowledge in drug abuse research was derived. Here, we report on various platform enhancements, including an updated DAO, new components for relationship and triple extraction, and tools for content analysis, trend detection and emerging patterns exploration, which enhance the capabilities of the PREDOSE platform. Given these enhancements, PREDOSE is now more equipped to impact drug abuse research by alleviating traditional labor-intensive content analysis tasks. METHODS: Using custom web crawlers that scrape UGC from publicly available web forums, PREDOSE first automates the collection of web-based social media content for subsequent semantic annotation. The annotation scheme is modeled in the DAO, and includes domain specific knowledge such as prescription (and related) drugs, methods of preparation, side effects, and routes of administration. The DAO is also used to help recognize three types of data, namely: (1) entities, (2) relationships and (3) triples. PREDOSE then uses a combination of lexical and semantic-based techniques to extract entities and relationships from the scraped content, and a top-down approach for triple extraction that uses patterns expressed in the DAO. In addition, PREDOSE uses publicly available lexicons to identify initial sentiment expressions in text, and then a probabilistic optimization algorithm (from related research) to extract the final sentiment expressions. Together, these techniques enable the capture of fine-grained semantic information, which facilitate search, trend analysis and overall content analysis using social media on prescription drug abuse. Moreover, extracted data are also made available to domain experts for the creation of training and test sets for use in evaluation and refinements in information extraction techniques. RESULTS: A recent evaluation of the information extraction techniques applied in the PREDOSE platform indicates 85% precision and 72% recall in entity identification, on a manually created gold standard dataset. In another study, PREDOSE achieved 36% precision in relationship identification and 33% precision in triple extraction, through manual evaluation by domain experts. Given the complexity of the relationship and triple extraction tasks and the abstruse nature of social media texts, we interpret these as favorable initial results. Extracted semantic information is currently in use in an online discovery support system, by prescription drug abuse researchers at the Center for Interventions, Treatment and Addictions Research (CITAR) at Wright State University. CONCLUSION: A comprehensive platform for entity, relationship, triple and sentiment extraction from such abstruse texts has never been developed for drug abuse research. PREDOSE has already demonstrated the importance of mining social media by providing data from which new findings in drug abuse research were uncovered. Given the recent platform enhancements, including the refined DAO, components for relationship and triple extraction, and tools for content, trend and emerging pattern analysis, it is expected that PREDOSE will play a significant role in advancing drug abuse epidemiology in future.

A graph-based recovery and decomposition of Swanson's hypothesis using semantic predications.

OBJECTIVES: This paper presents a methodology for recovering and decomposing Swanson's Raynaud Syndrome-Fish Oil hypothesis semi-automatically. The methodology leverages the semantics of assertions extracted from biomedical literature (called semantic predications) along with structured background knowledge and graph-based algorithms to semi-automatically capture the informative associations originally discovered manually by Swanson. Demonstrating that Swanson's manually intensive techniques can be undertaken semi-automatically, paves the way for fully automatic semantics-based hypothesis generation from scientific literature. METHODS: Semantic predications obtained from biomedical literature allow the construction of labeled directed graphs which contain various associations among concepts from the literature. By aggregating such associations into informative subgraphs, some of the relevant details originally articulated by Swanson have been uncovered. However, by leveraging background knowledge to bridge important knowledge gaps in the literature, a methodology for semi-automatically capturing the detailed associations originally explicated in natural language by Swanson, has been developed. RESULTS: Our methodology not only recovered the three associations commonly recognized as Swanson's hypothesis, but also decomposed them into an additional 16 detailed associations, formulated as chains of semantic predications. Altogether, 14 out of the 19 associations that can be attributed to Swanson were retrieved using our approach. To the best of our knowledge, such an in-depth recovery and decomposition of Swanson's hypothesis has never been attempted. CONCLUSION: In this work therefore, we presented a methodology to semi-automatically recover and decompose Swanson's RS-DFO hypothesis using semantic representations and graph algorithms. Our methodology provides new insights into potential prerequisites for semantics-driven Literature-Based Discovery (LBD). Based on our observations, three critical aspects of LBD include: (1) the need for more expressive representations beyond Swanson's ABC model; (2) an ability to accurately extract semantic information from text; and (3) the semantic integration of scientific literature and structured background knowledge.

ECG Recordings as Predictors of Very Early Autism Likelihood: A Machine Learning Approach.

In recent years, there has been a rise in the prevalence of autism spectrum disorder (ASD). The diagnosis of ASD requires behavioral observation and standardized testing completed by highly trained experts. Early intervention for ASD can begin as early as 1-2 years of age, but ASD diagnoses are not typically made until ages 2-5 years, thus delaying the start of intervention. There is an urgent need for non-invasive biomarkers to detect ASD in infancy. While previous research using physiological recordings has focused on brain-based biomarkers of ASD, this study investigated the potential of electrocardiogram (ECG) recordings as an ASD biomarker in 3-6-month-old infants. We recorded the heart activity of infants at typical and elevated familial likelihood for ASD during naturalistic interactions with objects and caregivers. After obtaining the ECG signals, features such as heart rate variability (HRV) and sympathetic and parasympathetic activities were extracted. Then we evaluated the effectiveness of multiple machine learning classifiers for classifying ASD likelihood. Our findings support our hypothesis that infant ECG signals contain important information about ASD familial likelihood. Amongthe various machine learning algorithms tested, KNN performed best according to sensitivity (0.70 ± 0.117), F1-score (0.689 ± 0.124), precision (0.717 ± 0.128), accuracy (0.70 ± 0.117, p-value = 0.02), and ROC (0.686 ± 0.122, p-value = 0.06). These results suggest that ECG signals contain relevant information about the likelihood of an infant developing ASD. Future studies should consider the potential of information contained in ECG, and other indices of autonomic control, for the development of biomarkers of ASD in infancy.

Ki-Cook: clustering multimodal cooking representations through knowledge-infused learning.

Cross-modal recipe retrieval has gained prominence due to its ability to retrieve a text representation given an image representation and vice versa. Clustering these recipe representations based on similarity is essential to retrieve relevant information about unknown food images. Existing studies cluster similar recipe representations in the latent space based on class names. Due to inter-class similarity and intraclass variation, associating a recipe with a class name does not provide sufficient knowledge about recipes to determine similarity. However, recipe title, ingredients, and cooking actions provide detailed knowledge about recipes and are a better determinant of similar recipes. In this study, we utilized this additional knowledge of recipes, such as ingredients and recipe title, to identify similar recipes, emphasizing attention especially on rare ingredients. To incorporate this knowledge, we propose a knowledge-infused multimodal cooking representation learning network, Ki-Cook, built on the procedural attribute of the cooking process. To the best of our knowledge, this is the first study to adopt a comprehensive recipe similarity determinant to identify and cluster similar recipe representations. The proposed network also incorporates ingredient images to learn multimodal cooking representation. Since the motivation for clustering similar recipes is to retrieve relevant information for an unknown food image, we evaluated the ingredient retrieval task. We performed an empirical analysis to establish that our proposed model improves the Coverage of Ground Truth by 12% and the Intersection Over Union by 10% compared to the baseline models. On average, the representations learned by our model contain an additional 15.33% of rare ingredients compared to the baseline models. Owing to this difference, our qualitative evaluation shows a 39% improvement in clustering similar recipes in the latent space compared to the baseline models, with an inter-annotator agreement of the Fleiss kappa score of 0.35.

Perceptions of COVID-19 and the Use of Health Information Technology Among People Who Are Uninsured: Multimethod Survey Study.

BACKGROUND: As of May 2023, the novel SARS-CoV-2 has claimed nearly 7 million lives globally and >1.1 million lives in the United States. Low-income populations are often disproportionately affected by risk factors such as lifestyle, employment, and limited health literacy. These populations may lack the knowledge of appropriate infection precautions or have reduced access to care during illness, particularly in countries without universal health care. OBJECTIVE: We aimed to explore the perceptions and experiences of COVID-19, including symptoms and risk factors among uninsured individuals seeking care at a free medical clinic, and to obtain respondents' perceptions of and suggestions for adapting a mobile health (mHealth) app to an uninsured population known to have low health literacy. METHODS: We conducted a prospective multimethod survey study with a convenience sample of uninsured adults seeking care at 3 free clinics in the United States. Respondents were questioned about their risk for and awareness of COVID-19 symptoms, COVID-19 testing, current technology use, and the use of technology to facilitate their health regarding COVID-19. Data were analyzed using descriptive statistics (eg, frequencies and mean differences). In addition, a small subset of respondents from one of the clinics (n=10) participated in interviews to provide feedback about the design of a COVID-19 web-based smartphone (mHealth) app. RESULTS: The survey respondents (N=240) were 53.8% (n=129) female, were primarily White (n=113, 47.1%), and had a mean age of 50.0 (SD 11.67; range 19-72) years. Most respondents (162/222, 73%) did not think that they were at risk for COVID-19. Although respondents reported only moderate confidence in their knowledge of the short- and long-term symptoms of COVID-19, their knowledge of the symptoms aligned well with reports published by the Centers for Disease Control and Prevention of the most common acute (590/610, 96.7%) and long-term (217/271, 80.1%) symptoms. Most respondents (159/224, 71%) reported an interest in using the mHealth app to gain additional information regarding COVID-19 and available community resources. Respondents who were interviewed provided suggestions to improve the mHealth app but had overall positive perceptions about the potential usefulness and usability of the app. CONCLUSIONS: It was encouraging that the knowledge of COVID-19 symptoms aligned well with the reports published by the Centers for Disease Control and Prevention and that respondents were enthusiastic about using an mHealth app to monitor symptoms. However, it was concerning that most respondents did not think they were at a risk of contracting COVID-19.

ProKnow: Process knowledge for safety constrained and explainable question generation for mental health diagnostic assistance.

Virtual Mental Health Assistants (VMHAs) are utilized in health care to provide patient services such as counseling and suggestive care. They are not used for patient diagnostic assistance because they cannot adhere to safety constraints and specialized clinical process knowledge (ProKnow) used to obtain clinical diagnoses. In this work, we define ProKnow as an ordered set of information that maps to evidence-based guidelines or categories of conceptual understanding to experts in a domain. We also introduce a new dataset of diagnostic conversations guided by safety constraints and ProKnow that healthcare professionals use (ProKnow-data). We develop a method for natural language question generation (NLG) that collects diagnostic information from the patient interactively (ProKnow-algo). We demonstrate the limitations of using state-of-the-art large-scale language models (LMs) on this dataset. ProKnow-algo incorporates the process knowledge through explicitly modeling safety, knowledge capture, and explainability. As computational metrics for evaluation do not directly translate to clinical settings, we involve expert clinicians in designing evaluation metrics that test four properties: safety, logical coherence, and knowledge capture for explainability while minimizing the standard cross entropy loss to preserve distribution semantics-based similarity to the ground truth. LMs with ProKnow-algo generated 89% safer questions in the depression and anxiety domain (tested property: safety). Further, without ProKnow-algo generations question did not adhere to clinical process knowledge in ProKnow-data (tested property: knowledge capture). In comparison, ProKnow-algo-based generations yield a 96% reduction in our metrics to measure knowledge capture. The explainability of the generated question is assessed by computing similarity with concepts in depression and anxiety knowledge bases. Overall, irrespective of the type of LMs, ProKnow-algo achieved an averaged 82% improvement over simple pre-trained LMs on safety, explainability, and process-guided question generation. For reproducibility, we will make ProKnow-data and the code repository of ProKnow-algo publicly available upon acceptance.

Drug Abuse Ontology to Harness Web-Based Data for Substance Use Epidemiology Research: Ontology Development Study.

BACKGROUND: Web-based resources and social media platforms play an increasingly important role in health-related knowledge and experience sharing. There is a growing interest in the use of these novel data sources for epidemiological surveillance of substance use behaviors and trends. OBJECTIVE: The key aims were to describe the development and application of the drug abuse ontology (DAO) as a framework for analyzing web-based and social media data to inform public health and substance use research in the following areas: determining user knowledge, attitudes, and behaviors related to nonmedical use of buprenorphine and illicitly manufactured opioids through the analysis of web forum data Prescription Drug Abuse Online Surveillance; analyzing patterns and trends of cannabis product use in the context of evolving cannabis legalization policies in the United States through analysis of Twitter and web forum data (eDrugTrends); assessing trends in the availability of novel synthetic opioids through the analysis of cryptomarket data (eDarkTrends); and analyzing COVID-19 pandemic trends in social media data related to 13 states in the United States as per Mental Health America reports. METHODS: The domain and scope of the DAO were defined using competency questions from popular ontology methodology (101 ontology development). The 101 method includes determining the domain and scope of ontology, reusing existing knowledge, enumerating important terms in ontology, defining the classes, their properties and creating instances of the classes. The quality of the ontology was evaluated using a set of tools and best practices recognized by the semantic web community and the artificial intelligence community that engage in natural language processing. RESULTS: The current version of the DAO comprises 315 classes, 31 relationships, and 814 instances among the classes. The ontology is flexible and can easily accommodate new concepts. The integration of the ontology with machine learning algorithms dramatically decreased the false alarm rate by adding external knowledge to the machine learning process. The ontology is recurrently updated to capture evolving concepts in different contexts and applied to analyze data related to social media and dark web marketplaces. CONCLUSIONS: The DAO provides a powerful framework and a useful resource that can be expanded and adapted to a wide range of substance use and mental health domains to help advance big data analytics of web-based data for substance use epidemiology research.