Use of Large Language Models to Assess the Likelihood of Epidemics From the Content of Tweets: Infodemiology Study.

BACKGROUND: Previous work suggests that Google searches could be useful in identifying conjunctivitis epidemics. Content-based assessment of social media content may provide additional value in serving as early indicators of conjunctivitis and other systemic infectious diseases. OBJECTIVE: We investigated whether large language models, specifically GPT-3.5 and GPT-4 (OpenAI), can provide probabilistic assessments of whether social media posts about conjunctivitis could indicate a regional outbreak. METHODS: A total of 12,194 conjunctivitis-related tweets were obtained using a targeted Boolean search in multiple languages from India, Guam (United States), Martinique (France), the Philippines, American Samoa (United States), Fiji, Costa Rica, Haiti, and the Bahamas, covering the time frame from January 1, 2012, to March 13, 2023. By providing these tweets via prompts to GPT-3.5 and GPT-4, we obtained probabilistic assessments that were validated by 2 human raters. We then calculated Pearson correlations of these time series with tweet volume and the occurrence of known outbreaks in these 9 locations, with time series bootstrap used to compute CIs. RESULTS: Probabilistic assessments derived from GPT-3.5 showed correlations of 0.60 (95% CI 0.47-0.70) and 0.53 (95% CI 0.40-0.65) with the 2 human raters, with higher results for GPT-4. The weekly averages of GPT-3.5 probabilities showed substantial correlations with weekly tweet volume for 44% (4/9) of the countries, with correlations ranging from 0.10 (95% CI 0.0-0.29) to 0.53 (95% CI 0.39-0.89), with larger correlations for GPT-4. More modest correlations were found for correlation with known epidemics, with substantial correlation only in American Samoa (0.40, 95% CI 0.16-0.81). CONCLUSIONS: These findings suggest that GPT prompting can efficiently assess the content of social media posts and indicate possible disease outbreaks to a degree of accuracy comparable to that of humans. Furthermore, we found that automated content analysis of tweets is related to tweet volume for conjunctivitis-related posts in some locations and to the occurrence of actual epidemics. Future work may improve the sensitivity and specificity of these methods for disease outbreak detection.

Large language models as decision aids in neuro-oncology: a review of shared decision-making applications.

Shared decision-making (SDM) is crucial in neuro-oncology, fostering collaborations between patients and healthcare professionals to navigate treatment options. However, the complexity of neuro-oncological conditions and the cognitive and emotional burdens on patients present significant barriers to achieving effective SDM. This discussion explores the potential of large language models (LLMs) such as OpenAI's ChatGPT and Google's Bard to overcome these barriers, offering a means to enhance patient understanding and engagement in their care. LLMs, by providing accessible, personalized information, could support but not supplant the critical insights of healthcare professionals. The hypothesis suggests that patients, better informed through LLMs, may participate more actively in their treatment choices. Integrating LLMs into neuro-oncology requires navigating ethical considerations, including safeguarding patient data and ensuring informed consent, alongside the judicious use of AI technologies. Future efforts should focus on establishing ethical guidelines, adapting healthcare workflows, promoting patient-oriented research, and developing training programs for clinicians on the use of LLMs. Continuous evaluation of LLM applications will be vital to maintain their effectiveness and alignment with patient needs. Ultimately, this exploration contends that the thoughtful integration of LLMs into SDM processes could significantly enhance patient involvement and strengthen the patient-physician relationship in neuro-oncology care.

ChatGPT vs Google for Queries Related to Dementia and Other Cognitive Decline: Comparison of Results.

BACKGROUND: People living with dementia or other cognitive decline and their caregivers (PLWD) increasingly rely on the web to find information about their condition and available resources and services. The recent advancements in large language models (LLMs), such as ChatGPT, provide a new alternative to the more traditional web search engines, such as Google. OBJECTIVE: This study compared the quality of the results of ChatGPT and Google for a collection of PLWD-related queries. METHODS: A set of 30 informational and 30 service delivery (transactional) PLWD-related queries were selected and submitted to both Google and ChatGPT. Three domain experts assessed the results for their currency of information, reliability of the source, objectivity, relevance to the query, and similarity of their response. The readability of the results was also analyzed. Interrater reliability coefficients were calculated for all outcomes. RESULTS: Google had superior currency and higher reliability. ChatGPT results were evaluated as more objective. ChatGPT had a significantly higher response relevance, while Google often drew upon sources that were referral services for dementia care or service providers themselves. The readability was low for both platforms, especially for ChatGPT (mean grade level 12.17, SD 1.94) compared to Google (mean grade level 9.86, SD 3.47). The similarity between the content of ChatGPT and Google responses was rated as high for 13 (21.7%) responses, medium for 16 (26.7%) responses, and low for 31 (51.6%) responses. CONCLUSIONS: Both Google and ChatGPT have strengths and weaknesses. ChatGPT rarely includes the source of a result. Google more often provides a date for and a known reliable source of the response compared to ChatGPT, whereas ChatGPT supplies more relevant responses to queries. The results of ChatGPT may be out of date and often do not specify a validity time stamp. Google sometimes returns results based on commercial entities. The readability scores for both indicate that responses are often not appropriate for persons with low health literacy skills. In the future, the addition of both the source and the date of health-related information and availability in other languages may increase the value of these platforms for both nonmedical and medical professionals.

Chatbots to Support People With Dementia and Their Caregivers: Systematic Review of Functions and Quality.

BACKGROUND: Over the past decade, there has been an increase in the use of information technologies to educate and support people with dementia and their family caregivers. At the same time, chatbot technologies have become increasingly popular for use by the public and have been identified as having benefits for health care delivery. However, little is known about how chatbot technologies may benefit people with dementia and their caregivers. OBJECTIVE: This study aims to identify the types of current commercially available chatbots that are designed for use by people with dementia and their caregivers and to assess their quality in terms of features and content. METHODS: Chatbots were identified through a systematic search on Google Play Store, Apple App Store, Alexa Skills, and the internet. An evidence-based assessment tool was used to evaluate the features and content of the identified apps. The assessment was conducted through interrater agreement among 4 separate reviewers. RESULTS: Of the 505 initial chatbots identified, 6 were included in the review. The chatbots assessed varied significantly in terms of content and scope. Although the chatbots were generally found to be easy to use, some limitations were noted regarding their performance and programmed content for dialog. CONCLUSIONS: Although chatbot technologies are well established and commonly used by the public, their development for people with dementia and their caregivers is in its infancy. Given the successful use of chatbots in other health care settings and for other applications, there are opportunities to integrate this technology into dementia care. However, more evidence-based chatbots that have undergone end user evaluation are needed to evaluate their potential to adequately educate and support these populations.

A new approach to capturing pain disclosure in daily life in-person and online.

This feasibility study employed a new approach to capturing pain disclosure in face-to-face and online interactions, using a newly developed tool. In Study 1, 13 rheumatoid arthritis and 52 breast cancer patients wore the Electronically Activated Recorder to acoustically sample participants' natural conversations. Study 2 obtained data from two publicly available online social networks: fibromyalgia (343,439 posts) and rheumatoid arthritis (12,430 posts). Pain disclosure, versus non-pain disclosure, posts had a greater number of replies, and greater engagement indexed by language style matching. These studies yielded novel, multimethod evidence of how pain disclosure unfolds in naturally occurring social contexts in everyday life.

Prediction of Medical Concepts in Electronic Health Records: Similar Patient Analysis.

BACKGROUND: Medicine 2.0-the adoption of Web 2.0 technologies such as social networks in health care-creates the need for apps that can find other patients with similar experiences and health conditions based on a patient's electronic health record (EHR). Concurrently, there is an increasing number of longitudinal EHR data sets with rich information, which are essential to fulfill this need. OBJECTIVE: This study aimed to evaluate the hypothesis that we can leverage similar EHRs to predict possible future medical concepts (eg, disorders) from a patient's EHR. METHODS: We represented patients' EHRs using time-based prefixes and suffixes, where each prefix or suffix is a set of medical concepts from a medical ontology. We compared the prefixes of other patients in the collection with the state of the current patient using various interpatient distance measures. The set of similar prefixes yields a set of suffixes, which we used to determine probable future concepts for the current patient's EHR. RESULTS: We evaluated our methods on the Multiparameter Intelligent Monitoring in Intensive Care II data set of patients, where we achieved precision up to 56.1% and recall up to 69.5%. For a limited set of clinically interesting concepts, specifically a set of procedures, we found that 86.9% (353/406) of the true-positives are clinically useful, that is, these procedures were actually performed later on the patient, and only 4.7% (19/406) of true-positives were completely irrelevant. CONCLUSIONS: These initial results indicate that predicting patients' future medical concepts is feasible. Effectively predicting medical concepts can have several applications, such as managing resources in a hospital.

Mental Health-Related Behaviors and Discussions Among Young Adults: Analysis and Classification.

BACKGROUND: There have been recurring reports of web-based harassment and abuse among adolescents and young adults through anonymous social networks. OBJECTIVE: This study aimed to explore discussions on the popular anonymous social network Yik Yak related to social and mental health messaging behaviors among college students, including cyberbullying, to provide insights into mental health behaviors on college campuses. METHODS: From April 6, 2016, to May 7, 2016, we collected anonymous conversations posted on Yik Yak at 19 universities in 4 different states and performed statistical analyses and text classification experiments on a subset of these messages. RESULTS: We found that prosocial messages were 5.23 times more prevalent than bullying messages. The frequency of cyberbullying messages was positively associated with messages seeking emotional help. We found significant geographic variation in the frequency of messages offering supportive vs bullying messages. Across campuses, bullying and political discussions were positively associated. We also achieved a balanced accuracy of over 0.75 for most messaging behaviors and topics with a support vector machine classifier. CONCLUSIONS: Our results show that messages containing data about students' mental health-related attitudes and behaviors are prevalent on anonymous social networks, suggesting that these data can be mined for real-time analysis. This information can be used in education and health care services to better engage with students, provide insight into conversations that lead to cyberbullying, and reach out to students who need support.

Classification of Health-Related Social Media Posts: Evaluation of Post Content-Classifier Models and Analysis of User Demographics.

BACKGROUND: The increasing volume of health-related social media activity, where users connect, collaborate, and engage, has increased the significance of analyzing how people use health-related social media. OBJECTIVE: The aim of this study was to classify the content (eg, posts that share experiences and seek support) of users who write health-related social media posts and study the effect of user demographics on post content. METHODS: We analyzed two different types of health-related social media: (1) health-related online forums-WebMD and DailyStrength-and (2) general online social networks-Twitter and Google+. We identified several categories of post content and built classifiers to automatically detect these categories. These classifiers were used to study the distribution of categories for various demographic groups. RESULTS: We achieved an accuracy of at least 84% and a balanced accuracy of at least 0.81 for half of the post content categories in our experiments. In addition, 70.04% (4741/6769) of posts by male WebMD users asked for advice, and male users' WebMD posts were more likely to ask for medical advice than female users' posts. The majority of posts on DailyStrength shared experiences, regardless of the gender, age group, or location of their authors. Furthermore, health-related posts on Twitter and Google+ were used to share experiences less frequently than posts on WebMD and DailyStrength. CONCLUSIONS: We studied and analyzed the content of health-related social media posts. Our results can guide health advocates and researchers to better target patient populations based on the application type. Given a research question or an outreach goal, our results can be used to choose the best online forums to answer the question or disseminate a message.

Health Effects Associated With Electronic Cigarette Use: Automated Mining of Online Forums.

BACKGROUND: Our previous infodemiological study was performed by manually mining health-effect data associated with electronic cigarettes (ECs) from online forums. Manual mining is time consuming and limits the number of posts that can be retrieved. OBJECTIVE: Our goal in this study was to automatically extract and analyze a large number (>41,000) of online forum posts related to the health effects associated with EC use between 2008 and 2015. METHODS: Data were annotated with medical concepts from the Unified Medical Language System using a modified version of the MetaMap tool. Of over 1.4 million posts, 41,216 were used to analyze symptoms (undiagnosed conditions) and disorders (physician-diagnosed terminology) associated with EC use. For each post, sentiment (positive, negative, and neutral) was also assigned. RESULTS: Symptom and disorder data were categorized into 12 organ systems or anatomical regions. Most posts on symptoms and disorders contained negative sentiment, and affected systems were similar across all years. Health effects were reported most often in the neurological, mouth and throat, and respiratory systems. The most frequently reported symptoms and disorders were headache (n=939), coughing (n=852), malaise (n=468), asthma (n=916), dehydration (n=803), and pharyngitis (n=565). In addition, users often reported linked symptoms (eg, coughing and headache). CONCLUSIONS: Online forums are a valuable repository of data that can be used to identify positive and negative health effects associated with EC use. By automating extraction of online information, we obtained more data than in our prior study, identified new symptoms and disorders associated with EC use, determined which systems are most frequently adversely affected, identified specific symptoms and disorders most commonly reported, and tracked health effects over 7 years.

Smartphone-Based Health Technologies for Dementia Care: Opportunities, Challenges, and Current Practices.

Most of the 5.4 million people affected by Alzheimer's disease and other forms of dementia (AD) are noninstitutionalized, receiving care by unpaid family caregivers and medically managed by a primary care provider (PCP). Health Information Technology has been recognized for its potential in improving efficiency and quality of AD care and support for AD caregivers. Simultaneously, smartphone technologies have become an increasingly common way to deliver physical and behavioral health care. However, little is known about how smartphone technologies have been used to support AD caregiving and care. This article highlights the current need for smartphone-based interventions for AD and systematically identified and appraised current smartphone apps targeting and available for AD caregivers. Findings indicate that individual available apps have limited functions (compared with the complex needs of caregivers) and little has been done to extend AD caregiving apps to Hispanic populations. Implications for research, practice, and policy are discussed.

The Potential of Information Technology to Navigate Caregiving Systems: Perspectives from Dementia Caregivers.

Technologies designed to support caregivers of adults with Alzheimer's disease and related dementias (AD/RD) have been developing at an increasingly rapid pace. However, little remains known about caregivers' perspectives on how technologies can and should help them navigate larger service systems they interact with to engage in caregiving. This study involved in-depth interviews and a beta test of an AD/RD caregiver app to learn more about how they currently use technologies and how potential technological features and functions can best meet their needs. Thematic findings suggest a conceptual model for designing AD/RD caregiver technologies. The findings suggest that eHealth and individual technologies may not fully meet the needs of caregivers as they navigate the larger systems within which they provide care. Findings highlight the need to develop technologies for caregivers that are effective, easy to use, and more widely disseminated - especially for caregivers from disadvantaged backgrounds.

Automatic Classification of Online Doctor Reviews: Evaluation of Text Classifier Algorithms.

BACKGROUND: An increasing number of doctor reviews are being generated by patients on the internet. These reviews address a diverse set of topics (features), including wait time, office staff, doctor's skills, and bedside manners. Most previous work on automatic analysis of Web-based customer reviews assumes that (1) product features are described unambiguously by a small number of keywords, for example, battery for phones and (2) the opinion for each feature has a positive or negative sentiment. However, in the domain of doctor reviews, this setting is too restrictive: a feature such as visit duration for doctor reviews may be expressed in many ways and does not necessarily have a positive or negative sentiment. OBJECTIVE: This study aimed to adapt existing and propose novel text classification methods on the domain of doctor reviews. These methods are evaluated on their accuracy to classify a diverse set of doctor review features. METHODS: We first manually examined a large number of reviews to extract a set of features that are frequently mentioned in the reviews. Then we proposed a new algorithm that goes beyond bag-of-words or deep learning classification techniques by leveraging natural language processing (NLP) tools. Specifically, our algorithm automatically extracts dependency tree patterns and uses them to classify review sentences. RESULTS: We evaluated several state-of-the-art text classification algorithms as well as our dependency tree-based classifier algorithm on a real-world doctor review dataset. We showed that methods using deep learning or NLP techniques tend to outperform traditional bag-of-words methods. In our experiments, the 2 best methods used NLP techniques; on average, our proposed classifier performed 2.19% better than an existing NLP-based method, but many of its predictions of specific opinions were incorrect. CONCLUSIONS: We conclude that it is feasible to classify doctor reviews. Automatically classifying these reviews would allow patients to easily search for doctors based on their personal preference criteria.

Correlating Ratings of Health Insurance Plans to Their Providers' Attributes.

BACKGROUND: There is a push towards quality measures in health care. As a consequence, the National Committee for Quality Assurance (NCQA) has been publishing insurance plan quality measures. OBJECTIVE: The objective of this study was to examine the relationship between insurance plan quality measures and the participating providers (doctors). METHODS: We collected and analyzed provider and insurance plan data from several online sources, including provider directories, provider referrals and awards, patient reviewing sites, and hospital rankings. The relationships between the provider attributes and the insurance plan quality measures were examined. RESULTS: Our analysis yielded several findings: (1) there is a moderate Pearson correlation (r=.376) between consumer satisfaction insurance plan scores and review ratings of the member providers, (2) referral frequency and provider awards are negligibly correlated to consumer satisfaction plan scores (correlations of r=.031 and r=.183, respectively), (3) there is weak positive correlation (r=.266) between the cost charged for the same procedures and consumer satisfaction plan scores, and (4) there is no significant correlation between member specialists' review ratings and specialty-specific insurance plan treatment scores for most specialties, except a surprising weak negative correlation for diabetes treatment (r=-.259). CONCLUSIONS: Our findings may be used by consumers to make informed choices about their insurance plans or by insurances to understand the relationship between patients' satisfaction and their network of providers.

Demographic-Based Content Analysis of Web-Based Health-Related Social Media.

BACKGROUND: An increasing number of patients from diverse demographic groups share and search for health-related information on Web-based social media. However, little is known about the content of the posted information with respect to the users' demographics. OBJECTIVE: The aims of this study were to analyze the content of Web-based health-related social media based on users' demographics to identify which health topics are discussed in which social media by which demographic groups and to help guide educational and research activities. METHODS: We analyze 3 different types of health-related social media: (1) general Web-based social networks Twitter and Google+; (2) drug review websites; and (3) health Web forums, with a total of about 6 million users and 20 million posts. We analyzed the content of these posts based on the demographic group of their authors, in terms of sentiment and emotion, top distinctive terms, and top medical concepts. RESULTS: The results of this study are: (1) Pregnancy is the dominant topic for female users in drug review websites and health Web forums, whereas for male users, it is cardiac problems, HIV, and back pain, but this is not the case for Twitter; (2) younger users (0-17 years) mainly talk about attention-deficit hyperactivity disorder (ADHD) and depression-related drugs, users aged 35-44 years discuss about multiple sclerosis (MS) drugs, and middle-aged users (45-64 years) talk about alcohol and smoking; (3) users from the Northeast United States talk about physical disorders, whereas users from the West United States talk about mental disorders and addictive behaviors; (4) Users with higher writing level express less anger in their posts. CONCLUSION: We studied the popular topics and the sentiment based on users' demographics in Web-based health-related social media. Our results provide valuable information, which can help create targeted and effective educational campaigns and guide experts to reach the right users on Web-based social chatter.

Provider attributes correlation analysis to their referral frequency and awards.

BACKGROUND: There has been a recent growth in health provider search portals, where patients specify filters-such as specialty or insurance-and providers are ranked by patient ratings or other attributes. Previous work has identified attributes associated with a provider's quality through user surveys. Other work supports that intuitive quality-indicating attributes are associated with a provider's quality. METHODS: We adopt a data-driven approach to study how quality indicators of providers are associated with a rich set of attributes including medical school, graduation year, procedures, fellowships, patient reviews, location, and technology usage. In this work, we only consider providers as individuals (e.g., general practitioners) and not organizations (e.g., hospitals). As quality indicators, we consider the referral frequency of a provider and a peer-nominated quality designation. We combined data from the Centers for Medicare and Medicaid Services (CMS) and several provider rating web sites to perform our analysis. RESULTS: Our data-driven analysis identified several attributes that correlate with and discriminate against referral volume and peer-nominated awards. In particular, our results consistently demonstrate that these attributes vary by locality and that the frequency of an attribute is more important than its value (e.g., the number of patient reviews or hospital affiliations are more important than the average review rating or the ranking of the hospital affiliations, respectively). We demonstrate that it is possible to build accurate classifiers for referral frequency and quality designation, with accuracies over 85 %. CONCLUSIONS: Our findings show that a one-size-fits-all approach to ranking providers is inadequate and that provider search portals should calibrate their ranking function based on location and specialty. Further, traditional filters of provider search portals should be reconsidered, and patients should be aware of existing pitfalls with these filters and educated on local factors that affect quality. These findings enable provider search portals to empower patients and to "load balance" patients between younger and older providers.

CareHeroes Web and Android™ Apps for Dementia Caregivers: A Feasibility Study.

The purpose of the current feasibility study was to examine the use, utility, and areas for refinement of a newly developed web-based and Android™ application (app) (i.e., CareHeroes) with multiple features to support individuals caring for loved ones with Alzheimer's disease or other forms of dementia (AD). The study was performed over an 11-week period with triads of AD caregivers, assigned home care case managers, and primary care providers (PCP). The study involved quantitative and qualitative methodologies. Eleven AD caregivers (seven daughters, two sons, and two spouses), six case managers, and five PCPs participated. Data demonstrate participants were mostly satisfied with the multiple features and ability to access and use CareHeroes. Barriers for use include concerns about time constraints and not being familiar with technology. Although the study findings are promising, a longer term study to evaluate the impact of the CareHeroes app is indicated.

A Study of the Demographics of Web-Based Health-Related Social Media Users.

BACKGROUND: The rapid spread of Web-based social media in recent years has impacted how patients share health-related information. However, little work has studied the demographics of these users. OBJECTIVE: Our aim was to study the demographics of users who participate in health-related Web-based social outlets to identify possible links to health care disparities. METHODS: We analyze and compare three different types of health-related social outlets: (1) general Web-based social networks, Twitter and Google+, (2) drug review websites, and (3) health Web forums. We focus on the following demographic attributes: age, gender, ethnicity, location, and writing level. We build and evaluate domain-specific classifiers to infer missing data where possible. The estimated demographic statistics are compared against various baselines, such as Internet and social networks usage of the population. RESULTS: We found that (1) drug review websites and health Web forums are dominated by female users, (2) the participants of health-related social outlets are generally older with the exception of the 65+ years bracket, (3) blacks are underrepresented in health-related social networks, (4) users in areas with better access to health care participate more in Web-based health-related social outlets, and (5) the writing level of users in health-related social outlets is significantly lower than the reading level of the population. CONCLUSIONS: We identified interesting and actionable disparities in the participation of various demographic groups to various types of health-related social outlets. These disparities are significantly distinct from the disparities in Internet usage or general social outlets participation.

Pharmaceutical drugs chatter on Online Social Networks.

The ubiquity of Online Social Networks (OSNs) is creating new sources for healthcare information, particularly in the context of pharmaceutical drugs. We aimed to examine the impact of a given OSN's characteristics on the content of pharmaceutical drug discussions from that OSN. We compared the effect of four distinguishing characteristics from ten different OSNs on the content of their pharmaceutical drug discussions: (1) General versus Health OSN; (2) OSN moderation; (3) OSN registration requirements; and (4) OSNs with a question and answer format. The effects of these characteristics were measured both quantitatively and qualitatively. Our results show that an OSN's characteristics indeed affect the content of its discussions. Based on their information needs, healthcare providers may use our findings to pick the right OSNs or to advise patients regarding their needs. Our results may also guide the creation of new and more effective domain-specific health OSNs. Further, future researchers of online healthcare content in OSNs may find our results informative while choosing OSNs as data sources. We reported several findings about the impact of OSN characteristics on the content of pharmaceutical drug discussion, and synthesized these findings into actionable items for both healthcare providers and future researchers of healthcare discussions on OSNs. Future research on the impact of OSN characteristics could include user demographics, quality and safety of information, and efficacy of OSN usage.

Adding home health care to the discussion on health information technology policy.

The potential for health information technology to improve the efficiency and effectiveness of health care has resulted in several U.S. policy initiatives aimed at integrating health information technology into health care systems. However, home health care agencies have been excluded from incentive programs established through policies, raising concerns on the extent to which health information technology may be used to improve the quality of care for older adults with chronic illness and disabilities. This analysis examines the potential issues stemming from this exclusion and explores potential opportunities of integrating home health care into larger initiatives aimed at establishing health information technology systems for meaningful use.

Leveraging user query sessions to improve searching of medical literature.

Published reports about searching medical literature do not refer to leveraging the query context, as expressed by previous queries in a session. We aimed to assess novel strategies for context-aware searching, hypothesizing that this would be better than baseline. Building upon methods using term frequency-inverse document frequency, we added extensions such as a function incorporating search results and terms of previous queries, with higher weights for more recent queries. Among 60 medical students generating queries against the TREC 9 benchmark dataset, we assessed recall and mean average precision. For difficult queries, we achieved improvement (27%) in average precision over baseline. Improvements in recall were also seen. Our methods outperformed baseline by 4% to 14% on average. Furthermore, the effectiveness of context-aware search was greater for longer query sessions, which are typically more challenging. In conclusion, leveraging the previous queries in a session improved overall search quality with this biomedical database.