Impact of a Collective Intelligence Tailored Messaging System on Smoking Cessation: The Perspect Randomized Experiment.

BACKGROUND: Outside health care, content tailoring is driven algorithmically using machine learning compared to the rule-based approach used in current implementations of computer-tailored health communication (CTHC) systems. A special class of machine learning systems ("recommender systems") are used to select messages by combining the collective intelligence of their users (ie, the observed and inferred preferences of users as they interact with the system) and their user profiles. However, this approach has not been adequately tested for CTHC. OBJECTIVE: Our aim was to compare, in a randomized experiment, a standard, evidence-based, rule-based CTHC (standard CTHC) to a novel machine learning CTHC: Patient Experience Recommender System for Persuasive Communication Tailoring (PERSPeCT). We hypothesized that PERSPeCT will select messages of higher influence than our standard CTHC system. This standard CTHC was proven effective in motivating smoking cessation in a prior randomized trial of 900 smokers (OR 1.70, 95% CI 1.03-2.81). METHODS: PERSPeCT is an innovative hybrid machine learning recommender system that selects and sends motivational messages using algorithms that learn from message ratings from 846 previous participants (explicit feedback), and the prior explicit ratings of each individual participant. Current smokers (N=120) aged 18 years or older, English speaking, with Internet access were eligible to participate. These smokers were randomized to receive either PERSPeCT (intervention, n=74) or standard CTHC tailored messages (n=46). The study was conducted between October 2014 and January 2015. By randomization, we compared daily message ratings (mean of smoker ratings each day). At 30 days, we assessed the intervention's perceived influence, 30-day cessation, and changes in readiness to quit from baseline. RESULTS: The proportion of days when smokers agreed/strongly agreed (daily rating ≥4) that the messages influenced them to quit was significantly higher for PERSPeCT (73%, 23/30) than standard CTHC (44%, 14/30, P=.02). Among less educated smokers (n=49), this difference was even more pronounced for days strongly agree (intervention: 77%, 23/30; comparison: 23%, 7/30, P<.001). There was no significant difference in the frequency which PERSPeCT randomized smokers agreed or strongly agreed that the intervention influenced them to quit smoking (P=.07) and use nicotine replacement therapy (P=.09). Among those who completed follow-up, 36% (20/55) of PERSPeCT smokers and 32% (11/34) of the standard CTHC group stopped smoking for one day or longer (P=.70). CONCLUSIONS: Compared to standard CTHC with proven effectiveness, PERSPeCT outperformed in terms of influence ratings and resulted in similar cessation rates. CLINICALTRIAL: Clinicaltrials.gov NCT02200432; https://clinicaltrials.gov/ct2/show/NCT02200432 (Archived by WebCite at http://www.webcitation.org/6lEJY1KEd).

Endogenous and Exogenous Thyrotoxicosis and Risk of Incident Cognitive Disorders in Older Adults.

Importance: Thyroid hormone is among the most common prescriptions in the US and up to 20% may be overtreated. Endogenous hyperthyroidism may be a risk factor for dementia, but data are limited for iatrogenic thyrotoxicosis. Objective: To determine whether thyrotoxicosis, both endogenous and exogenous, is associated with increased risk of cognitive disorders. Design, Setting, and Participants: This cohort study performed a longitudinal time-varying analysis of electronic health records for patients receiving primary care in the Johns Hopkins Community Physicians Network between January 1, 2014, and May 6, 2023. Patients 65 years and older with at least 2 visits 30 days apart to their primary care physicians were eligible. None of the 65 931 included patients had a history of low thyrotropin (TSH) level or cognitive disorder diagnoses within 6 months of their first visit. Data analysis was performed from January 1 through August 5, 2023. Exposure: The exposure variable was a low TSH level, characterized based on the clinical context as due to endogenous thyrotoxicosis, exogenous thyrotoxicosis, or unknown cause, excluding those attributable to acute illness or other medical factors such as medications. Main Outcomes and Measures: The outcome measure was cognitive disorders, including mild cognitive impairment and all-cause dementia, to improve sensitivity and account for the underdiagnosis of dementia in primary care. Results: A total of 65 931 patients were included in the analysis (median [IQR] age at first visit, 68.0 [65.0-74.0] years; 37 208 [56%] were female; 46 106 [69.9%] were White). Patients exposed to thyrotoxicosis had cognitive disorder incidence of 11.0% (95% CI, 8.4%-14.2%) by age 75 years vs 6.4% (95% CI, 6.0%-6.8%) for those not exposed. After adjustment, all-cause thyrotoxicosis was significantly associated with risk of cognitive disorder diagnosis (adjusted hazard ratio, 1.39; 95% CI, 1.18-1.64; P < .001) across age groups. When stratified by cause and severity, exogenous thyrotoxicosis remained a significant risk factor (adjusted hazard ratio, 1.34; 95% CI, 1.10-1.63; P = .003) with point estimates suggestive of a dose response. Conclusions and Relevance: In this cohort study among patients 65 years and older, a low TSH level from either endogenous or exogenous thyrotoxicosis was associated with higher risk of incident cognitive disorder. Iatrogenic thyrotoxicosis is a common result of thyroid hormone therapy. With thyroid hormone among the most common prescriptions in the US, understanding the negative effects of overtreatment is critical to help guide prescribing practice.

Translating clinical notes into quantitative measures-a real-world observation on the response to cholinesterase inhibitors or selective serotonin reuptake inhibitors prescribed to outpatients with dementia using electronic medical records.

Objective: Cholinesterase inhibitors (CEIs) are prescribed for dementia to maintain or improve memory. Selective serotonin reuptake inhibitors (SSRIs) are also prescribed to manage psychiatric symptoms seen in dementia. What proportion of outpatients actually responds to these drugs is still unclear. Our objective was to investigate the responder rates of these medications in an outpatient setting using the electronic medical record (EMR). Methods: We used the Johns Hopkins EMR system to identify patients with dementia who were prescribed a CEI or SSRI for the first time between 2010 and 2021. Treatment effects were assessed through routinely documented clinical notes and free-text entries in which healthcare providers record clinical findings and impressions of patients. Responses were scored using a three-point Likert scale named the NOte-based evaluation method for Treatment Efficacy (NOTE) in addition to the Clinician's Interview-Based Impression of Change Plus caregiver input (CIBIC-plus), a seven-point Likert scale used in clinical trials. To validate NOTE, the relationships between NOTE and CIBIC-plus and between NOTE and change in MMSE (Mini-Mental State Examination) before and after medication were examined. Inter-rater reliability was evaluated using Krippendorff's alpha. The responder rates were calculated. Results: NOTE showed excellent inter-rater reliability and correlated well with CIBIC-plus and changes in MMSEs. Out of 115 CEI cases, 27.0% reported improvement and 34.8% reported stable symptoms in cognition; out of 225 SSRI cases, 69.3% reported an improvement in neuropsychiatric symptoms. Conclusion: NOTE showed high validity in measuring the pharmacotherapy effects based on unstructured clinical entries. Although our real-world observation included various types of dementia, the results were remarkably similar to what was reported in controlled clinical trials of Alzheimer's disease and its related neuropsychiatric symptoms.

A bias evaluation checklist for predictive models and its pilot application for 30-day hospital readmission models.

OBJECTIVE: Health care providers increasingly rely upon predictive algorithms when making important treatment decisions, however, evidence indicates that these tools can lead to inequitable outcomes across racial and socio-economic groups. In this study, we introduce a bias evaluation checklist that allows model developers and health care providers a means to systematically appraise a model's potential to introduce bias. MATERIALS AND METHODS: Our methods include developing a bias evaluation checklist, a scoping literature review to identify 30-day hospital readmission prediction models, and assessing the selected models using the checklist. RESULTS: We selected 4 models for evaluation: LACE, HOSPITAL, Johns Hopkins ACG, and HATRIX. Our assessment identified critical ways in which these algorithms can perpetuate health care inequalities. We found that LACE and HOSPITAL have the greatest potential for introducing bias, Johns Hopkins ACG has the most areas of uncertainty, and HATRIX has the fewest causes for concern. DISCUSSION: Our approach gives model developers and health care providers a practical and systematic method for evaluating bias in predictive models. Traditional bias identification methods do not elucidate sources of bias and are thus insufficient for mitigation efforts. With our checklist, bias can be addressed and eliminated before a model is fully developed or deployed. CONCLUSION: The potential for algorithms to perpetuate biased outcomes is not isolated to readmission prediction models; rather, we believe our results have implications for predictive models across health care. We offer a systematic method for evaluating potential bias with sufficient flexibility to be utilized across models and applications.

Graphical methods for understanding changes in states: Understanding medication use pathways.

OBJECTIVES: As epidemiological studies become longer and larger, the field needs novel graphical methods to visualize complex longitudinal data. The aim of this study was to present the Slinkyplot, a longitudinal crosstabulation, to illustrate patterns of antidepressant use in a large prospective cohort of older adults with mild cognitive impairment. METHODS: Data from the National Alzheimer's Coordinating Center are used to track switches between different states and types of antidepressant use. A Slinkyplot is populated with rows representing the state of medication use at each timepoint and columns representing the state at each subsequent visit. RESULTS: The constructed Slinkyplots display the common practice of switching on and off different antidepressants over time, with citalopram, sertraline, and bupropion most commonly used followed by switching to another SSRI or SNRI as second-line treatment. CONCLUSIONS: Slinkyplots are an innovative graphical means of visualizing complex patterns of transitions between different states over time for large longitudinal studies.

Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing.

Machine learning-based clinical decision support tools for sepsis create opportunities to identify at-risk patients and initiate treatments at early time points, which is critical for improving sepsis outcomes. In view of the increasing use of such systems, better understanding of how they are adopted and used by healthcare providers is needed. Here, we analyzed provider interactions with a sepsis early detection tool (Targeted Real-time Early Warning System), which was deployed at five hospitals over a 2-year period. Among 9,805 retrospectively identified sepsis cases, the early detection tool achieved high sensitivity (82% of sepsis cases were identified) and a high rate of adoption: 89% of all alerts by the system were evaluated by a physician or advanced practice provider and 38% of evaluated alerts were confirmed by a provider. Adjusting for patient presentation and severity, patients with sepsis whose alert was confirmed by a provider within 3 h had a 1.85-h (95% CI 1.66-2.00) reduction in median time to first antibiotic order compared to patients with sepsis whose alert was either dismissed, confirmed more than 3 h after the alert or never addressed in the system. Finally, we found that emergency department providers and providers who had previous interactions with an alert were more likely to interact with alerts, as well as to confirm alerts on retrospectively identified patients with sepsis. Beyond efforts to improve the performance of early warning systems, efforts to improve adoption are essential to their clinical impact and should focus on understanding providers' knowledge of, experience with and attitudes toward such systems.

Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis.

Early recognition and treatment of sepsis are linked to improved patient outcomes. Machine learning-based early warning systems may reduce the time to recognition, but few systems have undergone clinical evaluation. In this prospective, multi-site cohort study, we examined the association between patient outcomes and provider interaction with a deployed sepsis alert system called the Targeted Real-time Early Warning System (TREWS). During the study, 590,736 patients were monitored by TREWS across five hospitals. We focused our analysis on 6,877 patients with sepsis who were identified by the alert before initiation of antibiotic therapy. Adjusting for patient presentation and severity, patients in this group whose alert was confirmed by a provider within 3 h of the alert had a reduced in-hospital mortality rate (3.3%, confidence interval (CI) 1.7, 5.1%, adjusted absolute reduction, and 18.7%, CI 9.4, 27.0%, adjusted relative reduction), organ failure and length of stay compared with patients whose alert was not confirmed by a provider within 3 h. Improvements in mortality rate (4.5%, CI 0.8, 8.3%, adjusted absolute reduction) and organ failure were larger among those patients who were additionally flagged as high risk. Our findings indicate that early warning systems have the potential to identify sepsis patients early and improve patient outcomes and that sepsis patients who would benefit the most from early treatment can be identified and prioritized at the time of the alert.

Deep Phenotyping of Parkinson's Disease.

Phenotype is the set of observable traits of an organism or condition. While advances in genetics, imaging, and molecular biology have improved our understanding of the underlying biology of Parkinson's disease (PD), clinical phenotyping of PD still relies primarily on history and physical examination. These subjective, episodic, categorical assessments are valuable for diagnosis and care but have left gaps in our understanding of the PD phenotype. Sensors can provide objective, continuous, real-world data about the PD clinical phenotype, increase our knowledge of its pathology, enhance evaluation of therapies, and ultimately, improve patient care. In this paper, we explore the concept of deep phenotyping-the comprehensive assessment of a condition using multiple clinical, biological, genetic, imaging, and sensor-based tools-for PD. We discuss the rationale for, outline current approaches to, identify benefits and limitations of, and consider future directions for deep clinical phenotyping.

rConverse: Moment by Moment Conversation Detection Using a Mobile Respiration Sensor.

Monitoring of in-person conversations has largely been done using acoustic sensors. In this paper, we propose a new method to detect moment-by-moment conversation episodes by analyzing breathing patterns captured by a mobile respiration sensor. Since breathing is affected by physical and cognitive activities, we develop a comprehensive method for cleaning, screening, and analyzing noisy respiration data captured in the field environment at individual breath cycle level. Using training data collected from a speech dynamics lab study with 12 participants, we show that our algorithm can identify each respiration cycle with 96.34% accuracy even in presence of walking. We present a Conditional Random Field, Context-Free Grammar (CRF-CFG) based conversation model, called rConverse, to classify respiration cycles into speech or non-speech, and subsequently infer conversation episodes. Our model achieves 82.7% accuracy for speech/non-speech classification and it identifies conversation episodes with 95.9% accuracy on lab data using a leave-one-subject-out cross-validation. Finally, the system is validated against audio ground-truth in a field study with 32 participants. rConverse identifies conversation episodes with 71.7% accuracy on 254 hours of field data. For comparison, the accuracy from a high-quality audio-recorder on the same data is 71.9%.

Learning Time Series Detection Models from Temporally Imprecise Labels.

In this paper, we consider the problem of learning time series detection models from temporally imprecise labels. In this problem, the data consist of a set of input time series, and supervision is provided by a sequence of noisy time stamps corresponding to the occurrence of positive class events. Such temporally imprecise labels occur in areas like mobile health research when human annotators are tasked with labeling the occurrence of very short duration events. We propose a general learning framework for this problem that can accommodate different base classifiers and noise models. We present results on real mobile health data showing that the proposed framework significantly outperforms a number of alternatives including assuming that the label time stamps are noise-free, transforming the problem into the multiple instance learning framework, and learning on labels that were manually aligned.

Hierarchical Span-Based Conditional Random Fields for Labeling and Segmenting Events in Wearable Sensor Data Streams.

The field of mobile health (mHealth) has the potential to yield new insights into health and behavior through the analysis of continuously recorded data from wearable health and activity sensors. In this paper, we present a hierarchical span-based conditional random field model for the key problem of jointly detecting discrete events in such sensor data streams and segmenting these events into high-level activity sessions. Our model includes higher-order cardinality factors and inter-event duration factors to capture domain-specific structure in the label space. We show that our model supports exact MAP inference in quadratic time via dynamic programming, which we leverage to perform learning in the structured support vector machine framework. We apply the model to the problems of smoking and eating detection using four real data sets. Our results show statistically significant improvements in segmentation performance relative to a hierarchical pairwise CRF.

Towards collaborative filtering recommender systems for tailored health communications.

The goal of computer tailored health communications (CTHC) is to promote healthy behaviors by sending messages tailored to individual patients. Current CTHC systems collect baseline patient "profiles" and then use expert-written, rule-based systems to target messages to subsets of patients. Our main interest in this work is the study of collaborative filtering-based CTHC systems that can learn to tailor future message selections to individual patients based explicit feedback about past message selections. This paper reports the results of a study designed to collect explicit feedback (ratings) regarding four aspects of messages from 100 subjects in the smoking cessation support domain. Our results show that most users have positive opinions of most messages and that the ratings for all four aspects of the messages are highly correlated with each other. Finally, we conduct a range of rating prediction experiments comparing several different model variations. Our results show that predicting future ratings based on each user's past ratings contributes the most to predictive accuracy.