Using remotely monitored patient activity patterns after hospital discharge to predict 30 day hospital readmission: a randomized trial.

Hospital readmission prediction models often perform poorly, but most only use information collected until the time of hospital discharge. In this clinical trial, we randomly assigned 500 patients discharged from hospital to home to use either a smartphone or wearable device to collect and transmit remote patient monitoring (RPM) data on activity patterns after hospital discharge. Analyses were conducted at the patient-day level using discrete-time survival analysis. Each arm was split into training and testing folds. The training set used fivefold cross-validation and then final model results are from predictions on the test set. A standard model comprised data collected up to the time of discharge including demographics, comorbidities, hospital length of stay, and vitals prior to discharge. An enhanced model consisted of the standard model plus RPM data. Traditional parametric regression models (logit and lasso) were compared to nonparametric machine learning approaches (random forest, gradient boosting, and ensemble). The main outcome was hospital readmission or death within 30 days of discharge. Prediction of 30-day hospital readmission significantly improved when including remotely-monitored patient data on activity patterns after hospital discharge and using nonparametric machine learning approaches. Wearables slightly outperformed smartphones but both had good prediction of 30-day hospital-readmission.

Long-term Effect of Machine Learning-Triggered Behavioral Nudges on Serious Illness Conversations and End-of-Life Outcomes Among Patients With Cancer: A Randomized Clinical Trial.

Importance: Serious illness conversations (SICs) between oncology clinicians and patients are associated with improved quality of life and may reduce aggressive end-of-life care. However, most patients with cancer die without a documented SIC. Objective: To test the impact of behavioral nudges to clinicians to prompt SICs on the SIC rate and end-of-life outcomes among patients at high risk of death within 180 days (high-risk patients) as identified by a machine learning algorithm. Design, Setting, and Participants: This prespecified 40-week analysis of a stepped-wedge randomized clinical trial conducted between June 17, 2019, and April 20, 2020 (including 16 weeks of intervention rollout and 24 weeks of follow-up), included 20 506 patients with cancer representing 41 021 encounters at 9 tertiary or community-based medical oncology clinics in a large academic health system. The current analyses were conducted from June 1, 2021, to May 31, 2022. Intervention: High-risk patients were identified using a validated electronic health record machine learning algorithm to predict 6-month mortality. The intervention consisted of (1) weekly emails to clinicians comparing their SIC rates for all patients against peers' rates, (2) weekly lists of high-risk patients, and (3) opt-out text messages to prompt SICs before encounters with high-risk patients. Main Outcomes and Measures: The primary outcome was SIC rates for all and high-risk patient encounters; secondary end-of-life outcomes among decedents included inpatient death, hospice enrollment and length of stay, and intensive care unit admission and systemic therapy close to death. Intention-to-treat analyses were adjusted for clinic and wedge fixed effects and clustered at the oncologist level. Results: The study included 20 506 patients (mean [SD] age, 60.0 [14.0] years) and 41 021 patient encounters: 22 259 (54%) encounters with female patients, 28 907 (70.5%) with non-Hispanic White patients, and 5520 (13.5%) with high-risk patients; 1417 patients (6.9%) died by the end of follow-up. There were no meaningful differences in demographic characteristics in the control and intervention periods. Among high-risk patient encounters, the unadjusted SIC rates were 3.4% (59 of 1754 encounters) in the control period and 13.5% (510 of 3765 encounters) in the intervention period. In adjusted analyses, the intervention was associated with increased SICs for all patients (adjusted odds ratio, 2.09 [95% CI, 1.53-2.87]; P < .001) and decreased end-of-life systemic therapy (7.5% [72 of 957 patients] vs 10.4% [24 of 231 patients]; adjusted odds ratio, 0.25 [95% CI, 0.11-0.57]; P = .001) relative to controls, but there was no effect on hospice enrollment or length of stay, inpatient death, or end-of-life ICU use. Conclusions and Relevance: In this randomized clinical trial, a machine learning-based behavioral intervention and behavioral nudges to clinicans led to an increase in SICs and reduction in end-of-life systemic therapy but no changes in other end-of-life outcomes among outpatients with cancer. These results suggest that machine learning and behavioral nudges can lead to long-lasting improvements in cancer care delivery. Trial Registration: ClinicalTrials.gov Identifier: NCT03984773.

A Randomized Trial of Behavioral Nudges Delivered Through Text Messages to Increase Influenza Vaccination Among Patients With an Upcoming Primary Care Visit.

PURPOSE: To evaluate if nudges delivered by text message prior to an upcoming primary care visit can increase influenza vaccination rates. DESIGN: Randomized, controlled trial. SETTING: Two health systems in the Northeastern US between September 2020 and March 2021. SUBJECTS: 74,811 adults. INTERVENTIONS: Patients in the 19 intervention arms received 1-2 text messages in the 3 days preceding their appointment that varied in their format, interactivity, and content. MEASURES: Influenza vaccination. ANALYSIS: Intention-to-treat. RESULTS: Participants had a mean (SD) age of 50.7 (16.2) years; 55.8% (41,771) were female, 70.6% (52,826) were White, and 19.0% (14,222) were Black. Among the interventions, 5 of 19 (26.3%) had a significantly greater vaccination rate than control. On average, the 19 interventions increased vaccination relative to control by 1.8 percentage points or 6.1% (P = .005). The top performing text message described the vaccine to the patient as "reserved for you" and led to a 3.1 percentage point increase (95% CI, 1.3 to 4.9; P < .001) in vaccination relative to control. Three of the top five performing messages described the vaccine as "reserved for you." None of the interventions performed worse than control. CONCLUSIONS: Text messages encouraging vaccination and delivered prior to an upcoming appointment significantly increased influenza vaccination rates and could be a scalable approach to increase vaccination more broadly.

Effect of a Default Order vs an Alert in the Electronic Health Record on Hepatitis C Virus Screening Among Hospitalized Patients: A Stepped-Wedge Randomized Clinical Trial.

Importance: Hepatitis C virus (HCV) screening has been recommended for patients born between 1945 and 1965, but rates remain low. Objective: To evaluate whether a default order within the admission order set increases HCV screening compared with a preexisting alert within the electronic health record. Design, Setting, and Participants: This stepped-wedge randomized clinical trial was conducted from June 23, 2020, to April 10, 2021, at 2 hospitals within an academic medical center. Hospitalized patients born between 1945 and 1965 with no history of screening were included in the analysis. Interventions: During wedge 1 (a preintervention period), both hospital sites had an electronic alert prompting clinicians to consider HCV screening. During wedge 2, the first intervention wedge, the hospital site randomized to intervention (hospital B) had a default order for HCV screening implemented within the admission order set. During wedge 3, the second intervention wedge, the hospital site randomized to control (hospital A) had the default order set implemented. Main Outcomes and Measures: Percentage of eligible patients who received HCV screening during the hospital stay. Results: The study included 7634 patients (4405 in the control group and 3229 in the intervention group). The mean (SD) age was 65.4 (5.8) years; 4246 patients (55.6%) were men; 2142 (28.1%) were Black and 4625 (60.6%) were White; and 2885 (37.8%) had commercial insurance and 3950 (51.7%) had Medicare. The baseline rate of HCV screening in wedge 1 was 585 of 1560 patients (37.5% [95% CI, 35.1%-40.0%]) in hospital A and 309 of 1003 patients (30.8% [95% CI, 27.9%-33.7%]) in hospital B. The main adjusted model showed an increase of 31.8 (95% CI, 29.7-33.8) percentage points in test completion in the intervention group compared with the control group (P <. 001). Conclusions and Relevance: This stepped-wedge randomized clinical trial found that embedding HCV screening as a default order in the electronic health record substantially increased ordering and completion of testing in the hospital compared with a conventional interruptive alert. Trial Registration: Clinicaltrials.gov: NCT04525690.

A 680,000-person megastudy of nudges to encourage vaccination in pharmacies.

Encouraging vaccination is a pressing policy problem. To assess whether text-based reminders can encourage pharmacy vaccination and what kinds of messages work best, we conducted a megastudy. We randomly assigned 689,693 Walmart pharmacy patients to receive one of 22 different text reminders using a variety of different behavioral science principles to nudge flu vaccination or to a business-as-usual control condition that received no messages. We found that the reminder texts that we tested increased pharmacy vaccination rates by an average of 2.0 percentage points, or 6.8%, over a 3-mo follow-up period. The most-effective messages reminded patients that a flu shot was waiting for them and delivered reminders on multiple days. The top-performing intervention included two texts delivered 3 d apart and communicated to patients that a vaccine was "waiting for you." Neither experts nor lay people anticipated that this would be the best-performing treatment, underscoring the value of simultaneously testing many different nudges in a highly powered megastudy.

Clinician perspectives on machine learning prognostic algorithms in the routine care of patients with cancer: a qualitative study.

PURPOSE: Oncologists may overestimate prognosis for patients with cancer, leading to delayed or missed conversations about patients' goals and subsequent low-quality end-of-life care. Machine learning algorithms may accurately predict mortality risk in cancer, but it is unclear how oncology clinicians would use such algorithms in practice. METHODS: The purpose of this qualitative study was to assess oncology clinicians' perceptions on the utility and barriers of machine learning prognostic algorithms to prompt advance care planning. Participants included medical oncology physicians and advanced practice providers (APPs) practicing in tertiary and community practices within a large academic healthcare system. Transcripts were coded and analyzed inductively using NVivo software. RESULTS: The study included 29 oncology clinicians (19 physicians, 10 APPs) across 6 practice sites (1 tertiary, 5 community) in the USA. Fourteen participants had previously had exposure to an automated machine learning-based prognostic algorithm as part of a pragmatic randomized trial. Clinicians believed that there was utility for algorithms in validating their own intuition about prognosis and prompting conversations about patient goals and preferences. However, this enthusiasm was tempered by concerns about algorithm accuracy, over-reliance on algorithm predictions, and the ethical implications around disclosure of an algorithm prediction. There was significant variation in tolerance for false positive vs. false negative predictions. CONCLUSION: While oncologists believe there are applications for advanced prognostic algorithms in routine care of patients with cancer, they are concerned about algorithm accuracy, confirmation and automation biases, and ethical issues of prognostic disclosure.

Predicting changes in glycemic control among adults with prediabetes from activity patterns collected by wearable devices.

The use of wearables is increasing and data from these devices could improve the prediction of changes in glycemic control. We conducted a randomized trial with adults with prediabetes who were given either a waist-worn or wrist-worn wearable to track activity patterns. We collected baseline information on demographics, medical history, and laboratory testing. We tested three models that predicted changes in hemoglobin A1c that were continuous, improved glycemic control by 5% or worsened glycemic control by 5%. Consistently in all three models, prediction improved when (a) machine learning was used vs. traditional regression, with ensemble methods performing the best; (b) baseline information with wearable data was used vs. baseline information alone; and (c) wrist-worn wearables were used vs. waist-worn wearables. These findings indicate that models can accurately identify changes in glycemic control among prediabetic adults, and this could be used to better allocate resources and target interventions to prevent progression to diabetes.

A megastudy of text-based nudges encouraging patients to get vaccinated at an upcoming doctor's appointment.

Many Americans fail to get life-saving vaccines each year, and the availability of a vaccine for COVID-19 makes the challenge of encouraging vaccination more urgent than ever. We present a large field experiment (N = 47,306) testing 19 nudges delivered to patients via text message and designed to boost adoption of the influenza vaccine. Our findings suggest that text messages sent prior to a primary care visit can boost vaccination rates by an average of 5%. Overall, interventions performed better when they were 1) framed as reminders to get flu shots that were already reserved for the patient and 2) congruent with the sort of communications patients expected to receive from their healthcare provider (i.e., not surprising, casual, or interactive). The best-performing intervention in our study reminded patients twice to get their flu shot at their upcoming doctor's appointment and indicated it was reserved for them. This successful script could be used as a template for campaigns to encourage the adoption of life-saving vaccines, including against COVID-19.

Effect of Integrating Machine Learning Mortality Estimates With Behavioral Nudges to Clinicians on Serious Illness Conversations Among Patients With Cancer: A Stepped-Wedge Cluster Randomized Clinical Trial.

IMPORTANCE: Serious illness conversations (SICs) are structured conversations between clinicians and patients about prognosis, treatment goals, and end-of-life preferences. Interventions that increase the rate of SICs between oncology clinicians and patients may improve goal-concordant care and patient outcomes. OBJECTIVE: To determine the effect of a clinician-directed intervention integrating machine learning mortality predictions with behavioral nudges on motivating clinician-patient SICs. DESIGN, SETTING, AND PARTICIPANTS: This stepped-wedge cluster randomized clinical trial was conducted across 20 weeks (from June 17 to November 1, 2019) at 9 medical oncology clinics (8 subspecialty oncology and 1 general oncology clinics) within a large academic health system in Pennsylvania. Clinicians at the 2 smallest subspecialty clinics were grouped together, resulting in 8 clinic groups randomly assigned to the 4 intervention wedge periods. Included participants in the intention-to-treat analyses were 78 oncology clinicians who received SIC training and their patients (N = 14 607) who had an outpatient oncology encounter during the study period. INTERVENTIONS: (1) Weekly emails to oncology clinicians with SIC performance feedback and peer comparisons; (2) a list of up to 6 high-risk patients (≥10% predicted risk of 180-day mortality) scheduled for the next week, estimated using a validated machine learning algorithm; and (3) opt-out text message prompts to clinicians on the patient's appointment day to consider an SIC. Clinicians in the control group received usual care consisting of weekly emails with cumulative SIC performance. MAIN OUTCOMES AND MEASURES: Percentage of patient encounters with an SIC in the intervention group vs the usual care (control) group. RESULTS: The sample consisted of 78 clinicians and 14 607 patients. The mean (SD) age of patients was 61.9 (14.2) years, 53.7% were female, and 70.4% were White. For all encounters, SICs were conducted among 1.3% in the control group and 4.6% in the intervention group, a significant difference (adjusted difference in percentage points, 3.3; 95% CI, 2.3-4.5; P < .001). Among 4124 high-risk patient encounters, SICs were conducted among 3.6% in the control group and 15.2% in the intervention group, a significant difference (adjusted difference in percentage points, 11.6; 95% CI, 8.2-12.5; P < .001). CONCLUSIONS AND RELEVANCE: In this stepped-wedge cluster randomized clinical trial, an intervention that delivered machine learning mortality predictions with behavioral nudges to oncology clinicians significantly increased the rate of SICs among all patients and among patients with high mortality risk who were targeted by the intervention. Behavioral nudges combined with machine learning mortality predictions can positively influence clinician behavior and may be applied more broadly to improve care near the end of life. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT03984773.

Integrating machine-generated mortality estimates and behavioral nudges to promote serious illness conversations for cancer patients: Design and methods for a stepped-wedge cluster randomized controlled trial.

INTRODUCTION: Patients with cancer often receive care that is not aligned with their personal values and goals. Serious illness conversations (SICs) between clinicians and patients can help increase a patient's understanding of their prognosis, goals and values. METHODS AND ANALYSIS: In this study, we describe the design of a stepped-wedge cluster randomized trial to evaluate the impact of an intervention that employs machine learning-based prognostic algorithms and behavioral nudges to prompt oncologists to have SICs with patients at high risk of short-term mortality. Data are collected on documented SICs, documented advance care planning discussions, and end-of-life care utilization (emergency room and inpatient admissions, chemotherapy and hospice utilization) for patients of all enrolled clinicians. CONCLUSION: This trial represents a novel application of machine-generated mortality predictions combined with behavioral nudges in the routine care of outpatients with cancer. Findings from the trial may inform strategies to encourage early serious illness conversations and the application of mortality risk predictions in clinical settings. TRIAL REGISTRATION: Clinicaltrials.gov Identifier: NCT03984773.

Prediction using a randomized evaluation of data collection integrated through connected technologies (PREDICT): Design and rationale of a randomized trial of patients discharged from the hospital to home.

BACKGROUND: Hospital readmission prediction models often perform poorly. A critical limitation is that they use data collected up until the time of discharge but do not leverage information on patient behaviors at home after discharge. METHODS: PREDICT is a two-arm, randomized trial comparing ways to use remotely-monitored patient activity levels after hospital discharge to improve hospital readmission prediction models. Patients are randomly assigned to use a wearable device or smartphone application to track physical activity data. The study collects also validated assessments on patient characteristics as well as disparate data on credit scores and medication adherence. Patients are followed for 6 months. We evaluate whether these data sources can improve prediction compared to standard modelling approaches. CONCLUSION: The PREDICT Trial tests a novel method of remotely-monitoring patient behaviors after hospital discharge. Findings from the trial could inform new ways to improve the identification of patients at high-risk for hospital readmission. TRIAL REGISTRATION: Clinicaltrials.gov Identifier: NCT02983812.

Impact of genomic testing and patient-reported outcomes on receipt of adjuvant chemotherapy.

Practice guidelines incorporate genomic tumor profiling, using results such as the Oncotype DX Recurrence Score (RS), to refine recurrence risk estimates for the large proportion of breast cancer patients with early-stage, estrogen receptor-positive disease. We sought to understand the impact of receiving genomic recurrence risk estimates on breast cancer patients' well-being and the impact of these patient-reported outcomes on receipt of adjuvant chemotherapy. Participants were 193 women (mean age 57) newly diagnosed with early-stage breast cancer. Women were interviewed before and 2-3 weeks after receiving the RS result between 2011 and 2015. We assessed subsequent receipt of chemotherapy from chart review. After receiving their RS, perceived pros (t = 4.27, P < .001) and cons (t = 8.54, P < .001) of chemotherapy increased from pre-test to post-test, while perceived risk of breast cancer recurrence decreased (t = 2.90, P = .004). Women with high RS tumors were more likely to receive chemotherapy than women with low RS tumors (88 vs. 5 %, OR 0.01, 0.00-0.02, P < .001). Higher distress (OR 2.19, 95 % CI 1.05-4.57, P < .05) and lower perceived cons of chemotherapy (OR 0.50, 95 % CI 0.26-0.97, P < .05) also predicted receipt of chemotherapy. Distressed patients who saw few downsides of chemotherapy received this treatment. Clinicians should consider these factors when discussing chemotherapy with breast cancer patients.

Mammographic breast density as a risk factor for breast cancer: awareness in a recently screened clinical sample.

BACKGROUND: Breast density is an established, independent risk factor for breast cancer. Despite this, density has not been included in standard risk models or routinely disclosed to patients. However, this is changing in the face of legal mandates and advocacy efforts. Little information exists regarding women's awareness of density as a risk factor, their personal risk, and risk management options. METHODS: We assessed awareness of density as a risk factor and whether sociodemographic variables, breast cancer risk factors. and perceived breast cancer risk were associated with awareness in 344 women with a recent screening mammogram at a tertiary care center. FINDINGS: Overall, 62% of women had heard about density as a risk factor and 33% had spoken to a provider about breast density. Of the sample, 18% reported that their provider indicated that they had high breast density. Awareness of density as a risk factor was greater among White women and those with other breast cancer risk factors. CONCLUSION: Our results suggest that although a growing number of women are aware of breast density as a risk factor, this awareness varies. Growing mandates for disclosure suggest the need for patient education interventions for women at increased risk for the disease and to ensure all women are equally aware of their risks.