Effect of Gamification, Financial Incentives, or Both to Increase Physical Activity Among Patients at High Risk of Cardiovascular Events: The BE ACTIVE Randomized Controlled Trial.

BACKGROUND: Physical activity is associated with a lower risk of major adverse cardiovascular events, but few individuals achieve guideline-recommended levels of physical activity. Strategies informed by behavioral economics increase physical activity, but their longer-term effectiveness is uncertain. We sought to determine the effect of behaviorally designed gamification, loss-framed financial incentives, or their combination on physical activity compared with attention control over 12-month intervention and 6-month postintervention follow-up periods. METHODS: Between May 2019 and January 2024, participants with clinical atherosclerotic cardiovascular disease or a 10-year risk of myocardial infarction, stroke, or cardiovascular death of ≥7.5% by the Pooled Cohort equation were enrolled in a pragmatic randomized clinical trial. Participants received a wearable device to track daily steps, established a baseline, selected a step goal increase, and were randomly assigned to control (n=151), behaviorally designed gamification (n=304), loss-framed financial incentives (n=302), or gamification+financial incentives (n=305). The primary outcome of the trial was the change in mean daily steps from baseline through the 12-month intervention period. RESULTS: A total of 1062 patients (mean±SD age, 67±8; 61% female; 31% non-White) were enrolled. Compared with control subjects, participants had significantly greater increases in mean daily steps from baseline during the 12-month intervention in the gamification arm (adjusted difference, 538.0 [95% CI, 186.2-889.9]; P=0.0027), financial incentives arm (adjusted difference, 491.8 [95% CI, 139.6-844.1]; P=0.0062), and gamification+financial incentives arm (adjusted difference, 868.0 [95% CI, 516.3-1219.7]; P<0.0001). During the 6-month follow-up, physical activity remained significantly greater in the gamification+financial incentives arm than in the control arm (adjusted difference, 576.2 [95% CI, 198.5-954]; P=0.0028), but it was not significantly greater in the gamification (adjusted difference, 459.8 [95% CI, 82.0-837.6]; P=0.0171) or financial incentives (adjusted difference, 327.9 [95% CI, -50.2 to 706]; P=0.09) arms after adjustment for multiple comparisons. CONCLUSIONS: Behaviorally designed gamification, loss-framed financial incentives, and the combination of both increased physical activity compared with control over a 12-month intervention period, with the largest effect in gamification+financial incentives. These interventions could be a useful component of strategies to reduce cardiovascular risk in high-risk patients. REGISTRATION: URL: https://clinicaltrials.gov; Unique Identifier: NCT03911141.

Operationalizing Equity, Inclusion, and Access in Research Practice at a Large Academic Institution.

INTRODUCTION: Healthcare advances are hindered by underrepresentation in prospective research; sociodemographic, data, and measurement infidelity in retrospective research; and a paucity of guidelines surrounding equitable research practices. OBJECTIVE: The Joint Research Practices Working Group was created in 2021 to develop and disseminate guidelines for the conduct of inclusive and equitable research. METHODS: Volunteer faculty and staff from two research centers at the University of Pennsylvania initiated a multi-pronged approach to guideline development, including literature searches, center-level feedback, and mutual learning with local experts. RESULTS: We developed guidelines for (1) participant payment and incentives; (2) language interpretation and translation; (3) plain language in research communications; (4) readability of study materials; and (5) inclusive language for scientific communications. Key recommendations include (1) offer cash payments and multiple payment options to participants when required actions are completed; (2) identify top languages of your target population, map points of contact, and determine available interpretation and translation resources; (3) assess reading levels of materials and simplify language, targeting 6th- to 8th-grade reading levels; (4) improve readability through text formatting and style, symbols, and visuals; and (5) use specific, humanizing terms as adjectives rather than nouns. CONCLUSIONS: Diversity, inclusion, and access are critical values for research conduct that promotes justice and equity. These values can be operationalized through organizational commitment that combines bottom-up and top-down approaches and through partnerships across organizations that promote mutual learning and synergy. While our guidelines represent best practices at one time, we recognize that practices evolve and need to be evaluated continuously for accuracy and relevance. Our intention is to bring awareness to these critical topics and form a foundation for important conversations surrounding equitable and inclusive research practices.

Differential Hospital Participation in Bundled Payments in Communities with Higher Shares of Marginalized Populations.

BACKGROUND: Medicare's voluntary bundled payment programs have demonstrated generally favorable results. However, it remains unknown whether uneven hospital participation in these programs in communities with greater shares of minorities and patients of low socioeconomic status results in disparate access to practice redesign innovations. OBJECTIVE: Examine whether communities with higher proportions of marginalized individuals were less likely to be served by a hospital participating in Bundled Payments for Care Improvement Advanced (BPCI-Advanced). DESIGN: Cross-sectional study using ordinary least squares regression controlling for patient and community factors. PARTICIPANTS: Medicare fee-for-service patients enrolled from 2015-2017 (pre-BPCI-Advanced) and residing in 2,058 local communities nationwide defined by Hospital Service Areas (HSAs). Each community's share of marginalized patients was calculated separately for each of the share of beneficiaries of Black race, Hispanic ethnicity, or dual eligibility for Medicare and Medicaid. MAIN MEASURES: Dichotomous variable indicating whether a given community had at least one hospital that ever participated in BPCI-Advanced from 2018-2022. KEY RESULTS: Communities with higher shares of dual-eligible individuals were less likely to be served by a hospital participating in BPCI-Advanced than communities with the lowest quartile of dual-eligible individuals (Q4: -15.1 percentage points [pp] lower than Q1, 95% CI: -21.0 to -9.1, p < 0.001). There was no consistent significant relationship between community proportion of Black beneficiaries and likelihood of having a hospital participating in BPCI-Advanced. Communities with higher shares of Hispanic beneficiaries were more likely to have a hospital participating in BPCI-Advanced than those in the lowest quartile (Q4: 19.2 pp higher than Q1, 95% CI: 13.4 to 24.9, p < 0.001). CONCLUSIONS: Communities with greater shares of dual-eligible beneficiaries, but not racial or ethnic minorities, were less likely to be served by a hospital participating in BPCI-Advanced Policymakers should consider approaches to incentivize more socioeconomically uniform participation in voluntary bundled payments.

Gamification-augmented home-based exercise for peripheral artery disease: Rationale and design of the GAMEPAD Study.

BACKGROUND: Supervised exercise therapy improves walking performance, functional capacity, and quality of life in patients with peripheral artery disease (PAD). However, few patients with PAD are enrolled in supervised exercise programs, and there are a number of logistical and financial barriers to their participation. A home-based walking intervention is likely to be more accessible to patients with PAD, but no fully home-based walking program has demonstrated efficacy. Concepts from behavioral economics have been used to design scalable interventions that increase daily physical activity in patients with atherosclerotic vascular disease, but whether a similar program would be effective in patients with PAD is uncertain. STUDY DESIGN AND OBJECTIVES: GAMEPAD (NCT04536012) is a pragmatic, virtual, randomized controlled trial designed to evaluate the effectiveness of a gamification strategy informed by concepts from behavioral economics to increase daily physical activity in patients with PAD who are seen in cardiology and vascular surgery clinics affiliated with the University of Pennsylvania Health System. Patients are contacted by email or text message, and complete enrollment and informed consent on the Penn Way to Health online platform. A GAMEPAD substudy will evaluate the effectiveness of opt-in versus opt-out framing when approaching patients for study participation. Patients are then provided with a wearable fitness tracker, establish a baseline daily step count, set a goal to increase daily step count by 33%-50%, and are randomized 1:1 to the gamification or control arms. Interventions continue for 16 weeks, including a 4-week period during which goal step count is gradually increased in the gamification arm, with follow-up for an additional 8 weeks to evaluate the durability of behavior change. The trial has met its enrollment goal of 102 participants, with a primary endpoint of change from baseline in daily steps over the 16-week intervention period. Key secondary endpoints include change from baseline in daily steps over the 8-week postintervention follow-up period and changes in patient-reported measures of PAD symptoms and quality of life over the intervention and follow-up periods. CONCLUSIONS: GAMEPAD is a virtual, pragmatic randomized clinical trial of a novel, fully home-based walking intervention informed by concepts from behavioral economics to increase physical activity and PAD-specific quality of life in patients with PAD. Its results will have important implications for the application of behavioral economic concepts to scalable home-based strategies to promote physical activity in patients with PAD and other disease processes where physical activity is limited by exertional symptoms. CLINICAL TRIAL REGISTRATION: clinicaltrials.gov; NCT04536012.

Quantifying and correcting bias due to outcome dependent self-reported weights in longitudinal study of weight loss interventions.

In response to the escalating global obesity crisis and its associated health and financial burdens, this paper presents a novel methodology for analyzing longitudinal weight loss data and assessing the effectiveness of financial incentives. Drawing from the Keep It Off trial-a three-arm randomized controlled study with 189 participants-we examined the potential impact of financial incentives on weight loss maintenance. Given that some participants choose not to weigh themselves because of small weight change or weight gains, which is a common phenomenon in many weight-loss studies, traditional methods, for example, the Generalized Estimating Equations (GEE) method tends to overestimate the effect size due to the assumption that data are missing completely at random. To address this challenge, we proposed a framework which can identify evidence of missing not at random and conduct bias correction using the estimating equation derived from pairwise composite likelihood. By analyzing the Keep It Off data, we found that the data in this trial are most likely characterized by non-random missingness. Notably, we also found that the enrollment time (i.e., duration time) would be positively associated with the weight loss maintenance after adjusting for the baseline participant characteristics (e.g., age, sex). Moreover, the lottery-based intervention was found to be more effective in weight loss maintenance compared with the direct payment intervention, though the difference was non-statistically significant. This framework's significance extends beyond weight loss research, offering a semi-parametric approach to assess missing data mechanisms and robustly explore associations between exposures (e.g., financial incentives) and key outcomes (e.g., weight loss maintenance). In essence, the proposed methodology provides a powerful toolkit for analyzing real-world longitudinal data, particularly in scenarios with data missing not at random, enriching comprehension of intricate dataset dynamics.

Qualitative analysis of a remote monitoring intervention for managing heart failure.

BACKGROUND: Heart failure (HF) is one of the most common reasons for hospital admission and is a major cause of morbidity, mortality, and increasing health care costs. The EMPOWER study was a randomized trial that used remote monitoring technology to track patients' weight and diuretic adherence and a state-of-the-art approach derived from behavioral economics to motivate adherence to the reverse monitoring technology. OBJECTIVE: The goal was to explore patient and clinician perceptions of the program and its impact on perceived health outcomes and better understand why some patients or clinicians did better or worse than others in response to the intervention. APPROACH: This was a retrospective qualitative study utilizing semi-structured interviews with 43 patients and 16 clinicians to understand the trial's processes, reflecting on successes and areas for improvement for future iterations of behavioral economic interventions. KEY RESULTS: Many patients felt supported, and they appreciated the intervention. Many also appreciated the lottery intervention, and while it was not an incentive for enrolling for many respondents, it may have increased adherence during the study. Clinicians felt that the intervention integrated well into their workflow, but the number of alerts was burdensome. Additionally, responses to alerts varied considerably by provider, perhaps because there are no professional guidelines for alerts unaccompanied by severe symptoms. CONCLUSION: Our qualitative analysis indicates potential areas for additional exploration and consideration to design better behavioral economic interventions to improve cardiovascular health outcomes for patients with HF. Patients appreciated lottery incentives for adhering to program requirements; however, many were too far along in their disease progression to benefit from the intervention. Clinicians found the amount and frequency of electronic alerts burdensome and felt they did not improve patient outcomes. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT02708654.

A randomized controlled trial of gamification, financial incentives, or both to increase physical activity among patients with elevated risk for cardiovascular disease: rationale and design of the be active study.

BACKGROUND: Higher levels of physical activity are associated with improvements in cardiovascular health, and consensus guidelines recommend that individuals with or at risk for atherosclerotic cardiovascular disease (ASCVD) participate in regular physical activity. However, most adults do not achieve recommended levels of physical activity. Concepts from behavioral economics have been used to design scalable interventions that increase physical activity over short time periods, but the longer-term efficacy of these strategies is uncertain. STUDY DESIGN AND OBJECTIVES: BE ACTIVE (NCT03911141) is a pragmatic, virtual, randomized controlled trial designed to evaluate the effectiveness of 3 strategies informed by behavioral economic concepts to increase daily physical activity in patients with established ASCVD or 10-year ASCVD risk > 7.5% who are seen in primary care and cardiology clinics affiliated with the University of Pennsylvania Health System. Patients are contacted by email or text message, and complete enrollment and informed consent on the Penn Way to Health online platform. Patients are then provided with a wearable fitness tracker, establish a baseline daily step count, set a goal to increase daily step count by 33% to 50%, and are randomized 1:2:2:2 to control, gamification, financial incentives, or both gamification and financial incentives. Interventions continue for 12 months, with follow-up for an additional 6 months to evaluate the durability of behavior change. The trial has met its enrollment goal of 1050 participants, with a primary endpoint of change from baseline in daily steps over the 12-month intervention period. Key secondary endpoints include change from baseline in daily steps over the 6-month post-intervention follow-up period and change in moderate to vigorous physical activity over the intervention and follow-up periods. If the interventions prove effective, their effects on life expectancy will be compared with their costs in cost-effectiveness analysis. CONCLUSIONS: BE ACTIVE is a virtual, pragmatic randomized clinical trial powered to demonstrate whether gamification, financial incentives, or both are superior to attention control in increasing physical activity. Its results will have important implications for strategies to promote physical activity in patients with or at risk for ASCVD, as well as for the design and implementation of pragmatic virtual clinical trials within health systems.

Qualitative Analysis of a Remote Monitoring Intervention for Managing Heart Failure.

Background : Heart failure (HF) is one of the most common reasons for hospital admission and is a major cause of morbidity, mortality, and increasing health care costs. The EMPOWER study was a randomized trial that used remote monitoring technology to track patients' weight and diuretic adherence and a state-of-the-art approach derived from behavioral economics to motivate adherence to the reverse monitoring technology. Objective : The goal was to explore patient and clinician perceptions of the program and its impact on health outcomes and better understand why some patients/clinicians did better/worse than others in response to the intervention. Approach : This was a retrospective qualitative study to understand the trial's processes, reflecting on successes and areas for improvement for future iterations of behavioral economic interventions. Key Results: Many patients felt supported, and they appreciated the intervention. Many also appreciated the lottery intervention, and while it was not an incentive for enrolling for many respondents, it may have increased adherence during the study. Clinicians felt that the intervention integrated well into their workflow, but the number of alerts was burdensome. Additionally, responses to alerts varied considerably by provider, perhaps because there are no professional guidelines for alerts unaccompanied by severe symptoms. Conclusion : Those interviews offer insights into the potential reasons for the study's null result and opportunities for improvements in the future. Trial Registration: ClinicalTrials.gov Identifier: NCT02708654.

Performance drift in a mortality prediction algorithm among patients with cancer during the SARS-CoV-2 pandemic.

Sudden changes in health care utilization during the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic may have impacted the performance of clinical predictive models that were trained prior to the pandemic. In this study, we evaluated the performance over time of a machine learning, electronic health record-based mortality prediction algorithm currently used in clinical practice to identify patients with cancer who may benefit from early advance care planning conversations. We show that during the pandemic period, algorithm identification of high-risk patients had a substantial and sustained decline. Decreases in laboratory utilization during the peak of the pandemic may have contributed to drift. Calibration and overall discrimination did not markedly decline during the pandemic. This argues for careful attention to the performance and retraining of predictive algorithms that use inputs from the pandemic period.

The Association between Bundled Payment Participation and Changes in Medical Episode Outcomes among High-Risk Patients.

Background: Bundled payments for medical conditions are associated with stable quality and savings through shorter skilled nursing facility (SNF) length of stay. However, effects among clinically higher-risk patients remain unknown. Objective: To evaluate whether the association between participation in bundled payments for medical conditions and episode outcomes differed for clinically high-risk versus other patients. Design: Retrospective difference-in-differences analysis; Participants: 471,421 Medicare patients hospitalized at bundled payment and propensity-matched non-participating hospitals. Exposures were 5 measures of clinically high-risk groups: advanced age (>85 years old), high case-mix, disabled, frail, and prior institutional post-acute care provider utilization. Main Measures: Primary outcomes were SNF length of stay and 90-day unplanned readmissions. Secondary outcomes included quality, utilization, and spending measures. Key Results: SNF length of stay was differentially lower among frail patients (aDID −0.4 days versus non-frail patients, 95% CI −0.8 to −0.1 days), patients with advanced age (aDID −0.8 days versus younger patients, 95% CI −1.2 to −0.3 days), and those with prior institutional post-acute care provider utilization (aDID −1.1 days versus patients without prior utilization, 95% CI −1.6 to −0.6 days), compared to non-frail, younger, and patients without prior utilization, respectively. BPCI participation was also associated with differentially greater SNF LOS among disabled patients (aDID 0.8 days versus non-disabled patients, 95% CI 0.4 to 1.2 days, p < 0.001). Bundled payment participation was not associated with differential changes in readmissions in any high-risk group but was associated with changes in secondary outcomes for some groups. Conclusions: Changes under medical bundles affected, but did not indiscriminately apply to, high-risk patient groups.

Performance of Physician Groups and Hospitals Participating in Bundled Payments Among Medicare Beneficiaries.

Importance: Hospital participation in bundled payment initiatives has been associated with financial savings and stable quality of care. However, how physician group practices (PGPs) perform in bundled payments compared with hospitals remains unknown. Objectives: To evaluate the association of PGP participation in the Bundled Payments for Care Improvement (BPCI) initiative with episode outcomes and to compare these with outcomes for participating hospitals. Design, Settings, and Participants: This cohort study with a difference-in-differences analysis used 2011 to 2018 Medicare claims data to compare the association of BPCI participation with episode outcomes for PGPs vs hospitals providing medical and surgical care to Medicare beneficiaries. Data analyses were conducted from January 1, 2020, to May 31, 2022. Exposures: Hospitalization for any of the 10 highest-volume episodes (5 medical and 5 surgical) included in the BPCI initiative for Medicare patients of participating PGPs and hospitals. Main Outcomes and Measures: The primary outcome was 90-day total episode spending. Secondary outcomes were 90-day readmissions and mortality. Results: The total sample comprised data from 1 288 781 Medicare beneficiaries, of whom 696 710 (mean [SD] age, 76.2 [10.8] years; 432 429 [59.7%] women; 619 655 [85.5%] White individuals) received care through 379 BPCI-participating hospitals and 1441 propensity-matched non-BPCI-participating hospitals, and 592 071 (mean [SD] age, 75.4 [10.9] years; 527 574 [86.6%] women; 360 835 [59.3%] White individuals) received care from 6405 physicians in BPCI-participating PGPs and 24 758 propensity-matched physicians in non-BPCI-participating PGPs. For PGPs, BPCI participation was associated with greater reductions in episode spending for surgical (difference, -$1368; 95% CI, -$1648 to -$1088) but not for medical episodes (difference, -$101; 95% CI, -$410 to $206). Hospital participation in BPCI was associated with greater reductions in episode spending for both surgical (-$1010; 95% CI, -$1345 to -$675) and medical (-$763; 95% CI, -$1139 to -$386) episodes. Conclusions and Relevance: This cohort study and difference-in-differences analysis of PGPs and hospital participation in BPCI found that bundled payments were associated with cost savings for surgical episodes for PGPs, and savings for both surgical and medical episodes for hospitals. Policy makers should consider the comparative performance of participant types when designing and evaluating bundled payment models.

The Proportion of Marginalized Individuals in US Communities and Hospital Participation in Bundled Payments.

Hospitals have demonstrated the benefits of both voluntary and mandatory bundled payments for joint replacement surgery. However, given generalizability and disparities concerns, it is critical to understand the availability of care through bundled payments to historically marginalized groups, such as racial and ethnic minorities and individuals with lower socioeconomic status (SES). This cross-sectional analysis of 3880 US communities evaluated the relationship between the proportion of Black and Hispanic individuals (minority share) or Medicare/Medicaid dual-eligible individuals (low SES share) and community-level participation in Bundled Payments for Care Improvement initiative (BPCI) (being a BPCI community) and Comprehensive Care for Joint Replacement (CJR) model (being a CJR community). An increase from the lowest to highest quartile of minority share was not associated with differences in the probability of being a BPCI community (3.5 percentage point [pp] difference, 95% confidence interval [CI] -1.2% to 8.3%, P = 0.15), but was associated with a 16.1 pp higher probability of being a CJR community (95% CI 10.3% to 22.0%, P < 0.0001). An increase from the lowest to highest quartile of low SES share was associated with a 6.0 pp lower probability of being a BPCI community (95% CI -10.9% to -1.2%, P = 0.02) and 19.0 pp lower probability of being a CJR community (95% CI -24.9% to -13.0%, P < 0.0001). These findings highlight that the greater the proportion of lower SES individuals in a community, the lower the likelihood that its hospitals participated in either voluntary or mandatory bundled payments. Policymakers should consider community socioeconomic characteristics when designing participation mechanisms for future bundled payment programs.

Remote Monitoring and Behavioral Economics in Managing Heart Failure in Patients Discharged From the Hospital: A Randomized Clinical Trial.

Importance: Close remote monitoring of patients following discharge for heart failure (HF) may reduce readmissions or death. Objective: To determine whether remote monitoring of diuretic adherence and weight changes with financial incentives reduces hospital readmissions or death following discharge with HF. Design, Setting, and Participants: The Electronic Monitoring of Patients Offers Ways to Enhance Recovery (EMPOWER) study, a 3-hospital pragmatic trial, randomized 552 adults recently discharged with HF to usual care (n = 280) or a compound intervention (n = 272) designed to inform clinicians of diuretic adherence and changes in patient weight. Patients were recruited from May 25, 2016, to April 8, 2019, and followed up for 12 months. Investigators were blinded to assignment but patients were not. Analysis was by intent to treat. Interventions: Participants randomized to the intervention arm received digital scales, electronic pill bottles for diuretic medication, and regret lottery incentives conditional on the previous day's adherence to both medication and weight measurement, with $1.40 expected daily value. Participants' physicians were alerted if participants' weights increased 1.4 kg in 24 hours or 2.3 kg in 72 hours or if diuretic medications were missed for 5 days. Alerts and weights were integrated into the electronic health record. Participants randomized to the control arm received usual care and no further study contact. Main Outcomes and Measures: Time to death or readmission for any cause within 12 months. Results: Of the 552 participants, 290 were men (52.5%); 291 patients (52.7%) were Black, 231 were White (41.8%), and 16 were Hispanic (2.9%); mean (SD) age was 64.5 (11.8) years. The mean (SD) ejection fraction was 43% (18.1%). Each month, approximately 75% of participants were 80% adherent to both medication and weight measurement. There were 423 readmissions and 26 deaths in the control group and 377 readmissions and 23 deaths in the intervention group. There was no significant difference between the 2 groups for the combined outcome of all-cause inpatient readmission or death (unadjusted hazard ratio, 0.91; 95% CI, 0.74-1.13; P = .40) and no significant differences in all-cause inpatient readmission or observation stay or death, all-cause cardiovascular readmission or death, time to first event, and total all-cause deaths. Participants in the intervention group were slightly more likely to spend fewer days in the hospital. Conclusions and Relevance: In this randomized clinical trial, there was no reduction in the combined outcome of readmission or mortality in a year-long intensive remote monitoring program with incentives for patients previously hospitalized for HF. Trial Registration: ClinicalTrials.gov Identifier: NCT02708654.

Performance Drift in a Mortality Prediction Algorithm during the SARS-CoV-2 Pandemic.

Research Objective: Health systems use clinical predictive algorithms to allocate resources to high-risk patients. Such algorithms are trained using historical data and are later implemented in clinical settings. During this implementation period, predictive algorithms are prone to performance changes ("drift") due to exogenous shocks in utilization or shifts in patient characteristics. Our objective was to examine the impact of sudden utilization shifts during the SARS-CoV-2 pandemic on the performance of an electronic health record (EHR)-based prognostic algorithm. Study Design: We studied changes in the performance of Conversation Connect, a validated machine learning algorithm that predicts 180-day mortality among outpatients with cancer receiving care at medical oncology practices within a large academic cancer center. Conversation Connect generates mortality risk predictions before each encounter using data from 159 EHR variables collected in the six months before the encounter. Since January 2019, Conversation Connect has been used as part of a behavioral intervention to prompt clinicians to consider early advance care planning conversations among patients with ≥10% mortality risk. First, we descriptively compared encounter-level characteristics in the following periods: January 2019-February 2020 ("pre-pandemic"), March-May 2020 ("early-pandemic"), and June-December 2020 ("later-pandemic"). Second, we quantified changes in high-risk patient encounters using interrupted time series analyses that controlled for pre-pandemic trends and demographic, clinical, and practice covariates. Our primary metric of performance drift was false negative rate (FNR). Third, we assessed contributors to performance drift by comparing distributions of key EHR inputs across periods and predicting later pandemic utilization using pre-pandemic inputs. Population Studied: 237,336 in-person and telemedicine medical oncology encounters. Principal Findings: Age, race, average patient encounters per month, insurance type, comorbidity counts, laboratory values, and overall mortality were similar among encounters in the pre-, early-, and later-pandemic periods. Relative to the pre-pandemic period, the later-pandemic period was characterized by a 6.5-percentage-point decrease (28.2% vs. 34.7%) in high-risk encounters (p<0.001). FNR increased from 41.0% (95% CI 38.0-44.1%) in the pre-pandemic period to 57.5% (95% CI 51.9-63.0%) in the later pandemic period. Compared to the pre-pandemic period, the early and later pandemic periods had higher proportions of telemedicine encounters (0.01% pre-pandemic vs. 20.0% early-pandemic vs. 26.4% later-pandemic) and encounters with no preceding laboratory draws (17.7% pre-pandemic vs. 19.8% early-pandemic vs. 24.1% later-pandemic). In the later pandemic period, observed laboratory utilization was lower than predicted (76.0% vs 81.2%, p<0.001). In the later-pandemic period, mean 180-day mortality risk scores were lower for telemedicine encounters vs. in-person encounters (10.3% vs 11.2%, p<0.001) and encounters with no vs. any preceding laboratory draws (1.5% vs. 14.0%, p<0.001). Conclusions: During the SARS-CoV-2 pandemic period, the performance of a machine learning prognostic algorithm used to prompt advance care planning declined substantially. Increases in telemedicine and declines in laboratory utilization contributed to lower performance. Implications for Policy or Practice: This is the first study to show algorithm performance drift due to SARS-CoV-2 pandemic-related shifts in telemedicine and laboratory utilization. These mechanisms of performance drift could apply to other EHR clinical predictive algorithms. Pandemic-related decreases in care utilization may negatively impact the performance of clinical predictive algorithms and warrant assessment and possible retraining of such algorithms.

The Electronic Health Record as the Primary Data Source in a Pragmatic Trial: A Case Study.

HIGHLIGHTS: Electronic health records are not a single system but a series of overlapping and legacy systems that require time and expertise to use efficiently.Commonly measured patient characteristics such as weight and body mass index are relatively easy to locate for most trial enrollees but less common characteristics, like ejection fraction, are not.Acquiring essential supplementary data-in this trial, state data on hospital admission-can be a lengthy and difficult process.

The Effect of Hospital Safety Net Status on the Association Between Bundled Payment Participation and Changes in Medical Episode Outcomes.

BACKGROUND: Under Medicare's Bundled Payments for Care Improvement (BPCI) program, hospitals have maintained quality and achieved savings for medical conditions. However, safety net hospitals may perform differently owing to financial constraints and organizational challenges. OBJECTIVE: To evaluate whether hospital safety net status affected the association between bundled payment participation and medical episode outcomes. DESIGN, SETTING, AND PARTICIPANTS: This observational difference-in-differences analysis was conducted in safety net and non-safety net hospitals participating in BPCI for medical episodes (BPCI hospitals) using data from 2011-2016 Medicare fee-for-service beneficiaries hospitalized for acute myocardial infarction, pneumonia, congestive heart failure, and chronic obstructive pulmonary disease. EXPOSURE(S): Hospital BPCI participation and safety net status. MAIN OUTCOME(S) AND MEASURE(S): The primary outcome was postdischarge spending. Secondary outcomes included quality and post-acute care utilization measures. RESULTS: Our sample consisted of 803 safety net and 2263 non-safety net hospitals. Safety net hospitals were larger and located in areas with more low-income individuals than non-safety net hospitals. Among BPCI hospitals, safety net status was not associated with differential postdischarge spending (adjusted difference-in-differences [aDID], $40; 95% CI, -$254 to $335; P = .79) or quality (mortality, readmissions). However, BPCI safety net hospitals had differentially greater discharge to institutional post-acute care (aDID, 1.06 percentage points; 95% CI, 0.37-1.76; P = .003) and lower discharge home with home health (aDID, -1.15 percentage points; 95% CI, -1.73 to -0.58; P < .001) than BPCI non-safety net hospitals. CONCLUSIONS: Under medical condition bundles, safety net hospitals perform differently from other hospitals in terms of post-acute care utilization, but not spending. Policymakers could support safety net hospitals and consider safety net status when evaluating bundled payment programs.

Effect of Financial Incentives for Process, Outcomes, or Both on Cholesterol Level Change: A Randomized Clinical Trial.

Importance: Financial incentives may improve health behaviors. It is unknown whether incentives are more effective if they target a key process (eg, medication adherence), an outcome (eg, low-density lipoprotein cholesterol [LDL-C] levels), or both. Objective: To determine whether financial incentives awarded daily for process (adherence to statins), awarded quarterly for outcomes (personalized LDL-C level targets), or awarded for process plus outcomes induce reductions in LDL-C levels compared with control. Design, Setting, and Participants: A randomized clinical trial was conducted from February 12, 2015, to October 3, 2018; data analysis was performed from October 4, 2018, to May 27, 2021, at the University of Pennsylvania Health System, Philadelphia. Participants included 764 adults with an active statin prescription, elevated risk of atherosclerotic cardiovascular disease, suboptimal LDL-C level, and evidence of imperfect adherence to statin medication. Interventions: Interventions lasted 12 months. All participants received a smart pill bottle to measure adherence and underwent LDL-C measurement every 3 months. In the process group, daily financial incentives were awarded for statin adherence. In the outcomes group, participants received incentives for achieving or sustaining at least a quarterly 10-mg/dL LDL-C level reduction. The process plus outcomes group participants were eligible for incentives split between statin adherence and quarterly LDL-C level targets. Main Outcomes and Measures: Change in LDL-C level from baseline to 12 months, determined using intention-to-treat analysis. Results: Of the 764 participants, 390 were women (51.2%); mean (SD) age was 62.4 (10.0) years, 310 (40.6%) had diabetes, 298 (39.0%) had hypertension, and mean (SD) baseline LDL-C level was 138.8 (37.6) mg/dL. Mean LDL-C level reductions from baseline to 12 months were -36.9 mg/dL (95% CI, -42.0 to -31.9 mg/dL) among control participants, -40.0 mg/dL (95% CI, -44.7 to -35.4 mg/dL) among process participants, -41.6 mg/dL (95% CI, -46.3 to -37.0 mg/dL) among outcomes participants, and -42.8 mg/dL (95% CI, -47.4 to -38.1 mg/dL) among process plus outcomes participants. In exploratory analysis among participants with diabetes and hypertension, no spillover effects of incentives were detected compared with the control group on hemoglobin A1c level and blood pressure over 12 months. Conclusions and Relevance: In this randomized clinical trial, process-, outcomes-, or process plus outcomes-based financial incentives did not improve LDL-C levels vs control. Trial Registration: ClinicalTrials.gov Identifier: NCT02246959.

Effect of Financial Incentives and Environmental Strategies on Weight Loss in the Healthy Weigh Study: A Randomized Clinical Trial.

Importance: Modest weight loss can lead to meaningful risk reduction in adults with obesity. Although both behavioral economic incentives and environmental change strategies have shown promise for initial weight loss, to date they have not been combined, or compared, in a randomized clinical trial. Objective: To test the relative effectiveness of financial incentives and environmental strategies, alone and in combination, on initial weight loss and maintenance of weight loss in adults with obesity. Design, Setting, and Participants: This randomized clinical trial was conducted from 2015 to 2019 at 3 large employers in Philadelphia, Pennsylvania. A 2-by-2 factorial design was used to compare the effects of lottery-based financial incentives, environmental strategies, and their combination vs usual care on weight loss and maintenance. Interventions were delivered via website, text messages, and social media. Participants included adult employees with a body mass index (BMI; weight in kilograms divided by height in meters squared) of 30 to 55 and at least 1 other cardiovascular risk factor. Data analysis was performed from June to July 2021. Interventions: Interventions included lottery-based financial incentives based on meeting weight loss goals, environmental change strategies tailored for individuals and delivered by text messages and social media, and combined incentives and environmental strategies. Main Outcome and Measures: The primary outcome was weight change from baseline to 18 months, measured in person. Results: A total of 344 participants were enrolled, with 86 participants each randomized to the financial incentives group, environmental strategies group, combined financial incentives and environmental strategies group, and usual care (control) group. Participants had a mean (SD) age of 45.6 (10.5) years and a mean (SD) BMI of 36.5 (7.1); 247 participants (71.8%) were women, 172 (50.0%) were Black, and 138 (40.1%) were White. At the primary end point of 18 months, participants in the incentives group lost a mean of 5.4 lb (95% CI, -11.3 to 0.5 lb [mean, 2.45 kg; 95% CI, -5.09 to 0.23 kg]), those in the environmental strategies group lost a mean of a 2.2 lb (95% CI, -7.7 to 3.3 lb [mean, 1.00 kg; 95% CI, -3.47 to 1.49 kg]), and the combination group lost a mean of 2.4 lb (95% CI, -8.2 to 3.3 lb [mean, 1.09 kg; 95% CI, -3.69 to 1.49 kg]) more than participants in the usual care group. Financial incentives, environmental change strategies, and their combination were not significantly more effective than usual care. At 24 months, after 6 months without an intervention, the difference in the change from baseline was similar to the 18-month results, with no significant differences among groups. Conclusions and Relevance: In this randomized clinical trial, across all study groups, participants lost a modest amount of weight but those who received financial incentives, environmental change, or the combined intervention did not lose significantly more weight than those in the usual care group. Employees with obesity may benefit from more intensive individualized weight loss strategies. Trial Registration: ClinicalTrials.gov Identifier: NCT02878343.