Effectiveness of an Emergency Department-Based Machine Learning Clinical Decision Support Tool to Prevent Outpatient Falls Among Older Adults: Protocol for a Quasi-Experimental Study.

BACKGROUND: Emergency department (ED) providers are important collaborators in preventing falls for older adults because they are often the first health care providers to see a patient after a fall and because at-home falls are often preceded by previous ED visits. Previous work has shown that ED referrals to falls interventions can reduce the risk of an at-home fall by 38%. Screening patients at risk for a fall can be time-consuming and difficult to implement in the ED setting. Machine learning (ML) and clinical decision support (CDS) offer the potential of automating the screening process. However, it remains unclear whether automation of screening and referrals can reduce the risk of future falls among older patients. OBJECTIVE: The goal of this paper is to describe a research protocol for evaluating the effectiveness of an automated screening and referral intervention. These findings will inform ongoing discussions about the use of ML and artificial intelligence to augment medical decision-making. METHODS: To assess the effectiveness of our program for patients receiving the falls risk intervention, our primary analysis will be to obtain referral completion rates at 3 different EDs. We will use a quasi-experimental design known as a sharp regression discontinuity with regard to intent-to-treat, since the intervention is administered to patients whose risk score falls above a threshold. A conditional logistic regression model will be built to describe 6-month fall risk at each site as a function of the intervention, patient demographics, and risk score. The odds ratio of a return visit for a fall and the 95% CI will be estimated by comparing those identified as high risk by the ML-based CDS (ML-CDS) and those who were not but had a similar risk profile. RESULTS: The ML-CDS tool under study has been implemented at 2 of the 3 EDs in our study. As of April 2023, a total of 1326 patient encounters have been flagged for providers, and 339 unique patients have been referred to the mobility and falls clinic. To date, 15% (45/339) of patients have scheduled an appointment with the clinic. CONCLUSIONS: This study seeks to quantify the impact of an ML-CDS intervention on patient behavior and outcomes. Our end-to-end data set allows for a more meaningful analysis of patient outcomes than other studies focused on interim outcomes, and our multisite implementation plan will demonstrate applicability to a broad population and the possibility to adapt the intervention to other EDs and achieve similar results. Our statistical methodology, regression discontinuity design, allows for causal inference from observational data and a staggered implementation strategy allows for the identification of secular trends that could affect causal associations and allow mitigation as necessary. TRIAL REGISTRATION: ClinicalTrials.gov NCT05810064; https://www.clinicaltrials.gov/study/NCT05810064. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/48128.

Reinforced risk prediction with budget constraint using irregularly measured data from electronic health records.

Uncontrolled glycated hemoglobin (HbA1c) levels are associated with adverse events among complex diabetic patients. These adverse events present serious health risks to affected patients and are associated with significant financial costs. Thus, a high-quality predictive model that could identify high-risk patients so as to inform preventative treatment has the potential to improve patient outcomes while reducing healthcare costs. Because the biomarker information needed to predict risk is costly and burdensome, it is desirable that such a model collect only as much information as is needed on each patient so as to render an accurate prediction. We propose a sequential predictive model that uses accumulating patient longitudinal data to classify patients as: high-risk, low-risk, or uncertain. Patients classified as high-risk are then recommended to receive preventative treatment and those classified as low-risk are recommended to standard care. Patients classified as uncertain are monitored until a high-risk or low-risk determination is made. We construct the model using claims and enrollment files from Medicare, linked with patient Electronic Health Records (EHR) data. The proposed model uses functional principal components to accommodate noisy longitudinal data and weighting to deal with missingness and sampling bias. The proposed method demonstrates higher predictive accuracy and lower cost than competing methods in a series of simulation experiments and application to data on complex patients with diabetes.

Insights into measuring health disparities using electronic health records from a statewide network of health systems: A case study.

Within Wisconsin, our residents experience some of the worst health disparities in the nation. Public reporting on disparities in the quality of care is important to achieving accountability for reducing disparities over time and has been associated with improvements in care. Disparities reporting using statewide electronic health records (EHR) data would allow efficient and regular reporting, but there are significant challenges with missing data and data harmonization. We report our experience in creating a statewide, centralized EHR data repository to support health systems in reducing health disparities through public reporting. We partnered with the Wisconsin Collaborative for Healthcare Quality (the "Collaborative"), which houses patient-level EHR data from 25 health systems including validated metrics of healthcare quality. We undertook a detailed assessment of potential disparity indicators (race and ethnicity, insurance status and type, and geographic disparity). Challenges for each indicator are described, with solutions encompassing internal (health system) harmonization, central (Collaborative) harmonization, and centralized data processing. Key lessons include engaging health systems in identifying disparity indicators, aligning with system priorities, measuring indicators already collected in the EHR to minimize burden, and facilitating workgroups with health systems to build relationships, improve data collection, and develop initiatives to address disparities in healthcare.

Primary Care Patients' and Staff's Perceptions of Self-Rooming as Alternative to Waiting Rooms.

PURPOSE: Most patients are escorted to exam rooms (escorted rooming) although patients directing themselves to their exam room (self-rooming) saves patient and staff time while increasing patient satisfaction. This study assesses patient and staff perceptions after pragmatic implementation of self-rooming. METHODS: In October-December 2020, we surveyed patients and staff in 25 primary care clinics after our institution expanded self-rooming from 4 specially built clinics during the COVID-19 pandemic. Semi-structured surveys asked about rooming process used, rooming process preferred, and perceptions of self-rooming compared with escorted rooming. RESULTS: Most patients (n = 1,561) preferred self-rooming (86%), especially among patients aged <65 years and in family medicine clinics. Few patients felt less welcomed (10.6%), less cared about (6.8%), more isolated (15.6%), more lost/confused (7.6%), or more frustrated (3.2%) with self-rooming compared with escorted rooming. Early-adopter clinics that implemented self-rooming ≤2016 had even lower rates of patients feeling more isolated, lost/confused, or frustrated with self-rooming compared with escorted rooming.Over one-half of staff (n = 241; 180 clinical, 61 nonclinical) preferred self-rooming (59%) and thought most patients liked self-rooming (65.8%), especially among clinical staff and in early adopter clinics (≤2016). Few staff reported worse waiting times for patients (12.4%), medical assistants (MAs) (15.9%), and clinicians (16.4%) or worse crowding in waiting areas (1.7%) and hallways (10.1%). Unlike patient-reported confusion (7.6%), most staff thought self-rooming led to more patient confusion (63.8%), except in early-adopter clinics (44.4%). CONCLUSIONS: Self-rooming is a patient-centered innovation that is also acceptable to staff. We demonstrated that pragmatic implementation is feasible across primary care without expensive technology or specially designed buildings.

Collaborative care implementation: lessons learned.

The authors drafted a "Shared Values of Collaborative Care" document with fundamental principles to make better group decisions in implementing collaborative care.

Impactability Modeling for Reducing Medicare Accountable Care Organization Payments and Hospital Events in High-Need High-Cost Patients: Longitudinal Cohort Study.

BACKGROUND: Impactability modeling promises to help solve the nationwide crisis in caring for high-need high-cost patients by matching specific case management programs with patients using a "benefit" or "impactability" score, but there are limitations in tailoring each model to a specific program and population. OBJECTIVE: We evaluated the impact on Medicare accountable care organization savings from developing a benefit score for patients enrolled in a historic case management program, prospectively implementing the score, and evaluating the results in a new case management program. METHODS: We conducted a longitudinal cohort study of 76,140 patients in a Medicare accountable care organization with multiple before-and-after measures of the outcome, using linked electronic health records and Medicare claims data from 2012 to 2019. There were 489 patients in the historic case management program, with 1550 matched comparison patients, and 830 patients in the new program, with 2368 matched comparison patients. The historic program targeted high-risk patients and assigned a centrally located registered nurse and social worker to each patient. The new program targeted high- and moderate-risk patients and assigned a nurse physically located in a primary care clinic. Our primary outcomes were any unplanned hospital events (admissions, observation stays, and emergency department visits), count of event-days, and Medicare payments. RESULTS: In the historic program, as expected, high-benefit patients enrolled in case management had fewer events, fewer event-days, and an average US $1.15 million reduction in Medicare payments per 100 patients over the subsequent year when compared with the findings in matched comparison patients. For the new program, high-benefit high-risk patients enrolled in case management had fewer events, while high-benefit moderate-risk patients enrolled in case management did not differ from matched comparison patients. CONCLUSIONS: Although there was evidence that a benefit score could be extended to a new case management program for similar (ie, high-risk) patients, there was no evidence that it could be extended to a moderate-risk population. Extending a score to a new program and population should include evaluation of program outcomes within key subgroups. With increased attention on value-based care, policy makers and measure developers should consider ways to incorporate impactability modeling into program design and evaluation.

Collaborative design and implementation of a clinical decision support system for automated fall-risk identification and referrals in emergency departments.

Of the 3 million older adults seeking fall-related emergency care each year, nearly one-third visited the Emergency Department (ED) in the previous 6 months. ED providers have a great opportunity to refer patients for fall prevention services at these initial visits, but lack feasible tools for identifying those at highest-risk. Existing fall screening tools have been poorly adopted due to ED staff/provider burden and lack of workflow integration. To address this, we developed an automated clinical decision support (CDS) system for identifying and referring older adult ED patients at risk of future falls. We engaged an interdisciplinary design team (ED providers, health services researchers, information technology/predictive analytics professionals, and outpatient Falls Clinic staff) to collaboratively develop a system that successfully met user requirements and integrated seamlessly into existing ED workflows. Our rapid-cycle development and evaluation process employed a novel combination of human-centered design, implementation science, and patient experience strategies, facilitating simultaneous design of the CDS tool and intervention implementation strategies. This included defining system requirements, systematically identifying and resolving usability problems, assessing barriers and facilitators to implementation (e.g., data accessibility, lack of time, high patient volumes, appointment availability) from multiple vantage points, and refining protocols for communicating with referred patients at discharge. ED physician, nurse, and patient stakeholders were also engaged through online surveys and user testing. Successful CDS design and implementation required integration of multiple new technologies and processes into existing workflows, necessitating interdisciplinary collaboration from the onset. By using this iterative approach, we were able to design and implement an intervention meeting all project goals. Processes used in this Clinical-IT-Research partnership can be applied to other use cases involving automated risk-stratification, CDS development, and EHR-facilitated care coordination.

Does a Change in Physician Compensation Lead to Changes in Care Delivery in Family Medicine Clinics?

BACKGROUND AND OBJECTIVES: Many highly capitated systems still pay physicians based on relative value units (RVU), which may lead to excessive office visits. We reviewed electronic health records from the family medicine clinic panel members of 97 physicians and 42 residents to determine if a change from RVUs to panel-based compensation influenced care delivery as defined by the number of office visits and telephone contacts per panel member per month. METHODS: A retrospective analysis of the electronic health records of patients seen in 4 residency training clinics, 10 community clinics, and 4 regional clinics was conducted. We assessed face-to-face care delivery and telephone call volume for the clinics individually and for the clinics pooled by clinic type from 1 year before to at least 1 year after the change. RESULTS: Change in physician compensation was not found to have an effect on office visits or telephone calls per panel member per month when pooled by clinic categories. Some significant effects were seen in individual clinics without any clear patterns by clinic size or type. CONCLUSIONS: Change in physician compensation was not a key driver of care delivery in family medicine clinics. Understanding changes in care delivery may require looking at a broad array of system, physician, and patient factors.

Estimation and inference on high-dimensional individualized treatment rule in observational data using split-and-pooled de-correlated score.

With the increasing adoption of electronic health records, there is an increasing interest in developing individualized treatment rules, which recommend treatments according to patients' characteristics, from large observational data. However, there is a lack of valid inference procedures for such rules developed from this type of data in the presence of high-dimensional covariates. In this work, we develop a penalized doubly robust method to estimate the optimal individualized treatment rule from high-dimensional data. We propose a split-and-pooled de-correlated score to construct hypothesis tests and confidence intervals. Our proposal adopts the data splitting to conquer the slow convergence rate of nuisance parameter estimations, such as non-parametric methods for outcome regression or propensity models. We establish the limiting distributions of the split-and-pooled de-correlated score test and the corresponding one-step estimator in high-dimensional setting. Simulation and real data analysis are conducted to demonstrate the superiority of the proposed method.

Using Statewide Electronic Health Record and Influenza Vaccination Data to Plan and Prioritize COVID-19 Vaccine Outreach and Communications in Wisconsin Communities.

The University of Wisconsin Neighborhood Health Partnerships Program used electronic health record and influenza vaccination data to estimate COVID-19 relative mortality risk and potential barriers to vaccination in Wisconsin ZIP Code Tabulation Areas. Data visualization revealed four groupings to use in planning and prioritizing vaccine outreach and communication based on ZIP Code Tabulation Area characteristics. The program provided data, visualization, and guidance to health systems, health departments, nonprofits, and others to support planning targeted outreach approaches to increase COVID-19 vaccination uptake. (Am J Public Health. 2021;111(12):2111-2114. https://doi.org/10.2105/AJPH.2021.306524).

Association of continuous positive airway pressure adherence with complications in patients with type 2 diabetes and obstructive sleep apnea.

STUDY OBJECTIVES: To explore the association of continuous positive airway pressure (CPAP) adherence with clinical outcomes in patients with type 2 diabetes and obstructive sleep apnea in a real-world setting. METHODS: This was a retrospective study of patients with type 2 diabetes diagnosed with obstructive sleep apnea between 2010 and 2017. CPAP adherence (usage for ≥ 4 h/night for ≥ 70% of nights) was determined from the first CPAP report following the polysomnography. Data including estimated glomerular filtration rate, hemoglobin A1c, systolic and diastolic blood pressure, lipid panel, and incident cardiovascular/peripheral vascular/cerebrovascular events were extracted from medical records. Mixed-effects linear regression modeling of longitudinal repeated measures within patients was utilized for continuous outcomes, and logistic regression modeling was used for binary outcomes. Models were controlled for age, sex, body mass index, medications, and baseline levels of outcomes. RESULTS: Of the 1,295 patients, 260 (20.7%) were CPAP adherent, 318 (24.5%) were CPAP nonadherent, and 717 (55.3%) had insufficient data. The follow-up period was, on average, 2.5 (1.7) years. Compared to those who were CPAP nonadherent, those who were adherent had a significantly lower systolic blood pressure (ß = -1.95 mm Hg, P = .001) and diastolic blood pressure (ß = -2.33 mm Hg, P < .0001). Among the patients who were CPAP adherent, a 17% greater CPAP adherence was associated with a 2 mm Hg lower systolic blood pressure. Lipids, hemoglobin A1c, estimated glomerular filtration rate, and incident cardiovascular/peripheral vascular/cerebrovascular events were not different between the 2 groups. CONCLUSIONS: Achieving CPAP adherence in patients with type 2 diabetes and obstructive sleep apnea was associated with significantly lower blood pressure. Greater CPAP use within patients who were adherent was associated with lower systolic blood pressure. CITATION: Sheth U, Monson RS, Prasad B, et al. Association of continuous positive airway pressure adherence with complications in patients with type 2 diabetes and obstructive sleep apnea. J Clin Sleep Med. 2021;17(8):1563-1569.

Identifying Substantial Racial and Ethnic Disparities in Health Outcomes and Care in Wisconsin Using Electronic Health Record Data.

BACKGROUND: Our goal was to identify racial and ethnic disparities in health outcome and care measures in Wisconsin. METHODS: We used electronic health record data from 25 health systems submitting to the Wisconsin Collaborative for Healthcare Quality to identify disparities in measures, including vaccinations, screenings, risk factors for chronic disease, and chronic disease management. RESULTS: American Indian/Alaska Native and Black populations experienced substantial disparities across multiple measures. Asian/Pacific Islander, Hispanic/Latino, and White populations experienced substantial disparities for 2 measures each. DISCUSSION: Reducing health disparities is a statewide imperative. Root causes of health disparities, such as systemic racism and socioeconomic factors, should be addressed for groups experiencing multiple disparities, with focused efforts on selected measures when indicated.

Sustaining Gains in Diabetic Eye Screening: Outcomes from a Stakeholder-Based Implementation Program for Teleophthalmology in Primary Care.

Background: Teleophthalmology is a validated method for diabetic eye screening that is underutilized in U.S. primary care clinics. Even when made available to patients, its long-term effectiveness for increasing screening rates is often limited. Introduction: We hypothesized that a stakeholder-based implementation program could increase teleophthalmology use and sustain improvements in diabetic eye screening. Materials and Methods:We used the NIATx Model to test a stakeholder-based teleophthalmology implementation program, I-SITE at one primary care clinic (Main) and compared teleophthalmology use and diabetic eye screening rates with those of other primary care clinics (Outreach) within a U.S. multipayer health system where teleophthalmology was underutilized.Results:Teleophthalmology use increased post-I-SITE implementation (odds ratio [OR] = 5.73 [p < 0.001]), and was greater at the Main than at the Outreach clinics (OR = 10.0 vs. 1.69, p < 0.001). Overall diabetic eye screening rates maintained an increase from 47.4% at baseline to 60.2% and 64.1% at 1 and 2 years post-I-SITE implementation, respectively (p < 0.001). Patients who were younger (OR = 0.98 per year of age, p = 0.02) and men (OR = 1.98, p = 0.002) were more likely to use teleophthalmology than in-person dilated eye examinations for diabetic eye screening.Discussion: Our stakeholder-based implementation program achieved a significant increase in overall teleophthalmology use and maintained increased post-teleophthalmology diabetic eye screening rates. Conclusion: Stakeholder-based implementation may increase the long-term reach and effectiveness of teleophthalmology to reduce vision loss from diabetes. Our approach may improve integration of telehealth interventions into primary care.

A practical model for research with learning health systems: Building and implementing effective complex case management.

For researchers to contribute meaningfully to the creation of learning health systems, practical tools are required to operationalize existing conceptual frameworks. We describe a model currently in use by the University of Wisconsin Health Innovation Program (HIP). The HIP model consolidates and enhances existing learning health system frameworks by defining specific steps needed to create sustainable change based on research conducted within the health system. As an example of the model's application, we describe its use to improve patient identification for the University of Wisconsin health system's case management program. Our case study shows the importance of culture, infrastructure, and strong leadership support in realizing a learning health systems research project and creating sustainable change within the health system. By articulating the foundational elements and steps to conduct research with learning health systems, our model supports researchers in achieving the challenge of moving learning health systems from concept to action.

The importance of health insurance claims data in creating learning health systems: evaluating care for high-need high-cost patients using the National Patient-Centered Clinical Research Network (PCORNet).

OBJECTIVE: Case management programs for high-need high-cost patients are spreading rapidly among health systems. PCORNet has substantial potential to support learning health systems in rapidly evaluating these programs, but access to complete patient data on health care utilization is limited as PCORNet is based on electronic health records not health insurance claims data. Because matching cases to comparison patients on baseline utilization is often a critical component of high-quality observational comparative effectiveness research for high-need high-cost patients, limited access to claims may negatively affect the quality of the matching process. We sought to determine whether the evaluation of programs for high-need high-cost patients required claims data to match cases to comparison patients. MATERIALS AND METHODS: A retrospective cohort study design with multiple measures of before-and-after health care utilization for 1935 case management patients and 3833 matched comparison patients aged 18 years and older from 2011 to 2015. EHR and claims data were extracted from 3 health systems participating in PCORNet. RESULTS: Without matching on claims-based health care utilization, the case management programs at 2 of 3 health systems were associated with fewer hospital admissions and emergency visits over the subsequent 12 months. With matching on claims-based health care utilization, case management was no longer associated with admissions and emergency visits at those 2 programs. DISCUSSION: The results of a PCORNet-facilitated evaluation of 3 programs for high-need high-cost patients differed substantially depending on whether claims data were available for matching cases to comparison patients. CONCLUSIONS: Partnering with learning health systems to rapidly evaluate programs for high-need high-cost patients will require that PCORNet facilitates comprehensive and timely access to both electronic health records and health insurance claims data.

Training and Interpreting Machine Learning Algorithms to Evaluate Fall Risk After Emergency Department Visits.

BACKGROUND: Machine learning is increasingly used for risk stratification in health care. Achieving accurate predictive models do not improve outcomes if they cannot be translated into efficacious intervention. Here we examine the potential utility of automated risk stratification and referral intervention to screen older adults for fall risk after emergency department (ED) visits. OBJECTIVE: This study evaluated several machine learning methodologies for the creation of a risk stratification algorithm using electronic health record data and estimated the effects of a resultant intervention based on algorithm performance in test data. METHODS: Data available at the time of ED discharge were retrospectively collected and separated into training and test datasets. Algorithms were developed to predict the outcome of a return visit for fall within 6 months of an ED index visit. Models included random forests, AdaBoost, and regression-based methods. We evaluated models both by the area under the receiver operating characteristic (ROC) curve, also referred to as area under the curve (AUC), and by projected clinical impact, estimating number needed to treat (NNT) and referrals per week for a fall risk intervention. RESULTS: The random forest model achieved an AUC of 0.78, with slightly lower performance in regression-based models. Algorithms with similar performance, when evaluated by AUC, differed when placed into a clinical context with the defined task of estimated NNT in a real-world scenario. CONCLUSION: The ability to translate the results of our analysis to the potential tradeoff between referral numbers and NNT offers decisionmakers the ability to envision the effects of a proposed intervention before implementation.

Identification of barriers, facilitators and system-based implementation strategies to increase teleophthalmology use for diabetic eye screening in a rural US primary care clinic: a qualitative study.

OBJECTIVE: Teleophthalmology for diabetic eye screening is an evidence-based intervention substantially underused in US multipayer primary care clinics, even when equipment and trained personnel are readily available. We sought to identify patient and primary care provider (PCP) barriers, facilitators, as well as strategies to increase teleophthalmology use. DESIGN: We conducted standardised open-ended, individual interviews and analysed the transcripts using both inductive and directed content analysis to identify barriers and facilitators to teleophthalmology use. The Chronic Care Model was used as a framework for the development of the interview guide and for categorising implementation strategies to increase teleophthalmology use. SETTING: A rural, US multipayer primary care clinic with an established teleophthalmology programme for diabetic eye screening. PARTICIPANTS: We conducted interviews with 29 participants (20 patients with diabetes and 9 PCPs). RESULTS: Major patient barriers to teleophthalmology use included being unfamiliar with teleophthalmology, misconceptions about diabetic eye screening and logistical challenges. Major patient facilitators included a recommendation from the patient's PCP and factors related to convenience. Major PCP barriers to referring patients for teleophthalmology included difficulty identifying when patients are due for diabetic eye screening and being unfamiliar with teleophthalmology. Major PCP facilitators included the ease of the referral process and the communication of screening results. Based on our results, we developed a model that maps where these key patient and PCP barriers occur in the teleophthalmology referral process. Patients and PCPs also identified implementation strategies to directly address barriers and facilitators to teleophthalmology use. CONCLUSIONS: Patients and PCPs have limited familiarity with teleophthalmology for diabetic eye screening. PCPs were expected to initiate teleophthalmology referrals, but reported significant difficulty identifying when patients are due for diabetic eye screening. System-based implementation strategies primarily targeting PCP barriers in conjunction with improved patient and provider education may increase teleophthalmology use in rural, US multipayer primary care clinics.

Rationale and study design of the MyHEART study: A young adult hypertension self-management randomized controlled trial.

Young adults (18-39 year-olds) with hypertension have a higher lifetime risk for cardiovascular disease. However, less than 50% of young adults achieve hypertension control in the United States. Hypertension self-management programs are recommended to improve control, but have been targeted to middle-aged and older populations. Young adults need hypertension self-management programs (i.e., home blood pressure monitoring and lifestyle modifications) tailored to their unique needs to lower blood pressure and reduce the risks and medication burden they may face over a lifetime. To address the unmet need in hypertensive care for young adults, we developed MyHEART (My Hypertension Education And Reaching Target), a multi-component, theoretically-based intervention designed to achieve self-management among young adults with uncontrolled hypertension. MyHEART is a patient-centered program, based upon the Self-Determination Theory, that uses evidence-based health behavior approaches to lower blood pressure. Therefore, the objective of this study is to evaluate MyHEART's impact on changes in systolic and diastolic blood pressure compared to usual care after 6 and 12 months in 310 geographically and racially/ethnically diverse young adults with uncontrolled hypertension. Secondary outcomes include MyHEART's impact on behavioral outcomes at 6 and 12 months, compared to usual clinical care (increased physical activity, decreased sodium intake) and to examine whether MyHEART's effects on self-management behavior are mediated through variables of perceived competence, autonomy, motivation, and activation (mediation outcomes). MyHEART is one of the first multicenter, randomized controlled hypertension trials tailored to young adults with primary care. The design and methodology will maximize the generalizability of this study. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT03158051.