Training the next generation of learning health system scientists.

Introduction: The learning health system (LHS) aligns science, informatics, incentives, stakeholders, and culture for continuous improvement and innovation. The Agency for Healthcare Research and Quality and the Patient-Centered Outcomes Research Institute designed a K12 initiative to grow the number of LHS scientists. We describe approaches developed by 11 funded centers of excellence (COEs) to promote partnerships between scholars and health system leaders and to provide mentored research training. Methods: Since 2018, the COEs have enlisted faculty, secured institutional resources, partnered with health systems, developed and implemented curricula, recruited scholars, and provided mentored training. Program directors for each COE provided descriptive data on program context, scholar characteristics, stakeholder engagement, scholar experiences with health system partnerships, roles following program completion, and key training challenges. Results: To date, the 11 COEs have partnered with health systems to train 110 scholars. Nine (82%) programs partner with a Veterans Affairs health system and 9 (82%) partner with safety net providers. Clinically trained scholars (n = 87; 79%) include 70 physicians and 17 scholars in other clinical disciplines. Non-clinicians (n = 29; 26%) represent diverse fields, dominated by population health sciences. Stakeholder engagement helps scholars understand health system and patient/family needs and priorities, enabling opportunities to conduct embedded research, improve outcomes, and grow skills in translating research methods and findings into practice. Challenges include supporting scholars through roadblocks that threaten to derail projects during their limited program time, ranging from delays in access to data to COVID-19-related impediments and shifts in organizational priorities. Conclusions: Four years into this novel training program, there is evidence of scholars' accomplishments, both in traditional academic terms and in terms of moving along career trajectories that hold the potential to lead and accelerate transformational health system change. Future LHS training efforts should focus on sustainability, including organizational support for scholar activities.

Identifying appropriate comparison groups for health system interventions in the COVID-19 era.

Introduction: COVID-19 has created additional challenges for the analysis of non-randomized interventions in health system settings. Our objective is to evaluate these challenges and identify lessons learned from the analysis of a medically tailored meals (MTM) intervention at Kaiser Permanente Northwest (KPNW) that began in April 2020. Methods: We identified both a historical and concurrent comparison group. The historical comparison group included patients living in the same area as the MTM recipients prior to COVID-19. The concurrent comparison group included patients admitted to contracted non-KPNW hospitals or admitted to a KPNW facility and living outside the service area for the intervention but otherwise eligible. We used two alternative propensity score methods in response to the loss of sample size with exact matching to evaluate the intervention. Results: We identified 452 patients who received the intervention, 3873 patients in the historical comparison group, and 5333 in the concurrent comparison group. We were able to mostly achieve balance on observable characteristics for the intervention and the two comparison groups. Conclusions: Lessons learned included: (a) The use of two different comparison groups helped to triangulate results; (b) the meaning of utilization measures changed pre- and post-COVID-19; and (c) that balance on observable characteristics can be achieved, especially when the comparison groups are meaningfully larger than the intervention group. These findings may inform the design for future evaluations of interventions during COVID-19.

"It's Not Just the Right Thing . . . It's a Survival Tactic": Disentangling Leaders' Motivations and Worries on Social Care.

Health care organizations face growing pressure to improve their patients' social conditions, such as housing, food, and economic insecurity. Little is known about the motivations and concerns of health care organizations when implementing activities aimed at improving patients' social conditions. We used semi-structured interviews with 29 health care organizations to explore their motivations and tensions around social care. Administrators described an interwoven set of motivations for delivering social care: (a) doing the right thing for their patients, (b) improving health outcomes, and (c) making the business case. Administrators expressed tensions around the optimal role for health care in social care including uncertainty around (a) who should be responsible, (b) whether health care has the needed capacity/skills, and (c) sustainability of social care activities. Health care administrators could use guidance and support from policy makers on how to effectively prioritize social care activities, partner with other sectors, and build the needed workforce.

Emerging models of care for individuals with multiple chronic conditions.

OBJECTIVE: To characterize emerging and current practice models to more effectively treat and support patients with multiple chronic conditions (MCC). DATA SOURCES/STUDY SETTING: We conducted a rapid literature scoping augmented by key informant interviews with clinicians knowledgeable about MCC care from a broad spectrum of US delivery systems and feedback from multidisciplinary experts at two virtual meetings. STUDY DESIGN: Literature findings were triangulated with data from semi-structured interviews with clinical experts. Reflections on early results were obtained from policy, research, clinical, advocacy, and patient representatives at two virtual meetings sponsored by the Agency for Healthcare Research and Quality. Emergent themes addressed were as follows: (1) more timely strategies for MCC care; and (2) trends not previously represented in the peer-reviewed literature. DATA COLLECTION/EXTRACTION METHODS: The rapid literature scoping relied on Ovid MEDLINE(R) and Epub Ahead of Print databases for the most recent 5-year period. Qualitative interviews were conducted by telephone. Virtual meetings provided oral and written (chat) captured inputs. PRINCIPAL FINDINGS: Although the literature scoping did not identify a specific set of evidence-based care models, key informant discussions identified eight themes reflecting emerging approaches to population-based MCC care. For example, addressing the needs of individuals with MCC through a complexity lens by assessing and addressing social risk factors; extending the care continuum with home-based care; understanding how to address ongoing patient and caregiver supports outside of clinical encounters; and engaging available community resources. CONCLUSIONS: Integrating care for MCC patient populations requires processes for determining different subpopulation needs in various settings and lived experiences. Innovation should be anchored at the nexus of payment systems, social risks, medical needs, and community-based resources. Our learnings suggest a need for an ongoing MCC care research agenda to inform new approaches to care delivery incorporating innovations in technology and home-based supports for patients and caregivers.

Improvement in Mortality With Early Fluid Bolus in Sepsis Patients With a History of Congestive Heart Failure.

OBJECTIVE: To determine whether rapid administration of a crystalloid bolus of 30 mL/kg within 3 hours of presentation harms or benefits hypotensive patients with sepsis with a history of congestive heart failure (CHF). PATIENTS AND METHODS: A retrospective cohort study using Medicare claims data enhanced by medical record data from members of the High Value Healthcare Collaborative from July 1, 2013, to June 30, 2015, examining patients with a history of CHF who did (fluid bundle compliant [FBC]) or did not (NFBC) receive a volume bolus of 30 mL/kg within 3 hours of presentation to the emergency department. A proportional Cox hazard model was used to evaluate the association of FBC with 1-year survival. RESULTS: Of the 211 patients examined, 190 were FBC and 21 were NFBC. The FBC patients had higher average hierarchical condition category scores but were otherwise similar to NFBC patients. The NFBC patients had higher adjusted in-hospital and postdischarge mortality rates. The risk-adjusted 1-year mortality rate was higher for NFBC patients (hazard ratio, 2.18; 95% CI, 1.2 to 4.0; P=.01) than for FBC patients. CONCLUSION: In a retrospective claim data-based study of elderly patients with a history of CHF presenting with severe sepsis or septic shock, there is an association of improved mortality with adherence to the initial fluid resuscitation guidelines as part of the 3-hour sepsis bundle.

How much can we trust electronic health record data?

Trust in EHR data is becoming increasingly important as a greater share of clinical and health services research use EHR data. We discuss reasons for distrust and acknowledge limitations. Researchers continue to use EHR data because of strengths including greater clinical detail than sources like administrative billing claims. Further, many limitations are addressable with existing methods including data quality checks and common data frameworks. We discuss how to build greater trust in the use of EHR data for research, including additional transparency and research priority areas that will both enhance existing strengths of the EHR and mitigate its limitations.

Decision Aid Implementation and Patients' Preferences for Hip and Knee Osteoarthritis Treatment: Insights from the High Value Healthcare Collaborative.

BACKGROUND: Shared decision making (SDM) research has emphasized the role of decision aids (DAs) for helping patients make treatment decisions reflective of their preferences, yet there have been few collaborative multi-institutional efforts to integrate DAs in orthopedic consultations and primary care encounters. OBJECTIVE: In the context of routine DA implementation for SDM, we investigate which patient-level characteristics are associated with patient preferences for surgery versus medical management before and after exposure to DAs. We explored whether DA implementation in primary care encounters was associated with greater shifts in patients' treatment preferences after exposure to DAs compared to DA implementation in orthopedic consultations. DESIGN: Retrospective cohort study. SETTING: 10 High Value Healthcare Collaborative (HVHC) health systems. STUDY PARTICIPANTS: A total of 495 hip and 1343 adult knee osteoarthritis patients who were exposed to DAs within HVHC systems between July 2012 to June 2015. RESULTS: Nearly 20% of knee patients and 17% of hip patients remained uncertain about their treatment preferences after viewing DAs. Older patients and patients with high pain levels had an increased preference for surgery. Older patients receiving DAs from three HVHC systems that transitioned DA implementation from orthopedics into primary care had lower odds of preferring surgery after DA exposure compared to older patients in seven HVHC systems that only implemented DAs for orthopedic consultations. CONCLUSION: Patients' treatment preferences were largely stable over time, highlighting that DAs for SDM largely do not necessarily shift preferences. DAs and SDM processes should be targeted at older adults and patients reporting high pain levels. Initiating treatment conversations in primary versus specialty care settings may also have important implications for engagement of patients in SDM via DAs.

Contextual Factors Influencing Implementation of Evidence-Based Care for Children Hospitalized With Asthma.

BACKGROUND AND OBJECTIVES: The translation of research findings into routine care remains slow and challenging. We previously reported successful implementation of an asthma evidence-based care process model (EB-CPM) at 8 (1 tertiary care and 7 community) hospitals, leading to a high health care provider (HCP) adherence with the EB-CPM and improved outcomes. In this study, we explore contextual factors perceived by HCPs to facilitate successful EB-CPM implementation. METHODS: Structured and open-ended questions were used to survey HCPs (n = 260) including physicians, nurses, and respiratory therapists, about contextual factors perceived to facilitate EB-CPM implementation. Quantitative analysis was used to identify significant factors (correlation coefficient ≥0.5; P ≤ .05) and qualitative analysis to assess additional facilitators. RESULTS: Factors perceived by HCPs to facilitate EB-CPM implementation were related to (1) inner setting (leadership support, adequate resources, communication and/or collaboration, culture, and previous experience with guideline implementation), (2) intervention characteristics (relevant and applicable to the HCP's practice), (3) individuals (HCPs) targeted (agreement with the EB-CPM and knowledge of supporting evidence), and (4) implementation process (participation of HCPs in implementation activities, teamwork, implementation team with a mix of expertise and professional's input, and data feedback). Additional facilitators included (1) having appropriate preparation and (2) providing education and training. CONCLUSIONS: Multiple factors were associated with successful EB-CPM implementation and may be used by others as a guide to facilitate implementation and dissemination of evidence-based interventions for pediatric asthma and other chronic diseases in the hospital setting.

Improving Health Care with Advanced Analytics: Practical Considerations.

Artificial intelligence (AI) is becoming ubiquitous in health care, largely through machine learning and predictive analytics applications. Recent applications of AI to common health care scenarios, such as screening and diagnosing, have fueled optimism about the use of advanced analytics to improve care. Careful and objective considerations need to be made before implementing an advanced analytics solution. Critical evaluation before, during, and after its implementation will ensure safe care, good outcomes, and the elimination of waste. In this commentary we offer basic practical considerations for developing, implementing, and evaluating such solutions based on many years of experience.

The Effect Of The Hospital Readmissions Reduction Program On Readmission And Observation Stay Rates For Heart Failure.

The Hospital Readmissions Reduction Program reduces Medicare prospective payments for hospitals with excess readmissions for selected diagnoses. By comparing data for patients who were readmitted or placed on observation status immediately before and immediately after the thirty-day cutoff for penalties, we sought to determine whether hospitals have responded to the program by shifting readmissions for heart failure to observation status. We used regression discontinuity, taking advantage of the cutoff to generate unbiased estimates of treatment effects. Overall, we found no evidence that the program has affected the use of observation stays. However, for nonpenalized hospitals, the use of observation status was 5.4 percent higher for patients returning to the hospital immediately before the thirty-day cutoff than for patients returning immediately after the cutoff, which suggests that some hospitals may have used observation status to help avoid penalties. Because differences in the cost-sharing rules may lead to higher out-of-pocket expenses for Medicare patients placed on observation status, the program could have an inequitable financial impact.

Mind the Gap: Putting Evidence into Practice in the Era of Learning Health Systems.

Due to the increasing amount of available published evidence and the continual need to apply and update evidence in practice, we propose a shift in the way evidence generated by learning health systems can be integrated into more traditional evidence reviews. This paper discusses two main mechanisms to close the evidence-to-practice gap: (1) integrating Learning Health System (LHS) results with existing systematic review evidence and (2) providing this combined evidence in a standardized, computable data format. We believe these efforts will better inform practice, thereby improving individual and population health.

Measuring the Effect of Social Determinants on Patient Outcomes: A Systematic Literature Review.

Given the movement towards value-based purchasing in the United States, health care leaders need methods to characterize and address the complex effect that social determinants have on health care outcomes. This systematic literature review was specifically designed to understand current research on the effect that patient material and social deprivation has on health care delivery outcomes and the potential benefit of clinical interventions designed to mediate this effect. A total of 310 studies were identified for review with 80 studies included in the final synthesis. Results highlight significant variation in the methods used to measure the effect of social determinants on health care outcomes and the need for common measurement standards. More robust identification of deprivation-sensitive diseases or conditions is needed to channel scarce program resources to effected conditions. Finally, further research is needed to evaluate the benefits of data-driven, tailored clinical interventions designed to serve the needs of materially-deprived patient populations.

Neighborhood Deprivation and Childhood Asthma Outcomes, Accounting for Insurance Coverage.

OBJECTIVES: Collecting social determinants data is challenging. We assigned patients a neighborhood-level social determinant measure, the area of deprivation index (ADI), by using census data. We then assessed the association between neighborhood deprivation and asthma hospitalization outcomes and tested the influence of insurance coverage. METHODS: A retrospective cohort study of children 2 to 17 years old admitted for asthma at 8 hospitals. An administrative database was used to collect patient data, including hospitalization outcomes and neighborhood deprivation status (ADI scores), which were grouped into quintiles (ADI 1, the least deprived neighborhoods; ADI 5, the most deprived neighborhoods). We used multivariable models, adjusting for covariates, to assess the associations and added a neighborhood deprivation status and insurance coverage interaction term. RESULTS: A total of 2270 children (median age 5 years; 40.6% girls) were admitted for asthma. We noted that higher ADI quintiles were associated with greater length of stay, higher cost, and more asthma readmissions (P < .05 for most quintiles). Having public insurance was independently associated with greater length of stay (ß: 1.171; 95% confidence interval [CI]: 1.117-1.228; P < .001), higher cost (ß: 1.147; 95% CI: 1.093-1.203; P < .001), and higher readmission odds (odds ratio: 1.81; 95% CI: 1.46-2.24; P < .001). There was a significant deprivation-insurance effect modification, with public insurance associated with worse outcomes and private insurance with better outcomes across ADI quintiles (P < .05 for most combinations). CONCLUSIONS: Neighborhood-level ADI measure is associated with asthma hospitalization outcomes. However, insurance coverage modifies this relationship and needs to be considered when using the ADI to identify and address health care disparities.

It takes a village: Exploring the impact of social determinants on delivery system outcomes for heart failure patients.

BACKGROUND: Local social determinants may act as effect modifiers for the impact of neighborhood material deprivation on patient-level healthcare outcomes. The objective of this study was to understand the mediating effect of local social determinants on neighborhood material deprivation and delivery outcomes in heart failure (HF) patients. MATERIAL AND METHODS: A retrospective cohort study was conducted using 4737 HF patients receiving inpatient care (n=6065 encounters) from an integrated healthcare delivery system from 2010 to 2014. Outcomes included post-discharge mortality, readmission risk and length of stay. Deprivation was measured using an area deprivation index by address of residence. Effect modifications measured included urban-rural residency and faith identification using generalized linear regression models. Patient-level data was drawn from the delivery system data warehouse. RESULTS: Faith identification had a significant protective effect on HF patients from deprived areas, lowering 30-day mortality odds by one-third over patients who did not identify with a faith (OR 0.35 95%CI:0.12-0.98;p=0.05). Significant effects persisted at the 90 and 180-day timeframes. In rural areas, lack of faith identification had a multiplicative effect on 30-day mortality for deprived patients (OR 14.0 95%CI:1.47-132.7;p=0.02). No significant effects were noted for other healthcare outcomes. CONCLUSIONS: The lack of expected association between area deprivation and healthcare outcomes in some communities may be explained by the presence of effect modifiers. IMPLICATIONS: Understanding existing effect modifiers for area deprivation in local communities that delivery systems serve can inform targeted quality improvement. These factors should also be considered when comparing delivery system performance for reimbursement and in population health management.

Stepping Back to Move Forward: Evaluating the Effectiveness of a Diabetes Prevention Program Within a Large Integrated Healthcare Delivery System.

OBJECTIVE: To evaluate the short-term effectiveness of the Intermountain Healthcare (IH) Diabetes Prevention Program (DPP) for patients with prediabetes (preDM) deployed within primary care clinics. STUDY DESIGN: A quasi-experimental study design was used to deploy the DPP within the IH system to identify patients with preDM and target a primary goal of a 5% weight loss within 6-12 months of enrollment. STUDY POPULATION: Adults (aged 18-75 years) who met the American Diabetes Association criteria for preDM were included for study. Patients who attended DPP counseling between August 2013 and July 2014 were considered as the intervention (or DPP) group. The DPP group was matched using propensity scores at a 1:4 ratio with a control group of patients with preDM who did not participate in DPP. RESULTS: Of the 17,142 patients who met the inclusion criteria for preDM, 40% had an in-person office visit with their provider. On average, patients were 58 years old, and greater than 60% were women. Based on multivariate logistic regression, the DPP group was more likely to achieve a 5% weight loss within 6-12 months after enrollment (OR = 1.70; 95% CI = 1.29-2.25; p < .001) when compared with the no-DPP group. CONCLUSIONS: Diabetes Prevention Program-based lifestyle interventions demonstrated significant reduction in body weight and incident Type 2 diabetes mellitus when compared with nonenrollees.

A Formative Evaluation of a Diabetes Prevention Program Using the RE-AIM Framework in a Learning Health Care System, Utah, 2013-2015.

INTRODUCTION: Evaluation of interventions can help to close the gap between research and practice but seldom takes place during implementation. Using the RE-AIM framework, we conducted a formative evaluation of the first year of the Intermountain Healthcare Diabetes Prevention Program (DPP). METHODS: Adult patients who met the criteria for prediabetes (HbA1c of 5.70%-6.49% or fasting plasma glucose of 100-125 mg/dL) were attributed to a primary care provider from August 1, 2013, through July 31, 2014. Physicians invited eligible patients to participate in the program during an office visit. We evaluated 1) reach, with data on patient eligibility, participation, and representativeness; 2) effectiveness, with data on attaining a 5% weight loss; 3) adoption, with data on providers and clinics that referred patients to the program; and 4) implementation, with data on patient encounters. We did not measure maintenance. RESULTS: Of the 6,862 prediabetes patients who had an in-person office visit with their provider, 8.4% of eligible patients enrolled. Likelihood of participation was higher among patients who were female, aged 70 years or older, or overweight; had depression and higher weight at study enrollment; or were prescribed metformin. DPP participants were more likely than nonparticipants to achieve a 5% weight loss (odds ratio, 1.70; 95% confidence interval, 1.29-2.25; P < .001). Providers from 7 of 8 regions referred patients to the DPP; 174 providers at 53 clinics enrolled patients. The mean number of DPP counseling encounters per patient was 2.3 (range, 1-16). CONCLUSION: The RE-AIM framework was useful for estimating the formative impact (ie, reach, effectiveness, adoption, and implementation fidelity) of a DPP-based lifestyle intervention deployed in a learning health care system.