Mind the Gap: Putting Evidence into Practice in the Era of Learning Health Systems.

Due to the increasing amount of available published evidence and the continual need to apply and update evidence in practice, we propose a shift in the way evidence generated by learning health systems can be integrated into more traditional evidence reviews. This paper discusses two main mechanisms to close the evidence-to-practice gap: (1) integrating Learning Health System (LHS) results with existing systematic review evidence and (2) providing this combined evidence in a standardized, computable data format. We believe these efforts will better inform practice, thereby improving individual and population health.

Early inpatient calculation of laboratory-based 30-day readmission risk scores empowers clinical risk modification during index hospitalization.

Improving 30-day readmission continues to be problematic for most hospitals. This study reports the creation and validation of sex-specific inpatient (i) heart failure (HF) risk scores using electronic data from the beginning of inpatient care for effective and efficient prediction of 30-day readmission risk. METHODS: HF patients hospitalized at Intermountain Healthcare from 2005 to 2012 (derivation: n=6079; validation: n=2663) and Baylor Scott & White Health (North Region) from 2005 to 2013 (validation: n=5162) were studied. Sex-specific iHF scores were derived to predict post-hospitalization 30-day readmission using common HF laboratory measures and age. Risk scores adding social, morbidity, and treatment factors were also evaluated. RESULTS: The iHF model for females utilized potassium, bicarbonate, blood urea nitrogen, red blood cell count, white blood cell count, and mean corpuscular hemoglobin concentration; for males, components were B-type natriuretic peptide, sodium, creatinine, hematocrit, red cell distribution width, and mean platelet volume. Among females, odds ratios (OR) were OR=1.99 for iHF tertile 3 vs. 1 (95% confidence interval [CI]=1.28, 3.08) for Intermountain validation (P-trend across tertiles=0.002) and OR=1.29 (CI=1.01, 1.66) for Baylor patients (P-trend=0.049). Among males, iHF had OR=1.95 (CI=1.33, 2.85) for tertile 3 vs. 1 in Intermountain (P-trend <0.001) and OR=2.03 (CI=1.52, 2.71) in Baylor (P-trend < 0.001). Expanded models using 182-183 variables had predictive abilities similar to iHF. CONCLUSIONS: Sex-specific laboratory-based electronic health record-delivered iHF risk scores effectively predicted 30-day readmission among HF patients. Efficient to calculate and deliver to clinicians, recent clinical implementation of iHF scores suggest they are useful and useable for more precise clinical HF treatment.

A Formative Evaluation of a Diabetes Prevention Program Using the RE-AIM Framework in a Learning Health Care System, Utah, 2013-2015.

INTRODUCTION: Evaluation of interventions can help to close the gap between research and practice but seldom takes place during implementation. Using the RE-AIM framework, we conducted a formative evaluation of the first year of the Intermountain Healthcare Diabetes Prevention Program (DPP). METHODS: Adult patients who met the criteria for prediabetes (HbA1c of 5.70%-6.49% or fasting plasma glucose of 100-125 mg/dL) were attributed to a primary care provider from August 1, 2013, through July 31, 2014. Physicians invited eligible patients to participate in the program during an office visit. We evaluated 1) reach, with data on patient eligibility, participation, and representativeness; 2) effectiveness, with data on attaining a 5% weight loss; 3) adoption, with data on providers and clinics that referred patients to the program; and 4) implementation, with data on patient encounters. We did not measure maintenance. RESULTS: Of the 6,862 prediabetes patients who had an in-person office visit with their provider, 8.4% of eligible patients enrolled. Likelihood of participation was higher among patients who were female, aged 70 years or older, or overweight; had depression and higher weight at study enrollment; or were prescribed metformin. DPP participants were more likely than nonparticipants to achieve a 5% weight loss (odds ratio, 1.70; 95% confidence interval, 1.29-2.25; P < .001). Providers from 7 of 8 regions referred patients to the DPP; 174 providers at 53 clinics enrolled patients. The mean number of DPP counseling encounters per patient was 2.3 (range, 1-16). CONCLUSION: The RE-AIM framework was useful for estimating the formative impact (ie, reach, effectiveness, adoption, and implementation fidelity) of a DPP-based lifestyle intervention deployed in a learning health care system.

Healthcare utilization in the first year after pediatric traumatic brain injury in an insured population.

OBJECTIVE: To compare the healthcare use by children with and without a traumatic brain injury (TBI) in the year following injury to understand whether children access primary care. PARTICIPANTS: Children 0 to 15 years with a TBI (N = 545) and (N = 2310) uninjured age and sex-matched comparisons. SETTING: A full benefits healthcare plan from 2000 to 2007. MAIN MEASURES: Mean annual healthcare utilization. RESULTS: Children with TBI had higher mean annual outpatient visits (4.2 vs. 3.5, P = .001), but similar mean annual general pediatric visits (2.7 vs. 2.8, P = .3) than comparison children. More cases than comparisons attended a general pediatric visit (80.0% vs. 73.3%, risk ratio = 1.1, 95% CI: 1.0-1.1). However, approximately 50% of children older than 7 years who had an intracranial injury did not attend a general pediatric visit and those were slightly more likely to receive specialty care (adjusted risk ratio = 1.1, 95% CI: 1.0-1.2). These children did not appear to be substituting specialty for primary care. CONCLUSIONS: Children with a full benefits insurance plan do not access primary care routinely after TBI. These findings present a challenge for designing a system to screen children after TBI.

Volume-related differences in emergency department performance.

BACKGROUND: Emergency departments (EDs) are an important source of care for a large segment of the population of the United States. In 2009 there were more than 136 million visits to the ED each year, and more than half of hospital admissions begin in the ED. Measurement and monitoring of emergency department performance has been prompted by The Joint Commission's patient flow standards. A study was conducted to attempt to correlate ED volume and other operating characteristics with performance on metrics. METHODS: A retrospective analysis of the Emergency Department Benchmarking Alliance annual ED survey data for the most recent year for which data were available (2009) was performed to explore observed patterns in ED performance relative to size and operating characteristics. The survey was based on 14.6 million ED visits in 358 hospitals across the United States, with an ED size representation (sampling) approximating that of the Emergency Medicine Network (EM Net). RESULTS: Larger EDs (with higher annual volumes) had longer lengths of stay (p < .0001), higher left without being seen rates (p < .0001), and longer door-to-physician times (p < .0001), all suggesting poorer operational performance. Operating characteristics indicative of higher acuity were associated with worsened performance on metrics and lower acuity characteristics with improved performance. CONCLUSION: ED volume, which also correlates with many operating characteristics, is the strongest predictor of operational performance on metrics and can be used to categorize EDs for comparative analysis. Operating characteristics indicative of acuity also influence performance. The findings suggest that ED performance measures should take ED volume, acuity, and other characteristics into account and that these features have important implications for ED design, operations, and policy decisions.

"It's Not Just the Right Thing . . . It's a Survival Tactic": Disentangling Leaders' Motivations and Worries on Social Care.

Health care organizations face growing pressure to improve their patients' social conditions, such as housing, food, and economic insecurity. Little is known about the motivations and concerns of health care organizations when implementing activities aimed at improving patients' social conditions. We used semi-structured interviews with 29 health care organizations to explore their motivations and tensions around social care. Administrators described an interwoven set of motivations for delivering social care: (a) doing the right thing for their patients, (b) improving health outcomes, and (c) making the business case. Administrators expressed tensions around the optimal role for health care in social care including uncertainty around (a) who should be responsible, (b) whether health care has the needed capacity/skills, and (c) sustainability of social care activities. Health care administrators could use guidance and support from policy makers on how to effectively prioritize social care activities, partner with other sectors, and build the needed workforce.

Identifying appropriate comparison groups for health system interventions in the COVID-19 era.

Introduction: COVID-19 has created additional challenges for the analysis of non-randomized interventions in health system settings. Our objective is to evaluate these challenges and identify lessons learned from the analysis of a medically tailored meals (MTM) intervention at Kaiser Permanente Northwest (KPNW) that began in April 2020. Methods: We identified both a historical and concurrent comparison group. The historical comparison group included patients living in the same area as the MTM recipients prior to COVID-19. The concurrent comparison group included patients admitted to contracted non-KPNW hospitals or admitted to a KPNW facility and living outside the service area for the intervention but otherwise eligible. We used two alternative propensity score methods in response to the loss of sample size with exact matching to evaluate the intervention. Results: We identified 452 patients who received the intervention, 3873 patients in the historical comparison group, and 5333 in the concurrent comparison group. We were able to mostly achieve balance on observable characteristics for the intervention and the two comparison groups. Conclusions: Lessons learned included: (a) The use of two different comparison groups helped to triangulate results; (b) the meaning of utilization measures changed pre- and post-COVID-19; and (c) that balance on observable characteristics can be achieved, especially when the comparison groups are meaningfully larger than the intervention group. These findings may inform the design for future evaluations of interventions during COVID-19.

Training the next generation of learning health system scientists.

Introduction: The learning health system (LHS) aligns science, informatics, incentives, stakeholders, and culture for continuous improvement and innovation. The Agency for Healthcare Research and Quality and the Patient-Centered Outcomes Research Institute designed a K12 initiative to grow the number of LHS scientists. We describe approaches developed by 11 funded centers of excellence (COEs) to promote partnerships between scholars and health system leaders and to provide mentored research training. Methods: Since 2018, the COEs have enlisted faculty, secured institutional resources, partnered with health systems, developed and implemented curricula, recruited scholars, and provided mentored training. Program directors for each COE provided descriptive data on program context, scholar characteristics, stakeholder engagement, scholar experiences with health system partnerships, roles following program completion, and key training challenges. Results: To date, the 11 COEs have partnered with health systems to train 110 scholars. Nine (82%) programs partner with a Veterans Affairs health system and 9 (82%) partner with safety net providers. Clinically trained scholars (n = 87; 79%) include 70 physicians and 17 scholars in other clinical disciplines. Non-clinicians (n = 29; 26%) represent diverse fields, dominated by population health sciences. Stakeholder engagement helps scholars understand health system and patient/family needs and priorities, enabling opportunities to conduct embedded research, improve outcomes, and grow skills in translating research methods and findings into practice. Challenges include supporting scholars through roadblocks that threaten to derail projects during their limited program time, ranging from delays in access to data to COVID-19-related impediments and shifts in organizational priorities. Conclusions: Four years into this novel training program, there is evidence of scholars' accomplishments, both in traditional academic terms and in terms of moving along career trajectories that hold the potential to lead and accelerate transformational health system change. Future LHS training efforts should focus on sustainability, including organizational support for scholar activities.

Emerging models of care for individuals with multiple chronic conditions.

OBJECTIVE: To characterize emerging and current practice models to more effectively treat and support patients with multiple chronic conditions (MCC). DATA SOURCES/STUDY SETTING: We conducted a rapid literature scoping augmented by key informant interviews with clinicians knowledgeable about MCC care from a broad spectrum of US delivery systems and feedback from multidisciplinary experts at two virtual meetings. STUDY DESIGN: Literature findings were triangulated with data from semi-structured interviews with clinical experts. Reflections on early results were obtained from policy, research, clinical, advocacy, and patient representatives at two virtual meetings sponsored by the Agency for Healthcare Research and Quality. Emergent themes addressed were as follows: (1) more timely strategies for MCC care; and (2) trends not previously represented in the peer-reviewed literature. DATA COLLECTION/EXTRACTION METHODS: The rapid literature scoping relied on Ovid MEDLINE(R) and Epub Ahead of Print databases for the most recent 5-year period. Qualitative interviews were conducted by telephone. Virtual meetings provided oral and written (chat) captured inputs. PRINCIPAL FINDINGS: Although the literature scoping did not identify a specific set of evidence-based care models, key informant discussions identified eight themes reflecting emerging approaches to population-based MCC care. For example, addressing the needs of individuals with MCC through a complexity lens by assessing and addressing social risk factors; extending the care continuum with home-based care; understanding how to address ongoing patient and caregiver supports outside of clinical encounters; and engaging available community resources. CONCLUSIONS: Integrating care for MCC patient populations requires processes for determining different subpopulation needs in various settings and lived experiences. Innovation should be anchored at the nexus of payment systems, social risks, medical needs, and community-based resources. Our learnings suggest a need for an ongoing MCC care research agenda to inform new approaches to care delivery incorporating innovations in technology and home-based supports for patients and caregivers.

Clindamycin-resistant group B Streptococcus and failure of intrapartum prophylaxis to prevent early-onset disease.

Guidelines recommend intrapartum antibiotic prophylaxis (IAP) for parturient women who have a screen positive for group B Streptococcus (GBS). Clindamycin should be used for IAP only if the maternal GBS isolate is susceptible. We report a case of clindamycin-resistant GBS disease in a newborn infant whose mother received clindamycin IAP, and we review clindamycin susceptibility testing.

Cost and quality impact of Intermountain's mental health integration program.

Most patients with mental health (MH) conditions, such as depression, receive care for their conditions from a primary care physician (PCP) in their health/medical home. Providing MH care, however, presents many challenges for the PCP, including (1) the difficulty of getting needed consultation from an MH specialist; (2) the time constraints of a busy PCP practice; (3) the complicated nature of recognizing depression, which may be described with only somatic complaints; (4) the barriers to reimbursement and compensation; and (5) associated medical and social comorbidities. Practice managers, emergency departments, and health plans are stretched to provide care for complex patients with unmet MH needs. At the same time, payment reform linked to accountable care organizations and/or episodic bundle payments, MH parity rules, and increasing MH costs to large employers and payers all highlight the critical need to identify high-quality, efficient, integrated MH care delivery practices. Over the past ten years, Intermountain Healthcare has developed a team-based approach-known as mental health integration (MHI)-for caring for these patients and their families. The team includes the PCPs and their staff, and they, in turn, are integrated with MH professionals, community resources, care management, and the patient and his or her family. The integration model goes far beyond co-location in its team-based approach; it is operationalized at the clinic, thereby improving both physician and staff satisfaction. Patients treated in MHI clinics also show improved satisfaction, lower costs, and better quality outcomes. The MHI program is financially sustainable in routinized clinics without subsidies. MHI is a successful approach to improving care for patients with MH conditions in primary care health homes.

Cost cutting in health systems without compromising quality care.

Intermountain Healthcare is a high-performing health system and a recognized leader in quality improvement. We use a clinical integration strategy focused on eight clinical programs to support the practice of evidence-based care. Accelerated improvements that enhance patient safety, clinical excellence, and operational efficiency are tested and then spread across the system via care process models and program-specific board goals. While we have nearly 60 evidence-based care process models in place (in addition to multiple operational effectiveness initiatives), we provide three exemplars to illustrate cost savings and the relative impact on hospital/medical group versus payer benefit. These clinical best practices include very early lung recruitment (VE LR) for neonates with respiratory distress syndrome, guidelines for elective inductions in labor and delivery, and prevention of congestive heart failure (CHF) readmissions. Due to perverse incentives in the third party payment system--where healthcare providers are often paid to do more tests and treatments as opposed to providing clinical value--doing what's right for our patients commonly yields savings to our payers while negatively impacting the delivery system budget. In this article, we present a suggested strategy for negotiated capture of these savings.

How much can we trust electronic health record data?

Trust in EHR data is becoming increasingly important as a greater share of clinical and health services research use EHR data. We discuss reasons for distrust and acknowledge limitations. Researchers continue to use EHR data because of strengths including greater clinical detail than sources like administrative billing claims. Further, many limitations are addressable with existing methods including data quality checks and common data frameworks. We discuss how to build greater trust in the use of EHR data for research, including additional transparency and research priority areas that will both enhance existing strengths of the EHR and mitigate its limitations.

Improvement in Mortality With Early Fluid Bolus in Sepsis Patients With a History of Congestive Heart Failure.

OBJECTIVE: To determine whether rapid administration of a crystalloid bolus of 30 mL/kg within 3 hours of presentation harms or benefits hypotensive patients with sepsis with a history of congestive heart failure (CHF). PATIENTS AND METHODS: A retrospective cohort study using Medicare claims data enhanced by medical record data from members of the High Value Healthcare Collaborative from July 1, 2013, to June 30, 2015, examining patients with a history of CHF who did (fluid bundle compliant [FBC]) or did not (NFBC) receive a volume bolus of 30 mL/kg within 3 hours of presentation to the emergency department. A proportional Cox hazard model was used to evaluate the association of FBC with 1-year survival. RESULTS: Of the 211 patients examined, 190 were FBC and 21 were NFBC. The FBC patients had higher average hierarchical condition category scores but were otherwise similar to NFBC patients. The NFBC patients had higher adjusted in-hospital and postdischarge mortality rates. The risk-adjusted 1-year mortality rate was higher for NFBC patients (hazard ratio, 2.18; 95% CI, 1.2 to 4.0; P=.01) than for FBC patients. CONCLUSION: In a retrospective claim data-based study of elderly patients with a history of CHF presenting with severe sepsis or septic shock, there is an association of improved mortality with adherence to the initial fluid resuscitation guidelines as part of the 3-hour sepsis bundle.

Decision Aid Implementation and Patients' Preferences for Hip and Knee Osteoarthritis Treatment: Insights from the High Value Healthcare Collaborative.

BACKGROUND: Shared decision making (SDM) research has emphasized the role of decision aids (DAs) for helping patients make treatment decisions reflective of their preferences, yet there have been few collaborative multi-institutional efforts to integrate DAs in orthopedic consultations and primary care encounters. OBJECTIVE: In the context of routine DA implementation for SDM, we investigate which patient-level characteristics are associated with patient preferences for surgery versus medical management before and after exposure to DAs. We explored whether DA implementation in primary care encounters was associated with greater shifts in patients' treatment preferences after exposure to DAs compared to DA implementation in orthopedic consultations. DESIGN: Retrospective cohort study. SETTING: 10 High Value Healthcare Collaborative (HVHC) health systems. STUDY PARTICIPANTS: A total of 495 hip and 1343 adult knee osteoarthritis patients who were exposed to DAs within HVHC systems between July 2012 to June 2015. RESULTS: Nearly 20% of knee patients and 17% of hip patients remained uncertain about their treatment preferences after viewing DAs. Older patients and patients with high pain levels had an increased preference for surgery. Older patients receiving DAs from three HVHC systems that transitioned DA implementation from orthopedics into primary care had lower odds of preferring surgery after DA exposure compared to older patients in seven HVHC systems that only implemented DAs for orthopedic consultations. CONCLUSION: Patients' treatment preferences were largely stable over time, highlighting that DAs for SDM largely do not necessarily shift preferences. DAs and SDM processes should be targeted at older adults and patients reporting high pain levels. Initiating treatment conversations in primary versus specialty care settings may also have important implications for engagement of patients in SDM via DAs.

Improving Health Care with Advanced Analytics: Practical Considerations.

Artificial intelligence (AI) is becoming ubiquitous in health care, largely through machine learning and predictive analytics applications. Recent applications of AI to common health care scenarios, such as screening and diagnosing, have fueled optimism about the use of advanced analytics to improve care. Careful and objective considerations need to be made before implementing an advanced analytics solution. Critical evaluation before, during, and after its implementation will ensure safe care, good outcomes, and the elimination of waste. In this commentary we offer basic practical considerations for developing, implementing, and evaluating such solutions based on many years of experience.