Kaiser Permanente's Social Needs Network for Evaluation and Translation Empirical Research Agenda.

Introduction: Healthcare systems such as Kaiser Permanente are increasingly focusing on patients' social health. However, there is limited evidence to guide social health integration strategy. The purpose of this study was to identify social health research opportunities using a stakeholder-driven process. Methods: A modified Concept Mapping approach was implemented from June 2021 to February 2022. Stakeholders (n=746) received the prompt, "One thing I wish we knew more about to advance my work addressing social health..." An inductive content analysis approach was used to assign topics and synthesize and refine research-focused statements into research questions. Questions were then rated on impact and priority by researcher stakeholders (n=16). Mean impact and priority scores and an overall combined score were calculated. Question rankings were generated using the combined score. Results: Brainstorming produced 148 research-focused statements. A final list of 59 research questions was generated for rating. Question topics were (1) Data, Measures, and Metrics; (2) Intervention Approach and Impact; (3) Technology; (4) Role of Healthcare Systems; (5) Community-Based Organizations; (6) Equity; (7) Funding; and (8) Social Health Integration. On a scale from 1 (low) to 10 (high), the mean impact score was 6.12 (range=4.14-7.79), and the mean priority score was 5.61 (range=3.07-8.64). Twenty-four statements were rated as both high impact (>6.12) and high priority (>5.61). Conclusions: The broad range of topics with high impact and priority scores reveals how nascent the evidence base is, with fundamental research on the nature of social risk and health system involvement still needed.

A sequential, multiple assignment randomized trial comparing web-based education to mobile video interpreter access for improving provider interpreter use in primary care clinics: the mVOCAL hybrid type 3 study protocol.

BACKGROUND: Individuals who use a language other than English for medical care are at risk for disparities related to healthcare safety, patient-centered care, and quality. Professional interpreter use decreases these disparities but remains underutilized, despite widespread access and legal mandates. In this study, we compare two discrete implementation strategies for improving interpreter use: (1) enhanced education targeting intrapersonal barriers to use delivered in a scalable format (interactive web-based educational modules) and (2) a strategy targeting system barriers to use in which mobile video interpreting is enabled on providers' own mobile devices. METHODS: We will conduct a type 3 hybrid implementation-effectiveness study in 3-5 primary care organizations, using a sequential multiple assignment randomized trial (SMART) design. Our primary implementation outcome is interpreter use, calculated by matching clinic visits to interpreter invoices. Our secondary effectiveness outcome is patient comprehension, determined by comparing patient-reported to provider-documented visit diagnosis. Enrolled providers (n = 55) will be randomized to mobile video interpreting or educational modules, plus standard interpreter access. After 9 months, providers with high interpreter use will continue as assigned; those with lower use will be randomized to continue as before or add the alternative strategy. After another 9 months, both strategies will be available to enrolled providers for 9 more months. Providers will complete 2 surveys (beginning and end) and 3 in-depth interviews (beginning, middle, and end) to understand barriers to interpreter use, based on the Theoretical Domains Framework. Patients who use a language other than English will be surveyed (n = 648) and interviewed (n = 75) following visits with enrolled providers to understand their experiences with communication. Visits will be video recorded (n = 100) to assess fidelity to assigned strategies. We will explore strategy mechanism activation to refine causal pathway models using a quantitative plus qualitative approach. We will also determine the incremental cost-effectiveness of each implementation strategy from a healthcare organization perspective, using administrative and provider survey data. DISCUSSION: Determining how these two scalable strategies, alone and in sequence, perform for improving interpreter use, the mechanisms by which they do so, and at what cost, will provide critical insights for addressing a persistent cause of healthcare disparities. TRIAL REGISTRATION: NCT05591586.

Social risks and social needs in a health insurance exchange sample: a longitudinal evaluation of utilization.

BACKGROUND: Health systems are increasingly attempting to intervene on social adversity as a strategy to improve health care outcomes. To inform health system efforts to screen for social adversity, we sought to explore the stability of social risk and interest in assistance over time and to evaluate whether the social risk was associated with subsequent healthcare utilization. METHODS: We surveyed Kaiser Permanente members receiving subsidies from the healthcare exchange in Southern California to assess their social risk and desire for assistance using the Accountable Health Communities instrument. A subset of initial respondents was randomized to be re-surveyed at either three or six months later. RESULTS: A total of 228 participants completed the survey at both time points. Social risks were moderate to strongly stable across three and six months (Kappa range = .59-.89); however, social adversity profiles that included participants' desire for assistance were more labile (3-month Kappa = .52; 95% CI = .41-.64 & 6-month Kappa = .48; 95% CI = .36-.6). Only housing-related social risks were associated with an increase in acute care (emergency, urgent care) six months after initial screening; no other associations between social risk and utilization were observed. CONCLUSIONS: This study suggests that screening for social risk may be appropriate at intervals of six months, or perhaps longer, but that assessing desire for assistance may need to occur more frequently. Housing risks were associated with increases in acute care. Health systems may need to engage in screening and referral to resources to improve overall care and ultimately patient total health.

Social Risk Factors and Desire for Assistance Among Patients Receiving Subsidized Health Care Insurance in a US-Based Integrated Delivery System.

PURPOSE: Because social conditions such as food insecurity and housing instability shape health outcomes, health systems are increasingly screening for and addressing patients' social risks. This study documented the prevalence of social risks and examined the desire for assistance in addressing those risks in a US-based integrated delivery system. METHODS: A survey was administered to Kaiser Permanente members on subsidized exchange health insurance plans (2018-2019). The survey included questions about 4 domains of social risks, desire for help, and attitudes. We conducted a descriptive analysis and estimated multivariate modified Poisson regression models. RESULTS: Of 438 participants, 212 (48%) reported at least 1 social risk factor. Housing instability was the most common (70%) factor reported. Members with social risks reported more discomfort being screened for social risks (14.2% vs 5.4%; P = .002) than those without risks, although 90% of participants believed that health systems should assist in addressing social risks. Among those with 1-2 social risks, however, only 27% desired assistance. Non-Hispanic Black participants who reported a social risk were more than twice as likely to desire assistance compared with non-Hispanic White participants (adjusted relative risk [RR] 2.2; 95% CI, 1.3-3.8). CONCLUSIONS: Athough most survey participants believed health systems have a role in addressing social risks, a minority of those reporting a risk wanted assistance and reported more discomfort being screened for risk factors than those without risks. Health systems should work to increase the comfort of patients in reporting risks, explore how to successfully assist them when desired, and offer resources to address these risks outside the health care sector.VISUAL ABSTRACT.

Comparing the performance of two social risk screening tools in a vulnerable subpopulation.

BACKGROUND: Research shows the profound impact of social factors on health, lead many healths systems to incorporate social risk screening. To help healthcare systems select among various screening tools we compared two tools, the Your Current Life Situation (YCLS) and the Accountable Health Communities (AHC) Screening tools, on key psychometric properties. METHOD: Kaiser Permanente Southern California subsidized exchange members (n = 1008) were randomly invited to complete a survey containing either the YCLS or the AHC tool, as well as other measures related to care experience and health. Healthcare use was measured through the electronic health record. Agreement between the AHC and YCLS was assessed using adjusted kappas for six domains (food - worry, food - pay, insecure housing, housing quality, transportation, utilities). To assess predictive validity, items on the AHC and YCLS were compared to self-rated health and receipt of a flu shot. RESULTS: Responders (n = 450) and non-responders (n = 558) significantly differed on sex, language, and depression (P < 0.05) but not anxiety, race/ethnicity, or healthcare use. Agreement between the AHC and YCLS tools was substantial on all items (kappas > 0.60) except for housing quality (kappa 0.52). Four out of six screening questions on the AHC tool and four out of seven on the YCLS tool were associated with self-rated health (P < 0.03). No social needs were associated with flu shot receipt except utilities on the AHC tool (P = 0.028). CONCLUSION: In this sample, the AHC and YCLS tools are similar in their ability to screen for social risks. Differences observed likely stem from the timeframe and wording of the questions, which can be used to guide selection in healthcare systems.

A systematic review of empirical studies examining mechanisms of implementation in health.

BACKGROUND: Understanding the mechanisms of implementation strategies (i.e., the processes by which strategies produce desired effects) is important for research to understand why a strategy did or did not achieve its intended effect, and it is important for practice to ensure strategies are designed and selected to directly target determinants or barriers. This study is a systematic review to characterize how mechanisms are conceptualized and measured, how they are studied and evaluated, and how much evidence exists for specific mechanisms. METHODS: We systematically searched PubMed and CINAHL Plus for implementation studies published between January 1990 and August 2018 that included the terms "mechanism," "mediator," or "moderator." Two authors independently reviewed title and abstracts and then full texts for fit with our inclusion criteria of empirical studies of implementation in health care contexts. Authors extracted data regarding general study information, methods, results, and study design and mechanisms-specific information. Authors used the Mixed Methods Appraisal Tool to assess study quality. RESULTS: Search strategies produced 2277 articles, of which 183 were included for full text review. From these we included for data extraction 39 articles plus an additional seven articles were hand-entered from only other review of implementation mechanisms (total = 46 included articles). Most included studies employed quantitative methods (73.9%), while 10.9% were qualitative and 15.2% were mixed methods. Nine unique versions of models testing mechanisms emerged. Fifty-three percent of the studies met half or fewer of the quality indicators. The majority of studies (84.8%) only met three or fewer of the seven criteria stipulated for establishing mechanisms. CONCLUSIONS: Researchers have undertaken a multitude of approaches to pursue mechanistic implementation research, but our review revealed substantive conceptual, methodological, and measurement issues that must be addressed in order to advance this critical research agenda. To move the field forward, there is need for greater precision to achieve conceptual clarity, attempts to generate testable hypotheses about how and why variables are related, and use of concrete behavioral indicators of proximal outcomes in the case of quantitative research and more directed inquiry in the case of qualitative research.

Psychometric and Pragmatic Properties of Social Risk Screening Tools: A Systematic Review.

CONTEXT: Health systems increasingly are exploring implementation of standardized social risk assessments. Implementation requires screening tools both with evidence of validity and reliability (psychometric properties) and that are low cost, easy to administer, readable, and brief (pragmatic properties). These properties for social risk assessment tools are not well understood and could help guide selection of assessment tools and future research. EVIDENCE ACQUISITION: The systematic review was conducted during 2018 and included literature from PubMed and CINAHL published between 2000 and May 18, 2018. Included studies were based in the U.S., included tools that addressed at least 2 social risk factors (economic stability, education, social and community context, healthcare access, neighborhood and physical environment, or food), and were administered in a clinical setting. Manual literature searching was used to identify empirical uses of included screening tools. Data on psychometric and pragmatic properties of each tool were abstracted. EVIDENCE SYNTHESIS: Review of 6,838 unique citations yielded 21 unique screening tools and 60 articles demonstrating empirical uses of the included screening tools. Data on psychometric properties were sparse, and few tools reported use of gold standard measurement development methods. Review of pragmatic properties indicated that tools were generally low cost, written for low-literacy populations, and easy to administer. CONCLUSIONS: Multiple low-cost, low literacy tools are available for social risk screening in clinical settings, but psychometric data are very limited. More research is needed on clinic-based screening tool reliability and validity as these factors should influence both adoption and utility. SUPPLEMENT INFORMATION: This article is part of a supplement entitled Identifying and Intervening on Social Needs in Clinical Settings: Evidence and Evidence Gaps, which is sponsored by the Agency for Healthcare Research and Quality of the U.S. Department of Health and Human Services, Kaiser Permanente, and the Robert Wood Johnson Foundation.

Part I: A Quantitative Study of Social Risk Screening Acceptability in Patients and Caregivers.

INTRODUCTION: Despite recent growth in healthcare delivery-based social risk screening, little is known about patient perspectives on these activities. This study evaluates patient and caregiver acceptability of social risk screening. METHODS: This was a cross-sectional survey of 969 adult patients and adult caregivers of pediatric patients recruited from 6 primary care clinics and 4 emergency departments across 9 states. Survey items included the Center for Medicare and Medicaid Innovation Accountable Health Communities' social risk screening tool and questions about appropriateness of screening and comfort with including social risk data in electronic health records. Logistic regressions evaluated covariate associations with acceptability measures. Data collection occurred from July 2018 to February 2019; data analyses were conducted in February‒March 2019. RESULTS: Screening was reported as appropriate by 79% of participants; 65% reported comfort including social risks in electronic health records. In adjusted models, higher perceived screening appropriateness was associated with previous exposure to healthcare-based social risk screening (AOR=1.82, 95% CI=1.16, 2.88), trust in clinicians (AOR=1.55, 95% CI=1.00, 2.40), and recruitment from a primary care setting (AOR=1.70, 95% CI=1.23, 2.38). Lower appropriateness was associated with previous experience of healthcare discrimination (AOR=0.66, 95% CI=0.45, 0.95). Higher comfort with electronic health record documentation was associated with previously receiving assistance with social risks in a healthcare setting (AOR=1.47, 95% CI=1.04, 2.07). CONCLUSIONS: A strong majority of adult patients and caregivers of pediatric patients reported that social risk screening was appropriate. Most also felt comfortable including social risk data in electronic health records. Although multiple factors influenced acceptability, the effects were moderate to small. These findings suggest that lack of patient acceptability is unlikely to be a major implementation barrier. SUPPLEMENT INFORMATION: This article is part of a supplement entitled Identifying and Intervening on Social Needs in Clinical Settings: Evidence and Evidence Gaps, which is sponsored by the Agency for Healthcare Research and Quality of the U.S. Department of Health and Human Services, Kaiser Permanente, and the Robert Wood Johnson Foundation.

Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda.

The field of implementation science was developed to better understand the factors that facilitate or impede implementation and generate evidence for implementation strategies. In this article, we briefly review progress in implementation science, and suggest five priorities for enhancing the impact of implementation strategies. Specifically, we suggest the need to: (1) enhance methods for designing and tailoring implementation strategies; (2) specify and test mechanisms of change; (3) conduct more effectiveness research on discrete, multi-faceted, and tailored implementation strategies; (4) increase economic evaluations of implementation strategies; and (5) improve the tracking and reporting of implementation strategies. We believe that pursuing these priorities will advance implementation science by helping us to understand when, where, why, and how implementation strategies improve implementation effectiveness and subsequent health outcomes.

Psychometric assessment of three newly developed implementation outcome measures.

BACKGROUND: Implementation outcome measures are essential for monitoring and evaluating the success of implementation efforts. Yet, currently available measures lack conceptual clarity and have largely unknown reliability and validity. This study developed and psychometrically assessed three new measures: the Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM). METHODS: Thirty-six implementation scientists and 27 mental health professionals assigned 31 items to the constructs and rated their confidence in their assignments. The Wilcoxon one-sample signed rank test was used to assess substantive and discriminant content validity. Exploratory and confirmatory factor analysis (EFA and CFA) and Cronbach alphas were used to assess the validity of the conceptual model. Three hundred twenty-six mental health counselors read one of six randomly assigned vignettes depicting a therapist contemplating adopting an evidence-based practice (EBP). Participants used 15 items to rate the therapist's perceptions of the acceptability, appropriateness, and feasibility of adopting the EBP. CFA and Cronbach alphas were used to refine the scales, assess structural validity, and assess reliability. Analysis of variance (ANOVA) was used to assess known-groups validity. Finally, half of the counselors were randomly assigned to receive the same vignette and the other half the opposite vignette; and all were asked to re-rate acceptability, appropriateness, and feasibility. Pearson correlation coefficients were used to assess test-retest reliability and linear regression to assess sensitivity to change. RESULTS: All but five items exhibited substantive and discriminant content validity. A trimmed CFA with five items per construct exhibited acceptable model fit (CFI = 0.98, RMSEA = 0.08) and high factor loadings (0.79 to 0.94). The alphas for 5-item scales were between 0.87 and 0.89. Scale refinement based on measure-specific CFAs and Cronbach alphas using vignette data produced 4-item scales (α's from 0.85 to 0.91). A three-factor CFA exhibited acceptable fit (CFI = 0.96, RMSEA = 0.08) and high factor loadings (0.75 to 0.89), indicating structural validity. ANOVA showed significant main effects, indicating known-groups validity. Test-retest reliability coefficients ranged from 0.73 to 0.88. Regression analysis indicated each measure was sensitive to change in both directions. CONCLUSIONS: The AIM, IAM, and FIM demonstrate promising psychometric properties. Predictive validity assessment is planned.

Intentional research design in implementation science: implications for the use of nomothetic and idiographic assessment.

The advancement of implementation science is dependent on identifying assessment strategies that can address implementation and clinical outcome variables in ways that are valid, relevant to stakeholders, and scalable. This paper presents a measurement agenda for implementation science that integrates the previously disparate assessment traditions of idiographic and nomothetic approaches. Although idiographic and nomothetic approaches are both used in implementation science, a review of the literature on this topic suggests that their selection can be indiscriminate, driven by convenience, and not explicitly tied to research study design. As a result, they are not typically combined deliberately or effectively. Thoughtful integration may simultaneously enhance both the rigor and relevance of assessments across multiple levels within health service systems. Background on nomothetic and idiographic assessment is provided as well as their potential to support research in implementation science. Drawing from an existing framework, seven structures (of various sequencing and weighting options) and five functions (Convergence, Complementarity, Expansion, Development, Sampling) for integrating conceptually distinct research methods are articulated as they apply to the deliberate, design-driven integration of nomothetic and idiographic assessment approaches. Specific examples and practical guidance are provided to inform research consistent with this framework. Selection and integration of idiographic and nomothetic assessments for implementation science research designs can be improved. The current paper argues for the deliberate application of a clear framework to improve the rigor and relevance of contemporary assessment strategies.

Methods to Improve the Selection and Tailoring of Implementation Strategies.

Implementing behavioral health interventions is a complicated process. It has been suggested that implementation strategies should be selected and tailored to address the contextual needs of a given change effort; however, there is limited guidance as to how to do this. This article proposes four methods (concept mapping, group model building, conjoint analysis, and intervention mapping) that could be used to match implementation strategies to identified barriers and facilitators for a particular evidence-based practice or process change being implemented in a given setting. Each method is reviewed, examples of their use are provided, and their strengths and weaknesses are discussed. The discussion includes suggestions for future research pertaining to implementation strategies and highlights these methods' relevance to behavioral health services and research.

A methodology for enhancing implementation science proposals: comparison of face-to-face versus virtual workshops.

BACKGROUND: With the current funding climate and need for advancements in implementation science, there is a growing demand for grantsmanship workshops to increase the quality and rigor of proposals. A group-based implementation science-focused grantsmanship workshop, the Implementation Development Workshop (IDW), is one methodology to address this need. This manuscript provides an overview of the IDW structure, format, and findings regarding its utility. RESULTS: The IDW methodology allows researchers to vet projects in the proposal stage in a structured format with a facilitator and two types of expert participants: presenters and attendees. The presenter uses a one-page handout and verbal presentation to present their proposal and questions. The facilitator elicits feedback from attendees using a format designed to maximize the number of unique points made. After each IDW, participants completed an anonymous survey assessing perceptions of the IDW. Presenters completed a funding survey measuring grant submission and funding success. Qualitative interviews were conducted with a subset of participants who participated in both delivery formats. Mixed method analyses were performed to evaluate the effectiveness and acceptability of the IDW and compare the delivery formats. Of those who participated in an IDW (N = 72), 40 participated in face-to-face only, 16 in virtual only, and 16 in both formats. Thirty-eight (face-to-face n = 12, 35 % response rate; virtual n = 26, 66.7 % response rate) responded to the surveys and seven (15.3 % response rate), who had attended both formats, completed an interview. Of 36 total presenters, 17 (face-to-face n = 12, 42.9 % response rate; virtual n = 5, 62.9 % response rate) responded to the funding survey. Mixed method analyses indicated that the IDW was effective for collaboration and growth, effective for enhancing success in obtaining grants, and acceptable. A third (35.3 %) of presenters ultimately received funding for their proposal, and more than 80 % of those who presented indicated they would present again in the future. The IDW structure and facilitation process were found to be acceptable, with both formats rated as equally strong. CONCLUSIONS: The IDW presents an acceptable and successful methodology for increasing competitiveness of implementation science grant proposals.