The Impact of Sparse Datasets When Harmonizing Data from Studies with Different Measures of the Same Construct.

Prevention science has increasingly turned to integrative data analysis (IDA) to combine individual participant-level data from multiple studies of the same topic, allowing us to evaluate overall effect size, test and model heterogeneity, and examine mediation. Studies included in IDA often use different measures for the same construct, leading to sparse datasets. We introduce a graph theory method for summarizing patterns of sparseness and use simulations to explore the impact of different patterns on measurement bias within three different measurement models: a single common factor, a hierarchical model, and a bifactor model. We simulated 1000 datasets with varying levels of sparseness and used Bayesian methods to estimate model parameters and evaluate bias. Results clarified that bias due to sparseness will depend on the strength of the general factor, the measurement model employed, and the level of indirect linkage among measures. We provide an example using a synthesis dataset that combined data on youth depression from 4146 youth who participated in 16 randomized field trials of prevention programs. Given that different synthesis datasets will embody different patterns of sparseness, we conclude by recommending that investigators use simulation methods to explore the potential for bias given the sparseness patterns they encounter.

The Impact of the Communities that Care Approach in Reducing Violence and Crime Within an Urban, High-Burden Community.

Community violence and crime are significant public health problems with serious and lasting effects on young people, families, and communities. This violence and crime have significant ripple effects, affecting not just those who are directly physically injured, but also those who witness violent episodes, those who have friends or loved ones killed or injured, and those who must everyday navigate streets that they know have been frequent sites of serious violence and crime. The current study presents evidence of the impact that a data-driven, collective impact approach - the Communities that Care prevention system - can have on violence and crime outcomes within a large urban, high-burden community. Established as one of the national Youth Violence Prevention Centers (YVPC) funded by the Centers for Disease Control and Prevention, the Chicago Center for Youth Violence Prevention is among the first to implement the CTC approach in a large, urban community. The current study's findings show reductions in violence (i.e., aggravated assaults and robberies) in the Bronzeville community, compared to similar communities in Chicago.

Utilization of genetic testing in men with advanced prostate cancer.

BACKGROUND: Genetic evaluation of men with advanced prostate cancer is recognized as imperative both to guide treatment decisions and to trigger cascade genetic testing of family members. Here we investigate utilization patterns of genetic testing among a contemporary cohort of men with advanced prostate cancer at our institution. METHODS: We queried the Northwestern Electronic Data Warehouse from January 2021 to present for all men diagnosed with National Comprehensive Cancer Network high-risk/very high-risk, regional, or metastatic prostate cancer. Patients were excluded from analyses if treated at an outside institution and/or presented for a second opinion evaluation. Statistics were performed using t-test, Chi-squared test, and univariable and multivariable logistic regression with significance defined as p < 0.05. RESULTS: Atotal of 320 men (52.5%) had local/regional disease and 290 (47.5%) had metastatic disease, 53 (18.3%) of whom had castrate resistant prostate cancer. Rates of germline genetic testing rate were low in patients with localized disease (9.4%) and metastatic disease (34.1%). Only 19 (35.8%) men diagnosed with metastatic castrate resistant prostate cancer underwent germline genetic evaluation. Germline testing was most frequently discussed or ordered by medical oncologists (52%) followed by urologists (20%). Men who underwent germline testing were younger (p < 0.001), more likely to have Medicaid or private insurance (p = 0.002), and more likely to have metastatic disease (p < 0.001). There were no statistically significant differences in baseline PSA, ethnicity, race, or castration sensitivity status. Age (odds ratio [OR]: 0.94, 95% confidence interval [CI]: 0.91-0.97, p < 0.001) and metastatic disease (OR: 5.71, 95% CI: 3.63-9.22, p < 0.001) were significant independent predictors of genetic testing on multivariable logistic regression. CONCLUSIONS: Here we report that utilization of genetic testing is associated with metastatic disease and inversely associated with age. Overall, utilization rates of genetic testing remain low in all patient groups, including in the metastatic castrate resistant setting, where genetic testing can identify patients with homologous recombination repair deficiency who may benefit from use of targeted therapeutics such as PARP inhibitors. Genetic testing in men with aggressive prostate cancer is critical and barriers to routine implementation of testing require further study to develop strategies to improve utilization rates.

Implementation Barriers and Experiences of Eligible Patients Who Failed to Enroll in Collaborative Care for Depression and Anxiety.

BACKGROUND: Effective and efficient implementation of the Collaborative Care Model (CoCM) for depression and anxiety is imperative for program success. Studies examining barriers to implementation often omit patient perspectives. OBJECTIVES: To explore experiences and attitudes of eligible patients referred to CoCM who declined participation or were unable to be reached, and identify implementation barriers to inform strategies. DESIGN: Convergent mixed-methods study with a survey and interview. PARTICIPANTS: Primary care patients at an academic medical center who were referred to a CoCM program for anxiety and depression by their primary care clinician (PCC) but declined participation or were unable to be reached by the behavioral health care manager to initiate care (n = 80). Interviews were conducted with 45 survey respondents. MAIN MEASURES: Survey of patients' referral experiences and behavioral health preferences as they related to failing to enroll in the program. Interview questions were developed using the Consolidated Framework for Implementation Research version 2.0 (CFIR 2.0) to identify implementation barriers to enrollment. KEY RESULTS: Survey results found that patients were uncertain about insurance coverage, did not understand the program, and felt services were not necessary. Referred patients who declined participation were concerned about how their mental health information would be used and preferred treatment without medication. Men agreed more that they did not need services. Qualitative results exhibited a variety of implementation determinants (n = 23) across the five CFIR 2.0 domains. Barriers included mental health stigma, perceiving behavioral health as outside of primary care practice guidelines, short or infrequent primary care appointments, prioritizing physical health over mental health, receiving inaccurate program information, low motivation to engage, and a less established relationship with their PCC. CONCLUSIONS: Multiple barriers to enrollment led to failing to link patients to care, which can inform implementation strategies to address the patient-reported experiences and concerns.

Promoting Evidence-Based Practice: The Influence of Novel Structural Change to Accelerate Translational Rehabilitation.

OBJECTIVE: To evaluate changes in clinicians' use of evidence-based practice (EBP), openness toward EBP, and their acceptance of organizational changes after a rehabilitation hospital transitioned to a new facility designed to accelerate clinician-researcher collaborations. DESIGN: Three repeated surveys of clinicians before, 7-9 months, and 2.5 years after transition to the new facility. SETTING: Inpatient rehabilitation hospital. PARTICIPANTS: Physicians, nurses, therapists, and other health care professionals (n=410, 442, and 448 respondents at Times 1, 2, and 3, respectively). INTERVENTIONS: Implementation of physical (architecture, design) and team-focused (champions, leaders, incentives) changes in a new model of care to promote clinician-researcher collaborations. MAIN OUTCOME MEASURES: Adapted versions of the Evidence-Based Practice Questionnaire (EBPQ), the Evidence-Based Practice Attitudes Scale (EBPAS), and the Organizational Change Recipients' Beliefs Scale (OCRBS) were used. Open-ended survey questions were analyzed through exploratory content analysis. RESULTS: Response rates at Times 1, 2, and 3 were 67% (n=410), 69% (n=422), and 71% (n=448), respectively. After accounting for familiarity with the model of care, there was greater reported use of EBP at Time 3 compared with Time 2 (adjusted meant2=3.51, standard error (SE)=0.05; adj. meant3=3.64, SE=0.05; P=.043). Attitudes toward EBPs were similar over time. Acceptance of the new model of care was lower at Time 2 compared with Time 1, but rebounded at Time 3 (adjusted meant1=3.44, SE=0.04; adj. meant2=3.19, SE=0.04; P<.0001; adj. meant3=3.51, SE=0.04; P<.0001). Analysis of open-ended responses suggested that clinicians' optimism for the model of care was greater over time, but continued quality improvement should focus on cultivating communication between clinicians and researchers. CONCLUSIONS: Accelerating clinician-researcher collaborations in a rehabilitation setting requires sustained effort for successful implementation beyond novel physical changes. Organizations must be responsive to clinicians' changing concerns to adapt and sustain a collaborative translational medicine model and allow sufficient time, probably years, for such transitions to occur.

Retrospective Psychometrics and Effect Heterogeneity in Integrated Data Analysis: Commentary on the Special Issue.

The current special issue of Prevention Science indicates that momentum in using individual participant data (IPD) and integrative data analysis (IDA) to combine and synthesize findings in prevention science has accelerated over the past decade. In this commentary, we focus on two general themes involving methods for harmonizing measures and findings of effect heterogeneity. We describe methods for harmonization as retrospective psychometrics, requiring that we attend to the assumptions necessary for accurate measurement, but adjust our methods given the constraints of working with existing datasets that often involve different measures in different studies. We point to novel approaches for increasing confidence that semantic matching and empirical modeling used in these studies will yield accurate and valid measurements that can be combined in IDA. We also review findings about effect heterogeneity, emphasizing the importance of using etiologic and action theories to identify and evaluate sources of such effects. We note that all of the papers in this issue deserve careful attention, as they illustrate how prevention scientists are approaching the complexities of IDA and exploring novel methods for overcoming its challenges.

Normalization of Prevention Principles and Practices to Reduce Substance Use Disorders Through an Integrated Dissemination and Implementation Framework.

Major research breakthroughs over the past 30 years in the field of substance use prevention have served to: (1) enhance understanding of pharmacological effects on the central and peripheral nervous systems and the health and social consequences of use of psychoactive substances, particularly for children and adolescents; (2) delineate the processes that increase vulnerability to or protect from initiation of substance use and progression to substance use disorders (SUDs) and, based on this understanding, (3) develop effective strategies and practices to prevent the initiation and escalation of substance use. The challenge we now face as a field is to "normalize" what we have learned from this research so that it is incorporated into the work of those involved in supporting, planning, and delivering prevention programming to populations around the world, is integrated into health and social service systems, and helps to shape public policies. But we wish to go further, to incorporate these effective prevention practices into everyday life and the mind-sets of the public, particularly parents and educators. This paper reviews the advances that have been made in the field of prevention and presents a framework and recommendations to achieve these objectives generated during several meetings of prevention and implementation science researchers sponsored by the International Consortium of Universities for Drug Demand Reduction (ICUDDR) that guides a roadmap to achieve "normalization."

A stepped-wedge randomized trial investigating the effect of the Leadership and Organizational Change for Implementation (LOCI) intervention on implementation and transformational leadership, and implementation climate.

BACKGROUND: This study evaluates the Leadership and Organizational Change for Implementation (LOCI) strategy and its effect on implementation leadership, transformational leadership, and implementation climate. METHODS: A stepped wedge cluster randomized study design enrolling 47 first-level leaders from child- and adult-specialized mental health clinics within Norwegian health trusts across three cohorts. All therapists (n = 790) received training in screening of trauma exposure and posttraumatic stress, and a subgroup of therapists (n = 248) received training in evidence-based treatment methods for posttraumatic stress disorder (PTSD). First-level leaders and therapists completed surveys at baseline, 4, 8-, 12-, 16-, and 20-months assessing leadership and implementation climate. General linear mixed-effects models were used to investigate whether the LOCI strategy would lead to greater therapist-rated scores on implementation leadership, transformational leadership, and implementation climate. RESULTS: After introducing the LOCI strategy, there was a significant increase in therapist-rated implementation and transformational leadership and implementation climate. The increase was sustained at all measurement time points compared to non-LOCI conditions, which demonstrated a steady decrease in scores before LOCI. CONCLUSIONS: The LOCI strategy can develop better transformational and implementation leadership skills and contribute to a more positive implementation climate, which may enhance successful EBP implementation. Thus, LOCI can help leaders create an organizational context conducive for effective EBP implementation. TRIAL REGISTRATION: Retrospectively registered: ClinicalTrials NCT03719651 , 25th of October 2018. The trial protocol can be accessed from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6417075/ .

High-Fidelity Agent-Based Modeling to Support Prevention Decision-Making: an Open Science Approach.

Preventing adverse health outcomes is complex due to the multi-level contexts and social systems in which these phenomena occur. To capture both the systemic effects, local determinants, and individual-level risks and protective factors simultaneously, the prevention field has called for adoption of system science methods in general and agent-based models (ABMs) specifically. While these models can provide unique and timely insight into the potential of prevention strategies, an ABM's ability to do so depends strongly on its accuracy in capturing the phenomenon. Furthermore, for ABMs to be useful, they need to be accepted by and available to decision-makers and other stakeholders. These two attributes of accuracy and acceptability are key components of open science. To ensure the creation of high-fidelity models and reliability in their outcomes and consequent model-based decision-making, we present a set of recommendations for adopting and using this novel method. We recommend ways to include stakeholders throughout the modeling process, as well as ways to conduct model verification, validation, and replication. Examples from HIV and overdose prevention work illustrate how these recommendations can be applied.

Re-envisioning, Retooling, and Rebuilding Prevention Science Methods to Address Structural and Systemic Racism and Promote Health Equity.

The historic momentum from national conversations on the roots and current impacts of racism in the USA presents an incredible window of opportunity for prevention scientists to revisit how common theories, measurement tools, methodologies, and interventions can be radically re-envisioned, retooled, and rebuilt to dismantle racism and promote equitable health for minoritized communities. Recognizing this opportunity, the NIH-funded Prevention Science and Methodology Group (PSMG) launched a series of presentations focused on the role of Prevention Science to address racism and discrimination guided by a commitment to social justice and health equity. The current manuscript aims to advance the field of Prevention Science by summarizing key issues raised during the series' presentations and proposing concrete research priorities and steps that hold promise for promoting health equity by addressing systemic racism. Being anti-racist is an active practice for all of us, whether we identify as methodologists, interventionists, practitioners, funders, community members, or an intersection of these identities. We implore prevention scientists and methodologists to take on these conversations with us to promote science and practice that offers every life the right to live in a just and equitable world.

Accounting for Context in Randomized Trials after Assignment.

Many preventive trials randomize individuals to intervention condition which is then delivered in a group setting. Other trials randomize higher levels, say organizations, and then use learning collaboratives comprised of multiple organizations to support improved implementation or sustainment. Other trials randomize or expand existing social networks and use key opinion leaders to deliver interventions through these networks. We use the term contextually driven to refer generally to such trials (traditionally referred to as clustering, where groups are formed either pre-randomization or post-randomization - i.e., a cluster-randomized trial), as these groupings or networks provide fixed or time-varying contexts that matter both theoretically and practically in the delivery of interventions. While such contextually driven trials can provide efficient and effective ways to deliver and evaluate prevention programs, they all require analytical procedures that take appropriate account of non-independence, something not always appreciated. Published analyses of many prevention trials have failed to take this into account. We discuss different types of contextually driven designs and then show that even small amounts of non-independence can inflate actual Type I error rates. This inflation leads to rejecting the null hypotheses too often, and erroneously leading us to conclude that there are significant differences between interventions when they do not exist. We describe a procedure to account for non-independence in the important case of a two-arm trial that randomizes units of individuals or organizations in both arms and then provides the active treatment in one arm through groups formed after assignment. We provide sample code in multiple programming languages to guide the analyst, distinguish diverse contextually driven designs, and summarize implications for multiple audiences.

Treatment for opioid use disorder in the Florida medicaid population: Using a cascade of care model to evaluate quality.

Background: A cascade of care (CoC) model may improve understanding of gaps in addiction treatment availability and quality over current single measure methods. Despite increased funding, opioid overdose rates remain high. Therefore, it is critical to understand where the health-care system is failing to provide appropriate care for people with opioid use disorder (OUD) diagnoses, and to assess disparities in receipt of medication for OUD (MOUD).Objective: Using a CoC framework, assess treatment quality and outcomes for OUD in the Florida Medicaid population in 2017/2018 by demographics and primary vs. secondary diagnosis.Methods: Data from Florida Medicaid claims for 2017 and 2018 were used to calculate the number of enrollees who were diagnosed, began MOUD, were retained on medication for a minimum of 180 days, and who died.Results: Only 28% of those diagnosed with OUD began treatment with an FDA approved MOUD (buprenorphine, methadone, or injectable naltrexone). Once on medication, 38% of newly diagnosed enrollees were retained in treatment for180 days. Those who remained on MOUD for 180 days had a hazard ratio of death of 0.226 (95% CI = 0.174 to 0.294) compared to those that did not initiate MOUD, a reduction in mortality from 10% without MOUD to 2% with MOUD.Conclusions: Initiating medication after OUD diagnosis offers the greatest opportunity for intervention to reduce overdose deaths, though efforts to increase retention are also warranted. Analyzing claims data with CoC identifies system functioning for specific populations, and suggests policies and clinical pathways to target for improvement.

Exploring Clusters of Risk and Association With Child Maltreatment in North Carolina Counties.

BACKGROUND: Decision makers face challenges in estimating local risk for child maltreatment and how best to prioritize which factors to intervene upon. METHODS: Using US Census and survey data for all US counties (N = 3141), we derived US county profiles characterized by the severity of child maltreatment risk factors observed at the county level, such as parental health, health care access, and economic distress. We estimated how five child maltreatment outcomes would vary across the profiles for North Carolina counties (n = 100): total maltreatment reports (including unsubstantiated and substantiated), substantiated neglect, substantiated abuse, whether services were received, and reported child's race/ethnicity. RESULTS: We derived three profiles of county-level child maltreatment risk: high, moderate, and low risk, denoting that predicted risk factors means within profiles were all high, moderate, or low levels compared to counties in other profiles. One risk factor did not follow this pattern: the drug overdose death rate. It was highest in the moderate-risk profile instead of the high-risk profile, as would have been consistent with other factor levels. Moderate-risk counties had the highest predicted rate of child maltreatment reports, with over 20 more reports per 10,000 residents compared to low-risk counties (95% CI, 1.38, 38.86). LIMITATIONS: We included only factors for which aggregate, county-level estimates were available, thus limiting inclusion of all relevant factors. CONCLUSIONS: Results suggest the need for increased family-based services and interventions that reduce risk factors such as economic distress and drug overdose deaths. We discuss the implications for tailoring county efforts to prevent child maltreatment.

Reducing Cardiovascular Disparities Through Community-Engaged Implementation Research: A National Heart, Lung, and Blood Institute Workshop Report.

Cardiovascular disparities remain pervasive in the United States. Unequal disease burden is evident among population groups based on sex, race, ethnicity, socioeconomic status, educational attainment, nativity, or geography. Despite the significant declines in cardiovascular disease mortality rates in all demographic groups during the last 50 years, large disparities remain by sex, race, ethnicity, and geography. Recent data from modeling studies, linked micromap plots, and small-area analyses also demonstrate prominent variation in cardiovascular disease mortality rates across states and counties, with an especially high disease burden in the southeastern United States and Appalachia. Despite these continued disparities, few large-scale intervention studies have been conducted in these high-burden populations to examine the feasibility of reducing or eliminating cardiovascular disparities. To address this challenge, on June 22 and 23, 2017, the National Heart, Lung, and Blood Institute convened experts from a broad range of biomedical, behavioral, environmental, implementation, and social science backgrounds to summarize the current state of knowledge of cardiovascular disease disparities and propose intervention strategies aligned with the National Heart, Lung, and Blood Institute mission. This report presents the themes, challenges, opportunities, available resources, and recommended actions discussed at the workshop.

Landscape of HIV Implementation Research Funded by the National Institutes of Health: A Mapping Review of Project Abstracts.

In 2019, the requisite biomedical and behavioral interventions to eliminate new HIV infections exist. "Ending the HIV Epidemic" now becomes primarily a challenge of will and implementation. This review maps the extent to which implementation research (IR) has been integrated into HIV research by reviewing the recent funding portfolio of the NIH. We searched NIH RePORTER for HIV and IR-related research projects funded from January 2013 to March 2018. The 4629 unique studies identified were screened using machine learning and manual methods. 216 abstracts met the eligibility criteria of HIV and IR. Key study characteristics were then abstracted. NIH currently funds HIV studies that are either formally IR (n = 109) or preparatory for IR (n = 107). Few (13%) projects mentioned a guiding implementation model, theory, or framework, and only 56% of all studies explicitly mentioned measuring an implementation outcome. Considering the study aims along an IR continuum, 18 (8%) studies examined barriers and facilitators, 43 (20%) developed implementation strategies, 46 (21%) piloted strategies, 73 (34%) tested a single strategy, and 35 (16%) compared strategies. A higher proportion of formal IR projects involved established interventions (e.g., integrated services) compared to newer interventions (e.g., pre-exposure prophylaxis). Prioritizing HIV-related IR in NIH and other federal funding opportunity announcements and expanded training in implementation science could have a substantial impact on ending the HIV pandemic. This review serves as a baseline by which to compare funding patterns and the sophistication of IR in HIV research over time.

Pragmatic adaptation of implementation research measures for a novel context and multiple professional roles: a factor analysis study.

BACKGROUND: Although some advances have been made in recent years, the lack of measures remains a major challenge in the field of implementation research. This results in frequent adaptation of implementation measures for different contexts-including different types of respondents or professional roles-than those for which they were originally developed and validated. The psychometric properties of these adapted measures are often not rigorously evaluated or reported. In this study, we examined the internal consistency, factor structure, and structural invariance of four well-validated measures of inner setting factors across four groups of respondents. The items in these measures were adapted as part of an evaluation of a large-scale organizational change in a rehabilitation hospital, which involved transitioning to a new building and a new model of patient care, facilitated by a significant redesign of patient care and research spaces. METHODS: Items were tailored for the context and perspective of different respondent groups and shortened for pragmatism. Confirmatory factor analysis was then used to test study hypotheses related to fit, internal consistency, and invariance across groups. RESULTS: The survey was administered to approximately 1208 employees; 785 responded (65% response rate) across the roles of clinician, researcher, leader, support staff, or dual clinician and researcher. For each of the four scales, confirmatory factor analysis demonstrated adequate fit that largely replicated the original measure. However, a few items loaded poorly and were removed from the final models. Internal consistencies of the final scales were acceptable. For scales that were administered to multiple professional roles, factor structures were not statistically different across groups, indicating structural invariance. CONCLUSIONS: The four inner setting measures were robust for use in this new context and across the multiple stakeholder groups surveyed. Shortening these measures did not significantly impair their measurement properties; however, as this study was cross sectional, future studies are required to evaluate the predictive validity and test-retest reliability of these measures. The successful use of adapted measures across contexts, across and between respondent groups, and with fewer items is encouraging, given the current emphasis on designing pragmatic implementation measures.

24-Month Outcomes of Primary Care Web-Based Depression Prevention Intervention in Adolescents: Randomized Clinical Trial.

BACKGROUND: Adolescent depression carries a high burden of disease worldwide, but access to care for this population is limited. Prevention is one solution to curtail the negative consequences of adolescent depression. Internet interventions to prevent adolescent depression can overcome barriers to access, but few studies examine long-term outcomes. OBJECTIVE: This study compares CATCH-IT (Competent Adulthood Transition with Cognitive Behavioral Humanistic and Interpersonal Training), an internet-based intervention, to a general health education active control for depression onset at 12 and 24 months in adolescents presenting to primary care settings. METHODS: A 2-site randomized trial, blinded to the principal investigators and assessors, was conducted comparing Competent Adulthood Transition with Cognitive Behavioral Humanistic and Interpersonal Training to health education to prevent depressive episodes in 369 adolescents (193 youths were randomly assigned to Competent Adulthood Transition with Cognitive Behavioral Humanistic and Interpersonal Training and 176 to health education) with subthreshold depressive symptoms or prior depressive episodes. Participants were recruited from primary care settings in the United States. The primary outcome was the occurrence of a depressive episode, determined by the Depression Symptom Rating. The secondary outcome was functioning, measured by the Global Assessment Scale. RESULTS: In intention-to-treat analyses, the adjusted hazard ratio favoring Competent Adulthood Transition with Cognitive Behavioral Humanistic and Interpersonal Training for first depressive episode was not statistically significant at 12 months (hazard ratio 0.77, 95% CI 0.42-1.40, P=.39) and 24 months (hazard ratio 0.87, 95% CI 0.52-1.47, P=.61). Competent Adulthood Transition with Cognitive Behavioral Humanistic and Interpersonal Training provided preventive benefit for first depressive episode for those with mild hopelessness or at least moderate paternal monitoring at baseline. Global Assessment Scale scores improved comparably in both groups (intention-to-treat). CONCLUSIONS: A technology-based intervention for adolescent depression prevention implemented in primary care did not have additional benefit at 12 or 24 months. Further research is necessary to determine whether internet interventions have long-term benefit. TRIAL REGISTRATION: ClinicalTrials.gov NCT01893749; http://clinicaltrials.gov/ct2/show/NCT01893749.

Developing a Multi-Criteria Decision Analysis Tool to Support the Adoption of Evidence-Based Child Maltreatment Prevention Programs.

Decision-makers need to consider a range of factors when selecting evidence-based programs (EBPs) for implementation, which can be especially challenging when addressing complex issues such as child maltreatment prevention. Multi-criteria decision analysis (MCDA) frameworks and tools are useful for evaluating such complex decisions. We describe the development and testing of the first MCDA tool to compare EBPs for child neglect prevention. To develop the tool, we engaged stakeholders (n = 8) to define the problem and identify 13 criteria and associated weights. In a pilot study, we tested the MCDA tool with decision-makers (n = 11) who were asked to rank three evidence-based child neglect prevention interventions both with and without the tool. The MCDA's weighted sum intervention ranking differed from the ranking without the tool in the majority of the sample (55%). Decision-makers provided guidance on criteria that should be clarified or added, resulting in 16 criteria in an iterated tool. The most frequent criterion suggestions related to community acceptance of the intervention, health equity, implementation supports, and sustainability. Decision-maker feedback guided user interface refinements. The MCDA tool was generally well accepted by decision-makers due to their trust in the stakeholder engagement process. More research is needed to understand the acceptability of MCDA approaches in additional contexts and whether EBPs adopted with decision support have different population health impacts compared with EBPs adopted without support. MCDA tools could facilitate evidence-based responses to federal policy and funding opportunities such as the Families First Preventive Services Act.

Three Flavorings for a Soup to Cure what Ails Mental Health Services.

With new tools from artificial intelligence and new perspectives on personalizing interventions, we could revolutionize the way mental health services are delivered and achieve major gains in improving the public's mental health. We examine Dr. Bickman's vision around these technological and paradigm changes that would usher in major scientific, workforce training, and societal cultural changes. We argue that additional efforts in research evaluations in implementation have the potential to scale up and adapt existing interventions and scale them out to diverse populations and service systems. The next stage of this work involves testing the effectiveness of personalized interventions that are preferred by the public and integrating these choices into sustainable service systems. We note cautions on the delivery of these programs as automated algorithmic recommendations are heretofore foreign to humans.

Design Considerations for Implementing eHealth Behavioral Interventions for HIV Prevention in Evolving Sociotechnical Landscapes.

PURPOSE OF REVIEW: Despite tremendous potential for public health impact and continued investments in development and evaluation, it is rare for eHealth behavioral interventions to be implemented broadly in practice. Intervention developers may not be planning for implementation when designing technology-enabled interventions, thus creating greater challenges for real-world deployment following a research trial. To facilitate faster translation to practice, we aimed to provide researchers and developers with an implementation-focused approach and set of design considerations as they develop new eHealth programs. RECENT FINDINGS: Using the Accelerated Creation-to-Sustainment model as a lens, we examined challenges and successes experienced during the development and evaluation of four diverse eHealth HIV prevention programs for young men who have sex with men: Keep It Up!, Harnessing Online Peer Education, Guy2Guy, and HealthMindr. HIV is useful for studying eHealth implementation because of the substantial proliferation of diverse eHealth interventions with strong evidence of reach and efficacy and the responsiveness to rapid and radical disruptions in the field. Rather than locked-down products to be disseminated, eHealth interventions are complex sociotechnical systems that require continual optimization, vigilance to monitor and troubleshoot technological issues, and decision rules to refresh content and functionality while maintaining fidelity to core intervention principles. Platform choice and sociotechnical relationships (among end users, implementers, and the technology) heavily influence implementation needs and challenges. We present a checklist of critical implementation questions to address during intervention development. In the absence of a clear path forward for eHealth implementation, deliberate design of an eHealth intervention's service and technological components in tandem with their implementation plans is critical to mitigating barriers to widespread use. The design considerations presented can be used by developers, evaluators, reviewers, and funders to prioritize the pragmatic scalability of eHealth interventions in research.