Effects of Opioid Prescribing Cap Laws on Opioid and Other Pain Treatments Among Persons with Chronic Pain.

BACKGROUND: Many states have adopted laws that limit the amount or duration of opioid prescriptions. These limits often focus on prescriptions for acute pain, but there may be unintended consequences for those diagnosed with chronic pain, including reduced opioid prescribing without substitution of appropriate non-opioid treatments. OBJECTIVE: To evaluate the effects of state opioid prescribing cap laws on opioid and non-opioid treatment among those diagnosed with chronic pain. DESIGN: We used a difference-in-differences approach that accounts for staggered policy adoption. Treated states included 32 states that implemented a prescribing cap law between 2017 and 2019. POPULATION: A total of 480,856 adults in the USA who were continuously enrolled in medical and pharmacy coverage from 2013 to 2019 and diagnosed with a chronic pain condition between 2013 and 2016. MAIN MEASURES: Among individuals with chronic pain in each state: proportion with at least one opioid prescription and with prescriptions of a specific duration or dose, average number of opioid prescriptions, average opioid prescription duration and dose, proportion with at least one non-opioid chronic pain prescription, average number of such prescriptions, proportion with at least one chronic pain procedure, and average number of such procedures. KEY RESULTS: State laws limiting opioid prescriptions were not associated with changes in opioid prescribing, non-opioid medication prescribing, or non-opioid chronic pain procedures among patients with chronic pain diagnoses. CONCLUSIONS: These findings do not support an association between state opioid prescribing cap laws and changes in the treatment of chronic non-cancer pain.

Causal and Associational Language in Observational Health Research: A Systematic Evaluation.

We estimated the degree to which language used in the high-profile medical/public health/epidemiology literature implied causality using language linking exposures to outcomes and action recommendations; examined disconnects between language and recommendations; identified the most common linking phrases; and estimated how strongly linking phrases imply causality. We searched for and screened 1,170 articles from 18 high-profile journals (65 per journal) published from 2010-2019. Based on written framing and systematic guidance, 3 reviewers rated the degree of causality implied in abstracts and full text for exposure/outcome linking language and action recommendations. Reviewers rated the causal implication of exposure/outcome linking language as none (no causal implication) in 13.8%, weak in 34.2%, moderate in 33.2%, and strong in 18.7% of abstracts. The implied causality of action recommendations was higher than the implied causality of linking sentences for 44.5% or commensurate for 40.3% of articles. The most common linking word in abstracts was "associate" (45.7%). Reviewers' ratings of linking word roots were highly heterogeneous; over half of reviewers rated "association" as having at least some causal implication. This research undercuts the assumption that avoiding "causal" words leads to clarity of interpretation in medical research.

Assumptions Not Often Assessed or Satisfied in Published Mediation Analyses in Psychology and Psychiatry.

Mediation analysis aims to investigate the mechanisms of action behind the effects of interventions or treatments. Given the history and common use of mediation in mental health research, we conducted this review to understand how mediation analysis is implemented in psychology and psychiatry and whether analyses adhere to, address, or justify the key underlying assumptions of their approaches. All articles (n = 206) were from top academic psychiatry or psychology journals in the PsycInfo database and were published in English from 2013 to 2018. Information extracted from each article related to study design, covariates adjusted for in the analysis, temporal ordering of variables, and the specific method used to perform the mediation analysis. In most studies, underlying assumptions were not adhered to. Only approximately 20% of articles had full temporal ordering of exposure, mediator, and outcome. Confounding of the exposure-mediator and/or mediator-outcome relationships was controlled for in fewer than half of the studies. In almost none of the articles were the underlying assumptions of their approaches discussed or causal mediation methods used. These results provide insights to how methodologists should aim to communicate methods, and motivation for more outreach to the research community on best practices for mediation analysis.

Scaling Interventions to Manage Chronic Disease: Innovative Methods at the Intersection of Health Policy Research and Implementation Science.

Policy implementation is a key component of scaling effective chronic disease prevention and management interventions. Policy can support scale-up by mandating or incentivizing intervention adoption, but enacting a policy is only the first step. Fully implementing a policy designed to facilitate implementation of health interventions often requires a range of accompanying implementation structures, like health IT systems, and implementation strategies, like training. Decision makers need to know what policies can support intervention adoption and how to implement those policies, but to date research on policy implementation is limited and innovative methodological approaches are needed. In December 2021, the Johns Hopkins ALACRITY Center for Health and Longevity in Mental Illness and the Johns Hopkins Center for Mental Health and Addiction Policy convened a forum of research experts to discuss approaches for studying policy implementation. In this report, we summarize the ideas that came out of the forum. First, we describe a motivating example focused on an Affordable Care Act Medicaid health home waiver policy used by some US states to support scale-up of an evidence-based integrated care model shown in clinical trials to improve cardiovascular care for people with serious mental illness. Second, we define key policy implementation components including structures, strategies, and outcomes. Third, we provide an overview of descriptive, predictive and associational, and causal approaches that can be used to study policy implementation. We conclude with discussion of priorities for methodological innovations in policy implementation research, with three key areas identified by forum experts: effect modification methods for making causal inferences about how policies' effects on outcomes vary based on implementation structures/strategies; causal mediation approaches for studying policy implementation mechanisms; and characterizing uncertainty in systems science models. We conclude with discussion of overarching methods considerations for studying policy implementation, including measurement of policy implementation, strategies for studying the role of context in policy implementation, and the importance of considering when establishing causality is the goal of policy implementation research.

Factors associated with county-level mental health during the COVID-19 pandemic.

The objective of this study is to determine county-level factors associated with anxiety, depression, and isolation during the coronavirus disease 2019 (COVID-19) pandemic. This study used daily data from 23,592,355 respondents of a nationwide Facebook-based survey from April 2020 to July 2021, aggregated to the week-county level to yield 212,581 observations. Mental distress prevalences were modeled using weighted linear mixed-effects models with a county random effect. These models revealed that weekly percentages of mental distress were higher in counties with higher unemployment rates, populations, and education levels; higher percentages of females, young adults, individuals with a medical condition, and individuals very worried about their finances and COVID-19; and lower percentages of individuals who were working outside the home, living with children, without health insurance, and Black. Anxiety peaked in April 2020, depression in October 2020, and isolation in December 2020. Therefore, United States counties experienced the mental health effects of the pandemic differently dependent upon their characteristics, and mental distress prevalence varied across time.

Generalizability of Subgroup Effects.

Generalizability methods are increasingly used to make inferences about the effect of interventions in target populations using a study sample. Most existing methods to generalize effects from sample to population rely on the assumption that subgroup-specific effects generalize directly. However, researchers may be concerned that in fact subgroup-specific effects differ between sample and population. In this brief report, we explore the generalizability of subgroup effects. First, we derive the bias in the sample average treatment effect estimator as an estimate of the population average treatment effect when subgroup effects in the sample do not directly generalize. Next, we present a Monte Carlo simulation to explore bias due to unmeasured heterogeneity of subgroup effects across sample and population. Finally, we examine the potential for bias in an illustrative data example. Understanding the generalizability of subgroup effects may lead to increased use of these methods for making externally valid inferences of treatment effects using a study sample.

State Laws That Require Coprescribing Opioids and Naloxone and Codispensing Practices.

INTRODUCTION: Coprescribing naloxone with opioids could reduce the risk of overdose. By the end of 2020, 8 U.S. states implemented coprescribing laws requiring the prescription of naloxone alongside certain opioid prescriptions. This study examined the impacts of state laws that require coprescribing opioids and naloxone on codispensing practices. METHODS: Data included opioid prescriptions for commercially insured adults between 2014 and 2020. Augmented synthetic control analyses were used to examine the impacts of 8 coprescribing requirement laws implemented between 2017 and 2020 on the proportion of opioid prescription fills with a naloxone coprescription fill. Analyses were completed in spring 2023. RESULTS: Changes in the proportion of opioid prescription fills with a naloxone coprescription fill attributable to the laws varied across states. In 4 states (New Jersey, New Mexico, Rhodes Island, and Virginia), laws were associated with 0.8 (95% CI=0.3, 1.3) to 4.4 (95% CI=3.4, 5.4) percentage point increases in the proportion of opioid prescriptions with a naloxone coprescription fill (p<0.05). There were no statistically significant changes attributable to the other state laws (Arizona, Florida, Vermont, Washington). CONCLUSIONS: Laws requiring coprescribing naloxone with certain opioid prescriptions are associated with small-in-magnitude increases in codispensing in some states. Broadening the categories of opioid prescriptions covered in naloxone coprescribing requirement laws and implementing health system strategies to encourage providers to coprescribe naloxone could help to magnify the impacts of these laws.

Causal mediation analysis: From simple to more robust strategies for estimation of marginal natural (in)direct effects.

This paper aims to provide practitioners of causal mediation analysis with a better understanding of estimation options. We take as inputs two familiar strategies (weighting and model-based prediction) and a simple way of combining them (weighted models), and show how a range of estimators can be generated, with different modeling requirements and robustness properties. The primary goal is to help build intuitive appreciation for robust estimation that is conducive to sound practice. We do this by visualizing the target estimand and the estimation strategies. A second goal is to provide a "menu" of estimators that practitioners can choose from for the estimation of marginal natural (in)direct effects. The estimators generated from this exercise include some that coincide or are similar to existing estimators and others that have not previously appeared in the literature. We note several different ways to estimate the weights for cross-world weighting based on three expressions of the weighting function, including one that is novel; and show how to check the resulting covariate and mediator balance. We use a random continuous weights bootstrap to obtain confidence intervals, and also derive general asymptotic variance formulas for the estimators. The estimators are illustrated using data from an adolescent alcohol use prevention study. R-code is provided.

Clarifying causal mediation analysis: Effect identification via three assumptions and five potential outcomes.

Causal mediation analysis is complicated with multiple effect definitions that require different sets of assumptions for identification. This article provides a systematic explanation of such assumptions. We define five potential outcome types whose means are involved in various effect definitions. We tackle their mean/distribution's identification, starting with the one that requires the weakest assumptions and gradually building up to the one that requires the strongest assumptions. This presentation shows clearly why an assumption is required for one estimand and not another, and provides a succinct table from which an applied researcher could pick out the assumptions required for identifying the causal effects they target. Using a running example, the article illustrates the assembling and consideration of identifying assumptions for a range of causal contrasts. For several that are commonly encountered in the literature, this exercise clarifies that identification requires weaker assumptions than those often stated in the literature. This attention to the details also draws attention to the differences in the positivity assumption for different estimands, with practical implications. Clarity on the identifying assumptions of these various estimands will help researchers conduct appropriate mediation analyses and interpret the results with appropriate caution given the plausibility of the assumptions.

Association Between State Opioid Prescribing Cap Laws and Receipt of Opioid Prescriptions Among Children and Adolescents.

Importance: High-dose and long-duration opioid prescriptions remain relatively common among children and adolescents, but there is insufficient research on the association of state laws limiting the dose and/or duration of opioid prescriptions (referred to as opioid prescribing cap laws) with opioid prescribing for this group. Objective: To examine the association between state opioid prescribing cap laws and the receipt of opioid prescriptions among children and adolescents. Design Setting and Participants: This repeated cross-sectional study used a difference-in-differences approach accounting for staggered policy adoption to assess the association of state opioid prescribing cap laws in the US from January 1, 2013, to December 31, 2019, with receipt of opioid prescriptions among children and adolescents. Analyses were conducted between March 22 and December 15, 2021. Data were obtained from the OptumLabs Data Warehouse, a national commercial insurance claims database. The analysis included 482 118 commercially insured children and adolescents aged 0 to 17 years with full calendar-year continuous insurance enrollment between 2013 and 2019. Individuals were included for every year in which they were continuously enrolled; they did not need to be enrolled for the entire 7-year study period. Those with any cancer diagnosis were excluded from analysis. Exposure: Implementation of a state opioid prescribing cap law between January 1, 2017, and July 1, 2019. This date range allowed analysis of the same number years for both pre-cap and post-cap data. Main Outcomes and Measures: Outcomes of interest included receipt of any opioid prescription and, among those with at least 1 opioid prescription, the mean number of opioid prescriptions, mean morphine milligram equivalents (MMEs) per day, and mean days' supply. Results: Among 482 118 children and adolescents (754 368 person-years of data aggregated to the state-year level), 245 178 (50.9%) were male, with a mean (SD) age of 9.8 (4.8) years at the first year included in the sample (data on race and ethnicity were not collected as part of this data set, which was obtained from insurance billing claims). Overall, 10 659 children and adolescents (2.2%) received at least 1 opioid prescription during the study period. Among those with at least 1 prescription, the mean (SD) number of filled opioid prescriptions was 1.2 (0.8) per person per year. No statistically significant association was found between state opioid prescribing cap laws and any outcome. After opioid prescribing cap laws were implemented, a -0.001 (95% CI, -0.005 to 0.002) percentage point decrease in the proportion of youths receiving any opioid prescription was observed. In addition, percentage point decreases of -0.01 (95% CI, -0.10 to 0.09) in high-dose opioid prescriptions (>50 MMEs per day) and -0.02 (95% CI, -0.12 to 0.08) in long-duration opioid prescriptions (>7 days' supply) were found after cap laws were implemented. Conclusions and Relevance: In this cross-sectional study, no association was observed between state opioid prescribing cap laws and the receipt of opioid prescriptions among children and adolescents. Alternative strategies, such as opioid prescribing guidelines tailored to youths, are needed.

Comparing the performance of statistical methods that generalize effect estimates from randomized controlled trials to much larger target populations.

Policymakers use results from randomized controlled trials to inform decisions about whether to implement treatments in target populations. Various methods - including inverse probability weighting, outcome modeling, and Targeted Maximum Likelihood Estimation - that use baseline data available in both the trial and target population have been proposed to generalize the trial treatment effect estimate to the target population. Often the target population is significantly larger than the trial sample, which can cause estimation challenges. We conduct simulations to compare the performance of these methods in this setting. We vary the size of the target population, the proportion of the target population selected into the trial, and the complexity of the true selection and outcome models. All methods performed poorly when the trial size was only 2% of the target population size or the target population included only 1,000 units. When the target population or the proportion of units selected into the trial was larger, some methods, such as outcome modeling using Bayesian Additive Regression Trees, performed well. We caution against generalizing using these existing approaches when the target population is much larger than the trial sample and advocate future research strives to improve methods for generalizing to large target populations.

State prescribing cap laws' association with opioid analgesic prescribing and opioid overdose.

INTRODUCTION: In response to the role of opioid prescribing in the U.S. opioid crisis, states have enacted laws intended to curb high risk opioid prescribing practices. This study assessed the effects of state prescribing cap laws that limit the dose and/or duration of dispensed opioid prescriptions on opioid prescribing patterns and opioid overdose. METHODS: We identified 1,414,908 adults from a large U.S. administrative insurance claims database. Treatment states included 32 states that implemented a prescribing cap law between 2017 and 2019. Comparison states included 16 states and DC without a prescribing cap law by 2019. A difference-in-differences approach with staggered policy adoption was used to assess effects of these laws on opioid analgesic prescribing and opioid overdose. RESULTS: State opioid prescribing cap laws were not associated with changes in the proportion of people receiving opioid analgesic prescriptions, the dose or duration of opioid prescriptions, or opioid overdose. States with laws that imposed days' supply limits only versus days' supply and dosage limits, as well as with specific law provisions also showed no association with opioid prescribing or opioid overdose outcomes. CONCLUSIONS: State opioid prescribing cap laws did not appear to impact outcomes related to opioid analgesic prescribing or opioid overdose. These findings are potentially due to the limited scope of these laws, which often apply only to a subset of opioid prescriptions and include professional judgment exemptions.

Effects of state opioid prescribing cap laws on opioid prescribing after surgery.

OBJECTIVE: To evaluate the effects of state opioid prescribing cap laws on opioid prescribing after surgery. DATA SOURCES: OptumLabs Data Warehouse administrative claims data covering all 50 states from July 2012 through June 2019. STUDY DESIGN: We included individuals from 20 states that had implemented prescribing cap laws without exemptions for postsurgical pain by June 2019 and individuals from 16 control states plus the District of Columbia. We used a difference-in-differences approach accounting for differential timing in law implementation across states to estimate the effects of state prescribing cap laws on postsurgical prescribing of opioids. Outcome measures included filling an opioid prescription within 30 days after surgery; filling opioid prescriptions of specific doses or durations; and the number, days' supply, daily dose, and pill quantity of opioid prescriptions. To assess the validity of the parallel counterfactual trends assumption, we examined differences in outcome trends between law-implementing and control states in the years preceding law implementation using an equivalence testing framework. DATA COLLECTION/EXTRACTION METHODS: We included the first surgery in the study period for opioid-naïve individuals undergoing one of eight common surgical procedures. PRINCIPAL FINDINGS: State prescribing cap laws were associated with 0.109 lower days' supply of postsurgical opioids on the log scale (95% Confidence Interval [CI]: -0.139, -0.080) but were not associated with the number (Average treatment effect on the treated [ATT]: -0.011; 95% CI: -0.043, 0.021) or daily dose of postsurgical opioid prescriptions (ATT: -0.013; 95% CI: -0.030, 0.005). The negative association observed between prescribing cap laws and the probability of filling a postsurgical opioid prescription (ATT: -0.041; 95% CI: -0.054, -0.028) was likely spurious, given differences between law-implementing and control states in the pre-law period. CONCLUSIONS: Prescribing cap laws appear to have minimal effects on postsurgical opioid prescribing.

Trends in cannabis use among U.S. adults amid the COVID-19 pandemic.

BACKGROUND: The COVID-19 pandemic has had an impact on mental health and alcohol use in the US, however there is little research on its impacts on cannabis use. Considering the role of cannabis as a coping strategy or self-medicating behavior, there is a need to understand how individuals who use cannabis have adapted their use amid the pandemic. Therefore, this study examined changes in self-reported cannabis use among US adults in the context of COVID-19 pandemic by (1) describing trends of use during the first 8 months of the pandemic among adults who used cannabis in this period; and (2) characterizing trends of use within sociodemographic subgroups and by state cannabis policy status. METHODS: The sample consisted of 1,761 US adults who used cannabis at least once during the 8-month study period from the nationally representative Understanding America Study. Linear mixed-effect models were used to model changes in the number of days of past-week cannabis use across 16 waves from March 10, 2020, to November 11, 2020. RESULTS: Compared to early March, the number of days cannabis was used per week was significantly higher at the start of April (ß=0.11, 95% CI=0.03, 0.18) and May (ß=0.21,95% CI=0.05, 0.36). In subsequent months (June - November), the number of days of cannabis use attenuated to levels comparable to March. Trends of cannabis use across the study period generally did not differ across sociodemographic characteristics and state cannabis policy status. CONCLUSION: Though increases in use were marginal among many groups, the evolving pandemic and the growing concern for the mental health of segments of the U.S. population warrant close monitoring of coping behaviors, including substance use.

Problems with evidence assessment in COVID-19 health policy impact evaluation: a systematic review of study design and evidence strength.

INTRODUCTION: Assessing the impact of COVID-19 policy is critical for informing future policies. However, there are concerns about the overall strength of COVID-19 impact evaluation studies given the circumstances for evaluation and concerns about the publication environment. METHODS: We included studies that were primarily designed to estimate the quantitative impact of one or more implemented COVID-19 policies on direct SARS-CoV-2 and COVID-19 outcomes. After searching PubMed for peer-reviewed articles published on 26 November 2020 or earlier and screening, all studies were reviewed by three reviewers first independently and then to consensus. The review tool was based on previously developed and released review guidance for COVID-19 policy impact evaluation. RESULTS: After 102 articles were identified as potentially meeting inclusion criteria, we identified 36 published articles that evaluated the quantitative impact of COVID-19 policies on direct COVID-19 outcomes. Nine studies were set aside because the study design was considered inappropriate for COVID-19 policy impact evaluation (n=8 pre/post; n=1 cross-sectional), and 27 articles were given a full consensus assessment. 20/27 met criteria for graphical display of data, 5/27 for functional form, 19/27 for timing between policy implementation and impact, and only 3/27 for concurrent changes to the outcomes. Only 4/27 were rated as overall appropriate. Including the 9 studies set aside, reviewers found that only four of the 36 identified published and peer-reviewed health policy impact evaluation studies passed a set of key design checks for identifying the causal impact of policies on COVID-19 outcomes. DISCUSSION: The reviewed literature directly evaluating the impact of COVID-19 policies largely failed to meet key design criteria for inference of sufficient rigour to be actionable by policy-makers. More reliable evidence review is needed to both identify and produce policy-actionable evidence, alongside the recognition that actionable evidence is often unlikely to be feasible.

Problems with Evidence Assessment in COVID-19 Health Policy Impact Evaluation: A systematic review of study design and evidence strength.

INTRODUCTION: Assessing the impact of COVID-19 policy is critical for informing future policies. However, there are concerns about the overall strength of COVID-19 impact evaluation studies given the circumstances for evaluation and concerns about the publication environment. This study systematically reviewed the strength of evidence in the published COVID-19 policy impact evaluation literature. METHODS: We included studies that were primarily designed to estimate the quantitative impact of one or more implemented COVID-19 policies on direct SARS-CoV-2 and COVID-19 outcomes. After searching PubMed for peer-reviewed articles published on November 26, 2020 or earlier and screening, all studies were reviewed by three reviewers first independently and then to consensus. The review tool was based on previously developed and released review guidance for COVID-19 policy impact evaluation, assessing what impact evaluation method was used, graphical display of outcomes data, functional form for the outcomes, timing between policy and impact, concurrent changes to the outcomes, and an overall rating. RESULTS: After 102 articles were identified as potentially meeting inclusion criteria, we identified 36 published articles that evaluated the quantitative impact of COVID-19 policies on direct COVID-19 outcomes. The majority (n=23/36) of studies in our sample examined the impact of stay-at-home requirements. Nine studies were set aside because the study design was considered inappropriate for COVID-19 policy impact evaluation (n=8 pre/post; n=1 cross-section), and 27 articles were given a full consensus assessment. 20/27 met criteria for graphical display of data, 5/27 for functional form, 19/27 for timing between policy implementation and impact, and only 3/27 for concurrent changes to the outcomes. Only 1/27 studies passed all of the above checks, and 4/27 were rated as overall appropriate. Including the 9 studies set aside, reviewers found that only four of the 36 identified published and peer-reviewed health policy impact evaluation studies passed a set of key design checks for identifying the causal impact of policies on COVID-19 outcomes. DISCUSSION: The reviewed literature directly evaluating the impact of COVID-19 policies largely failed to meet key design criteria for inference of sufficient rigor to be actionable by policymakers. This was largely driven by the circumstances under which policies were passed making it difficult to attribute changes in COVID-19 outcomes to particular policies. More reliable evidence review is needed to both identify and produce policy-actionable evidence, alongside the recognition that actionable evidence is often unlikely to be feasible.

Clarifying causal mediation analysis for the applied researcher: Defining effects based on what we want to learn.

The incorporation of causal inference in mediation analysis has led to theoretical and methodological advancements-effect definitions with causal interpretation, clarification of assumptions required for effect identification, and an expanding array of options for effect estimation. However, the literature on these results is fast-growing and complex, which may be confusing to researchers unfamiliar with causal inference or unfamiliar with mediation. The goal of this article is to help ease the understanding and adoption of causal mediation analysis. It starts by highlighting a key difference between the causal inference and traditional approaches to mediation analysis and making a case for the need for explicit causal thinking and the causal inference approach in mediation analysis. It then explains in as-plain-as-possible language existing effect types, paying special attention to motivating these effects with different types of research questions, and using concrete examples for illustration. This presentation differentiates 2 perspectives (or purposes of analysis): the explanatory perspective (aiming to explain the total effect) and the interventional perspective (asking questions about hypothetical interventions on the exposure and mediator, or hypothetically modified exposures). For the latter perspective, the article proposes tapping into a general class of interventional effects that contains as special cases most of the usual effect types-interventional direct and indirect effects, controlled direct effects and also a generalized interventional direct effect type, as well as the total effect and overall effect. This general class allows flexible effect definitions which better match many research questions than the standard interventional direct and indirect effects. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Implementing statistical methods for generalizing randomized trial findings to a target population.

Randomized trials are considered the gold standard for assessing the causal effects of a drug or intervention in a study population, and their results are often utilized in the formulation of health policy. However, there is growing concern that results from trials do not necessarily generalize well to their respective target populations, in which policies are enacted, due to substantial demographic differences between study and target populations. In trials related to substance use disorders (SUDs), especially, strict exclusion criteria make it challenging to obtain study samples that are fully "representative" of the populations that policymakers may wish to generalize their results to. In this paper, we provide an overview of post-trial statistical methods for assessing and improving upon the generalizability of a randomized trial to a well-defined target population. We then illustrate the different methods using a randomized trial related to methamphetamine dependence and a target population of substance abuse treatment seekers, and provide software to implement the methods in R using the "generalize" package. We discuss several practical considerations for researchers who wish to utilize these tools, such as the importance of acquiring population-level data to represent the target population of interest, and the challenges of data harmonization.