Visualizations throughout pharmacoepidemiology study planning, implementation, and reporting.

Transparency is increasingly promoted to instill trust in nonrandomized studies using real-world data. Graphics and data visualizations support transparency by aiding communication and understanding, and can inform study design and analysis decisions. However, other than graphical representation of a study design and flow diagrams (e.g., a Consolidated Standards of Reporting Trials [CONSORT] like diagram), specific standards on how to maximize validity and transparency with visualization are needed. This paper provides guidance on how to use visualizations throughout the life cycle of a pharmacoepidemiology study-from initial study design to final report-to facilitate rationalized and transparent decision-making about study design and implementation, and clear communication of study findings. Our intent is to help researchers align their practices with current consensus statements on transparency.

Categorization of COVID-19 severity to determine mortality risk.

PURPOSE: Algorithms for classification of inpatient COVID-19 severity are necessary for confounding control in studies using real-world data. METHODS: Using Healthverity chargemaster and claims data, we selected patients hospitalized with COVID-19 between April 2020 and February 2021, and classified them by severity at admission using an algorithm we developed based on respiratory support requirements (supplemental oxygen or non-invasive ventilation, O2/NIV, invasive mechanical ventilation, IMV, or NEITHER). To evaluate the utility of the algorithm, patients were followed from admission until death, discharge, or a 28-day maximum to report mortality risks and rates overall and by stratified by severity. Trends for heterogeneity in mortality risk and rate across severity classifications were evaluated using Cochran-Armitage and Logrank trend tests, respectively. RESULTS: Among 118 117 patients, the algorithm categorized patients in increasing severity as NEITHER (36.7%), O2/NIV (54.3%), and IMV (9.0%). Associated mortality risk (and 95% CI) was 11.8% (11.6-12.0%) overall and increased with severity [3.4% (3.2-3.5%), 11.5% (11.3-11.8%), 47.3% (46.3-48.2%); p < 0.001]. Mortality rate per 1000 person-days (and 95% CI) was 15.1 (14.9-15.4) overall and increased with severity [5.7 (5.4-6.0), 14.5 (14.2-14.9), 32.7 (31.8-33.6); p < 0.001]. CONCLUSION: As expected, we observed a positive association between the algorithm-defined severity on admission and 28-day mortality risk and rate. Although performance remains to be validated, this provides some assurance that this algorithm may be used for confounding control or stratification in treatment effect studies.

Consensus of recommendations guiding comparative effectiveness research methods.

PURPOSE: Because of an increasing demand for quality comparative effectiveness research (CER), methods guidance documents have been published, such as those from the Agency for Healthcare Research and Quality (AHRQ) and the Patient-Centered Outcomes Research Institute (PCORI). Our objective was to identify CER methods guidance documents and compare them to produce a summary of important recommendations which could serve as a consensus of CER method recommendations. METHODS: We conducted a systematic literature review to identify CER methods guidance documents published through 2014. Identified documents were analyzed for methods guidance recommendations. Individual recommendations were categorized to determine the degree of overlap. RESULTS: We identified nine methods guidance documents, which contained a total of 312 recommendations, 97% of which were present in two or more documents. All nine documents recommended transparency and adaptation for relevant stakeholders in the interpretation and dissemination of results. Other frequently shared CER methods recommendations included: study design and operational definitions should be developed a priori and allow for replication (n = 8 documents); focus on areas with gaps in current clinical knowledge that are relevant to decision-makers (n = 7); validity of measures, instruments, and data should be assessed and discussed (n = 7); outcomes, including benefits and harms, should be clinically meaningful, and objectively measured (n = 7). Assessment for and strategies to minimize bias (n = 6 documents), confounding (n = 6), and heterogeneity (n = 4) were also commonly shared recommendations between documents. CONCLUSIONS: We offer a field-consensus guide based on nine CER methods guidance documents that will aid researchers in designing CER studies and applying CER methods. Copyright © 2016 John Wiley & Sons, Ltd.

Methodological comparison of marginal structural model, time-varying Cox regression, and propensity score methods: the example of antidepressant use and the risk of hip fracture.

BACKGROUND: Observational studies including time-varying treatments are prone to confounding. We compared time-varying Cox regression analysis, propensity score (PS) methods, and marginal structural models (MSMs) in a study of antidepressant [selective serotonin reuptake inhibitors (SSRIs)] use and the risk of hip fracture. METHODS: A cohort of patients with a first prescription for antidepressants (SSRI or tricyclic antidepressants) was extracted from the Dutch Mondriaan and Spanish Base de datos para la Investigación Farmacoepidemiológica en Atención Primaria (BIFAP) general practice databases for the period 2001-2009. The net (total) effect of SSRI versus no SSRI on the risk of hip fracture was estimated using time-varying Cox regression, stratification and covariate adjustment using the PS, and MSM. In MSM, censoring was accounted for by inverse probability of censoring weights. RESULTS: The crude hazard ratio (HR) of SSRI use versus no SSRI use on hip fracture was 1.75 (95%CI: 1.12, 2.72) in Mondriaan and 2.09 (1.89, 2.32) in BIFAP. After confounding adjustment using time-varying Cox regression, stratification, and covariate adjustment using the PS, HRs increased in Mondriaan [2.59 (1.63, 4.12), 2.64 (1.63, 4.25), and 2.82 (1.63, 4.25), respectively] and decreased in BIFAP [1.56 (1.40, 1.73), 1.54 (1.39, 1.71), and 1.61 (1.45, 1.78), respectively]. MSMs with stabilized weights yielded HR 2.15 (1.30, 3.55) in Mondriaan and 1.63 (1.28, 2.07) in BIFAP when accounting for censoring and 2.13 (1.32, 3.45) in Mondriaan and 1.66 (1.30, 2.12) in BIFAP without accounting for censoring. CONCLUSIONS: In this empirical study, differences between the different methods to control for time-dependent confounding were small. The observed differences in treatment effect estimates between the databases are likely attributable to different confounding information in the datasets, illustrating that adequate information on (time-varying) confounding is crucial to prevent bias.

Epidemiology's continuing contribution to public health: The power of "Then and Now".

The 47th annual meeting of the Society for Epidemiologic Research hosted 17 invited speakers charged by the Executive Committee with presenting some of the many ways that epidemiologists have improved the health of the general population. There were 9 "Then and Now" sessions that were structured to focus on how early epidemiologists overcame research hurdles and advanced health through innovative strategies. For most topics, a longstanding expert was paired with an excellent contemporary epidemiologist working in the area, and both were given the freedom to deliver an integrated story about epidemiology's temporal role in protecting and promoting public health. Epidemiologic discoveries in cardiovascular, cancer, and perinatal epidemiology were discussed on day 1, followed by discussions of accomplishments in reducing exposures that adversely impact health (nutrition, environment/occupation, and tobacco use) on day 2. Topics with relevancy for many aspects of epidemiology were presented on day 3, including infectious diseases, social forces, and causal thinking in epidemiologic research. Given the large number of outstanding senior and junior epidemiologists that attended the meeting, choosing speakers was a unique challenge. What became evident from all sessions was the passion that epidemiologists have for population health, tempered with concerns for remaining true to epidemiologic principles, the timely adoption of innovative methods, and the responsible interpretation of research findings.

Toward a clarification of the taxonomy of "bias" in epidemiology textbooks.

Epidemiology textbooks typically divide biases into 3 general categories-confounding, selection bias, and information bias. Despite the ubiquity of this categorization, authors often use these terms to mean different things. This hinders communication among epidemiologists and confuses students who are just learning about the field. To understand the sources of this problem, we reviewed current general epidemiology textbooks to examine how the authors defined and categorized biases. We found that much of the confusion arises from different definitions of "validity" and from a mixing of 3 overlapping organizational features in defining and differentiating among confounding, selection bias, and information bias: consequence, the result of the problem; cause, the processes that give rise to the problem; and cure, how these biases can be addressed once they occur. By contrast, a consistent taxonomy would provide (1) a clear and consistent definition of what unites confounding, selection bias, and information bias and (2) a clear articulation and consistent application of the feature that distinguishes these categories. Based on a distillation of these textbook discussions, we provide an example of a taxonomy that we think meets these criteria.

An organizational schema for epidemiologic causal effects.

Epidemiologic textbooks and methodological papers define multiple causal effects. These causal effects can differ substantially; yet, the causal effect of interest is rarely specified in published epidemiologic studies perhaps because their distinctions are underappreciated. Here, we provide an organizational schema that distinguishes causal effects based on six characteristics. We use simple numeric examples to demonstrate the variability across effects and show why specifying the causal effect is necessary for an accurate intervention interpretation even under the simplest scenarios. The objective of our schema was to illuminate the distinguishing characteristics of various causal effects and clarify their interpretation, thus guiding epidemiologists in choosing an appropriate causal effect to estimate.

Linking clinical trial participants to their U.S. real-world data through tokenization: A practical guide.

In drug development, the use of real-world data (RWD) has augmented our understanding of patients' health care experiences and the effects of treatments beyond clinical trials. Although electronic health record (EHR) data integration at clinical trial sites is a widely adopted practice, primarily for recruitment and data capture, a challenge to data utility is the fragmentation of health data across different sources. Linking RWD sources to each other and to trial data -- while preserving patient privacy through tokenization -- aids in filling evidence gaps with outcome data and facilitates the generalization of effects from controlled trial environments to real-world settings. This paper describes the applications of RWD linkage and how they benefit both clinical development and real-world decision-making. Trial benefits include improving interpretability and generalizability (e.g., by remediating missing data or losses to follow-up), extending follow-up beyond trial closeout, and characterizing the applicability of trial results to under-represented groups. The operational aspects of linking trial data to RWD are addressed, emphasizing the importance of using privacy-preserving record linking systems with established metrics of accuracy and precision, managing consent, and providing the necessary training and resources at trial sites to inform participants about providing access to their RWD through data linkage.

SURF: A Screening Tool (for Sponsors) to Evaluate Whether Using Real-World Data to Support an Effectiveness Claim in an FDA Application Has Regulatory Feasibility.

Based on recent guidance and publicly available approvals, the US Food and Drug Administration (FDA) has demonstrated its openness to considering evidence of effectiveness from real-world data (RWD) and nonrandomized studies (or "real-world evidence (RWE)") in its decision making. Through analysis of the FDA approvals, several authors have identified methodologic issues commonly discussed by FDA reviewers. However, in our analysis of FDA guidance and use cases, acceptability of RWE also critically depends on whether the characteristics of the clinical development program align with circumstances in which the FDA may have flexibility in considering evidence from real-world study designs relative to more robust designs. Successful use of RWD requires advance planning to allocate the necessary resources and time to undertake principled epidemiology approaches to study design, data selection, and implementation of analyses as well as address regulatory feedback. Thus, sponsors benefit from early identification of programs in which successful RWD use is more likely, to ensure efficient use of resources required for the next steps of scientific feasibility assessment. We developed SURF, a screening tool intended to help a sponsor decide whether to prioritize resources for further exploring the feasibility of using an RWD approach to meet the FDA's effectiveness evidentiary standards for a particular clinical development program. Here, we provide an analysis of FDA guidance, highlighting the circumstances in which RWD approaches may be most acceptable, and demonstrate the use of this screening tool, including the rationale for its construction, types of evidence needed, and application to publicly available approvals as support of concept.

A Structured Process to Identify Fit-for-Purpose Study Design and Data to Generate Valid and Transparent Real-World Evidence for Regulatory Uses.

Generating evidence from real-world data requires fit-for-purpose study design and data. In addition to validity, decision makers require transparency in the reasoning that underlies study design and data source decisions. The 2019 Structured Preapproval and Postapproval Comparative Study Design Framework to Generate Valid and Transparent Real-World Evidence (SPACE) and the 2021 Structured Process to Identify Fit-For-Purpose Data (SPIFD)-intended to be used together-provide a step-by-step guide to identify decision grade, fit-for-purpose study design and data. In this update (referred to as "SPIFD2" to encompass both the design and data aspects) we provide an update to these frameworks that combines the templates into one, more explicitly calls for articulation of the hypothetical target trial and sources of bias that may arise in the real-world emulation, and provides explicit references to the Structured Template and Reporting Tool for Real-World Evidence (STaRT-RWE) tables that we suggest using immediately after invoking the SPIFD2 framework. Following the steps recommended in the SPIFD2 process requires due diligence on the part of the researcher to ensure that every aspect of study design and data selection is rationalized and supported by evidence. The resulting stepwise documentation enables reproducibility and clear communication with decision makers, and it increases the likelihood that the evidence generated is valid, fit-for-purpose, and sufficient to support healthcare and regulatory decisions.

A Descriptive Cohort Study of Drug Utilization Patterns Among Patients Hospitalized With Coronavirus Disease 2019 in the United States, January 2021-February 2022.

Background: There is a dearth of drug utilization studies for coronavirus disease 2019 (COVID-19) treatments in 2021 and beyond after the introduction of vaccines and updated guidelines; such studies are needed to contextualize ongoing COVID-19 treatment effectiveness studies during these time periods. This study describes utilization patterns for corticosteroids, interleukin-6 (IL-6) inhibitors, Janus kinase inhibitors, and remdesivir among hospitalized adults with COVID-19, over the entire hospitalization, and within hospitalization periods categorized by respiratory support requirements. Methods: This descriptive cohort study included United States adults hospitalized with COVID-19 admitted from 1 January 2021 through 1 February 2022; data included HealthVerity claims and hospital chargemaster. The number and distribution of patients were reported for the first 3 drug regimen lines initiated. Results: The cohort included 51 066 patients; the most common initial drug regimens were corticosteroids (23.4%), corticosteroids plus remdesivir (25.1%), and remdesivir (4.4%). IL-6 inhibitors and Janus kinase inhibitors were included in later drug regimens and were more commonly administered with both corticosteroids and remdesivir than with corticosteroids alone. IL-6 inhibitors were more commonly administered than Janus kinase inhibitors when patients received high-flow oxygen or ventilation. Conclusions: These findings provide important context for comparative studies of COVID-19 treatments with study periods extending into 2021 and later. While prescribing generally aligned with National Institutes of Health COVID-19 treatment guidelines during this period, these findings suggest that prescribing preference, potential confounding by indication, and confounding by prior/concomitant use of other therapeutics should be considered in the design and interpretation of comparative studies.

Extending the sufficient component cause model to describe the Stable Unit Treatment Value Assumption (SUTVA).

Causal inference requires an understanding of the conditions under which association equals causation. The exchangeability or no confounding assumption is well known and well understood as central to this task. More recently the epidemiologic literature has described additional assumptions related to the stability of causal effects. In this paper we extend the Sufficient Component Cause Model to represent one expression of this stability assumption--the Stable Unit Treatment Value Assumption. Approaching SUTVA from an SCC model helps clarify what SUTVA is and reinforces the connections between interaction and SUTVA.

Treatment effectiveness in a rare oncology indication: Lessons from an external control cohort study.

Real-world data (RWD) reflecting patient treatment in routine clinical practice can be used to develop external control groups for single-arm trials. External controls can provide valuable benchmark results on potential comparator drug effectiveness, particularly in rare indications when randomized controlled trials are either infeasible or unethical. This paper describes lessons learned from a descriptive real-world external control cohort study conducted to provide benchmark data for a single-arm clinical trial in a rare oncology biomarker driven disease. Conducting external control cohort studies to evaluate treatment effectiveness in rare indications likely will present data and analysis challenges as seen in the example study. However, there are mitigating measures that can be applied in the study design, identification of RWD sources, and data analysis. The lessons learned and reported here with a proposal of an external control study framework can provide guidance for future research in this area, and may be applicable as well in other rare indications. Taking these learnings into consideration, the use of real-world external controls to contextualize treatment effectiveness in rare indications is a valuable approach and warrants further application in the future.

The Structured Process to Identify Fit-For-Purpose Data: A Data Feasibility Assessment Framework.

To complement real-world evidence (RWE) guidelines, the 2019 Structured Preapproval and Postapproval Comparative study design framework to generate valid and transparent real-world Evidence (SPACE) framework elucidated a process for designing valid and transparent real-world studies. As an extension to SPACE, here, we provide a structured framework for conducting feasibility assessments-a step-by-step guide to identify decision grade, fit-for-purpose data, which complements the United States Food and Drug Administration (FDA)'s framework for a RWE program. The process was informed by our collective experience conducting systematic feasibility assessments of existing data sources for pharmacoepidemiology studies to support regulatory decisions. Used with the SPACE framework, the Structured Process to Identify Fit-For-Purpose Data (SPIFD) provides a systematic process for conducting feasibility assessments to determine if a data source is fit for decision making, helping ensure justification and transparency throughout study development, from articulation of a specific and meaningful research question to identification of fit-for-purpose data and study design.

Real-world comparative effectiveness of mRNA-1273 and BNT162b2 vaccines among immunocompromised adults identified in administrative claims data in the United States.

INTRODUCTION: Head-to-head studies comparing COVID-19 mRNA vaccine effectiveness in immunocompromised individuals, who are vulnerable to severe disease are lacking, as large sample sizes are required to make meaningful inferences. METHODS: This observational comparative effectiveness study was conducted in closed administrative claims data from the US HealthVerity database (December 11, 2020-January 10, 2022, before omicron). A 2-dose mRNA-1273 versus BNT162b2 regimen was assessed for preventing medically-attended breakthrough COVID-19 diagnosis and hospitalizations among immunocompromised adults. Inverse probability of treatment weighting was applied to balance baseline characteristics between vaccine groups. Incidence rates from patient-level data and hazard ratios (HRs) using weighted Cox proportional hazards models were calculated. RESULTS: Overall, 57,898 and 66,981 individuals received a 2-dose regimen of mRNA-1273 or BNT161b2, respectively. Among the weighted population, mean age was 51 years, 53 % were female, and baseline immunodeficiencies included prior blood transplant (8%-9%), prior organ transplant (7%), active cancer (12%-13%), primary immunodeficiency (5-6%), HIV (20%-21%), and immunosuppressive therapy use (60%-61%). Rates per 1,000 person-years (PYs; 95% confidence intervals [CI]s) of breakthrough medically-attended COVID-19 were 25.82 (23.83-27.97) with mRNA-1273 and 30.98 (28.93, 33.18) with BNT162b2 (HR, 0.83; 95% CI, 0.75-0.93). When requiring evidence of an antigen or polymerase chain reaction test before COVID-19 diagnosis, the HR for medically-attended COVID-19 was 0.78 (0.67-0.92). Breakthrough COVID-19 hospitalization rates per 1,000 PYs (95% CI) were 3.66 (2.96-4.51) for mRNA-1273 and 4.68 (3.91-5.59) for BNT162b2 (HR, 0.78; 0.59-1.03). Utilizing open and closed claims for outcome capture only, or both cohort entry/outcome capture, produced HRs (95% CIs) for COVID-19 hospitalization of 0.72 (0.57-0.92) and 0.66 (0.58-0.76), respectively. CONCLUSIONS: Among immunocompromised adults, a 2-dose mRNA-1273 regimen was more effective in preventing medically-attended COVID-19 in any setting (inpatient and outpatient) than 2-dose BNT162b2. Results were similar for COVID-19 hospitalization, although statistical power was limited when using closed claims only. STUDY REGISTRATION: NCT05366322.

Describing characteristics and treatment patterns of patients hospitalized with COVID-19 by race and ethnicity in a national RWD during the early months of the pandemic.

OBJECTIVE: To describe differences by race and ethnicity in treatment patterns among hospitalized COVID-19 patients in the US from March-August 2020. METHODS: Among patients in de-identified Optum electronic health record data hospitalized with COVID-19 (March-August 2020), we estimated odds ratios of receiving COVID-19 treatments of interest (azithromycin, dexamethasone, hydroxychloroquine, remdesivir, and other steroids) at hospital admission, by race and ethnicity, after adjusting for key covariates of interest. RESULTS: After adjusting for key covariates, Black/African American patients were less likely to receive dexamethasone (adj. OR [95% CI]: 0.83 [0.71, 0.96]) and more likely to receive other steroids corticosteroids (adj. OR [95% CI]: 2.13 [1.90, 2.39]), relative to White patients. Hispanic/Latino patients were less likely to receive dexamethasone than Not Hispanic/Latino patients (adj. OR [95% CI]: 0.69 [0.58, 0.82]). CONCLUSIONS: Our findings suggest that COVID-19 treatments patients received in Optum varied by race and ethnicity after adjustment for other possible explanatory factors. In the face of rapidly evolving treatment landscapes, policies are needed to ensure equitable access to novel and repurposed therapeutics to avoid disparities in care by race and ethnicity.

Organized structure of real-world evidence best practices: moving from fragmented recommendations to comprehensive guidance.

Decision-makers have become increasingly interested in incorporating real-world evidence (RWE) into their decision-making process. Due to concerns regarding the reliability and quality of RWE, stakeholders have issued numerous recommendation documents to assist in setting RWE standards. The fragmented nature of these documents poses a challenge to researchers and decision-makers looking for guidance on what is 'high-quality' RWE and how it can be used in decision-making. We offer researchers and decision-makers a structure to organize the landscape of RWE recommendations and identify consensus and gaps in the current recommendations. To provide researchers with a much needed pathway for generating RWE, we discuss how decision-makers can move from fragmented recommendations to comprehensive guidance.

Real-World Evidence for Assessing Pharmaceutical Treatments in the Context of COVID-19.

The emergence and global spread of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has resulted in an urgent need for evidence on medical interventions and outcomes of the resulting disease, coronavirus disease 2019 (COVID-19). Although many randomized controlled trials (RCTs) evaluating treatments and vaccines for COVID-19 are already in progress, the number of clinical questions of interest greatly outpaces the available resources to conduct RCTs. Therefore, there is growing interest in whether nonrandomized real-world evidence (RWE) can be used to supplement RCT evidence and aid in clinical decision making, but concerns about nonrandomized RWE have been highlighted by a proliferation of RWE studies on medications and COVID-19 outcomes with widely varying conclusions. The objective of this paper is to review some clinical questions of interest, potential data types, challenges, and merits of RWE in COVID-19, resulting in recommendations for nonrandomized RWE designs and analyses based on established RWE principles.

Determining the Suitability of Registries for Embedding Clinical Trials in the United States: A Project of the Clinical Trials Transformation Initiative.

BACKGROUND: Patient registries are organized systems that use observational methods to collect uniform data on specified outcomes in a population defined by a particular disease, condition, or exposure. Data collected in registries often coincide with data that could support clinical trials. Integrating clinical trials within registries to create registry-embedded clinical trials offers opportunities to reduce duplicative data collection, identify and recruit patients more efficiently, decrease time to database lock, accelerate time to regulatory decision-making, and reduce clinical trial costs. This article describes a project of the Clinical Trials Transformation Initiative (CTTI) intended to help clinical trials researchers determine when a registry could potentially serve as the platform for the conduct of a clinical trial. METHODS: Through a review of registry-embedded clinical trials and commentaries, semi-structured interviews with experts, and a multi-stakeholder expert meeting, the project team addressed how to identify and describe essential registry characteristics, practices, and processes required to for conducting embedded clinical trials intended for regulatory submissions in the United States. RESULTS: Recommendations, suggested practices, and decision trees that facilitate the assessment of whether a registry is suitable for embedding clinical trials were developed, as well as considerations for the design of new registries. Essential registry characteristics include relevancy, robustness, reliability, and assurance of patient protections. CONCLUSIONS: The project identifies a clear role for registries in creating a sustainable and reusable infrastructure to conduct clinical trials. Adoption of these recommendations will facilitate the ability to perform high-quality and efficient prospective registry-based clinical trials.