Building transparency and reproducibility into the practice of pharmacoepidemiology and outcomes research.

Real-world evidence (RWE) studies are increasingly used to inform policy and clinical decisions. However, there remain concerns about the credibility and reproducibility of RWE studies. Observational researchers should highlight the level of transparency of their studies by providing a succinct statement addressing study transparency with the publication of every paper, poster, or presentation that reports on a RWE study. In this paper, we propose a framework for an explicit transparency statement that declares the level of transparency a given RWE study has achieved across five key domains: 1) protocol, 2) pre-registration, 3) data, 4) code sharing, and 5) reporting checklists.

Emulating Randomized Trials by Observational Database Studies: The RCT-DUPLICATE Initiative in COPD and Asthma.

Observational studies are increasingly used to provide real-world evidence in regulatory decision-making. The RCT-DUPLICATE initiative conducted observational studies emulating two trials in patients with asthma and three in COPD. For each trial, new-user cohorts were constructed from two US healthcare claims databases, comparing initiators of the study and comparator drugs, matched on propensity scores. Proportional hazards models were used to compare the treatments on study outcomes. The observational studies involved more subjects than the corresponding trials, with treatment arms well-matched on baseline characteristics. An asthma example involved emulation of the 26-week FDA-mandated D5896 trial. With 6,494 asthma patients per arm, the hazard ratio (HR) of a serious asthma-related event with budesonide-formoterol versus budesonide was 1.29 (95% CI: 0.63-2.65), compared with 1.07 (95% CI: 0.70-1.65) in the trial. A COPD example is the emulation of the one-year IMPACT trial. With 4,365 COPD patients per arm, the HR of a COPD exacerbation with triple therapy versus dual bronchodilators was 1.08 (95% CI: 1.00-1.17), compared with 0.84 (95% CI: 0.78-0.91) in the trial. We found mainly discordant results between observational analyses and randomized trials, likely from the forced discontinuation of treatments prior to randomization in the trials, not mimicable in the observational analyses.

Tree-based scan statistics to generate drug repurposing hypotheses: a test case using sodium-glucose cotransporter-2 inhibitors.

Most drug repurposing studies using real-world data focused on validating, instead of generating, hypotheses. We used tree-based scan statistics to generate repurposing hypotheses for sodium-glucose cotransporter-2 inhibitors (SGLT2i). We used an active-comparator, new-user design to create a 1:1 propensity-score matched cohort of SGLT2i and dipeptidyl peptidase-4 inhibitors (DPP4i) initiators in the MerativeTM MarketScan® Research Databases. Tree-based scan statistics were estimated across an ICD-10-CM-based hierarchical outcome tree using incident outcomes identified from hospital and outpatient diagnoses. We used an adjusted P≤0.01 as the threshold for statistical alert to prioritize associations for evaluation as repurposing signals. We varied the analyses by tree size, scanning level, and clinical settings for outcomes. There were 80,510 matched SGLT2i-DPP4i initiator pairs with 215,333 outcomes among SGLT2i initiators and 223,428 outcomes among DPP4i initiators. There were 18 prioritized associations, which included chronic kidney disease (P=0.0001), an expected signal, and anemia (P=0.0001). Heart failure (P=0.0167), another expected signal, was identified slightly beyond the statistical alert threshold. Narrowing the outcome tree, scanning at different tree levels, and including outcomes from different clinical settings influenced the scan statistics. We identified signals aligning with recently approved indications of SGLT2i, plus potential repurposing signals supported by existing evidence but requiring future validation.

Hierarchical Clustering Analysis to Inform Classification of Congenital Malformations for Surveillance of Medication Safety in Pregnancy.

There is growing interest in the secondary use of healthcare data to evaluate medication safety in pregnancy. Tree-based scan statistics (TBSS) offer an innovative approach to help identify potential safety signals. TBSS utilize hierarchically organized outcomes, generally based on existing clinical coding systems that group outcomes by organ system. When assessing teratogenicity, such groupings often lack a sound embryologic basis given the etiologic heterogeneity of congenital malformations. The study objective was to enhance the grouping of congenital malformations to be used in scanning approaches through implementation of hierarchical clustering analysis (HCA) and to pilot test an HCA-enhanced TBSS approach for medication safety surveillance in pregnancy in two test cases using >4.2 million mother-child dyads from two US-nationwide databases. HCA identified (1) malformation combinations belonging to the same organ system already grouped in existing classifications, (2) known combinations across different organ systems not previously grouped, (3) unknown combinations not previously grouped, and (4) malformations seemingly standing on their own. Testing the approach with valproate and topiramate identified expected signals, and a signal for an HCA-cluster missed by traditional classification. Augmenting existing classifications with clusters identified through large data exploration may be promising when defining phenotypes for surveillance and causal inference studies.

Targeted Learning with an Undersmoothed Lasso Propensity Score Model for Large-Scale Covariate Adjustment in Healthcare Database Studies.

Lasso regression is widely used for large-scale propensity score (PS) estimation in healthcare database studies. In these settings, previous work has shown that undersmoothing (overfitting) Lasso PS models can improve confounding control, but it can also cause problems of non-overlap in covariate distributions. It remains unclear how to select the degree of undersmoothing when fitting large-scale Lasso PS models to improve confounding control while avoiding issues that can result from reduced covariate overlap. Here, we used simulations to evaluate the performance of using collaborative-controlled targeted learning to data-adaptively select the degree of undersmoothing when fitting large-scale PS models within both singly and doubly robust frameworks to reduce bias in causal estimators. Simulations showed that collaborative learning can data-adaptively select the degree of undersmoothing to reduce bias in estimated treatment effects. Results further showed that when fitting undersmoothed Lasso PS-models, the use of cross-fitting was important for avoiding non-overlap in covariate distributions and reducing bias in causal estimates.

Unified and Versatile Multiplex Platform for Expedited Method Development and Comprehensive Characterization of Therapeutic Peptides.

Agile analytical approaches are needed for fast and comprehensive characterization of peptide drug candidates. In this study, a unified and versatile multiplex platform was developed to expedite method development and enable the routine determination of multiple quality attributes simultaneously. The platform integrates the automation of size exclusion chromatography (SEC), reversed phase liquid chromatography coupled to reversed phase liquid chromatography (RPLC-RPLC), and hydrophilic interaction liquid chromatography hyphenated to charged aerosol detection (HILIC-CAD). Various therapeutic peptide constructs, including macrocyclic peptides and disulfide constrained peptides, across different lots were studied. The effect of the mobile phase acetonitrile content on the impurity profiles was systematically studied using two SEC columns. A prototype MaxPeak Premier SEC 125 Å column packed with BEH PEO particles achieved the separation of impurities (>2.0% area), whereas no impurities could be observed with an ACQUITY UPLC Protein BEH SEC 125 Å column packed with BEH diol particles. Comprehensive impurity profiling and expedited method development was performed utilizing RPLC-RPLC. Each peptide was analyzed by a combination of 12 conditions in the second dimension, using four columns with octadecyl, phenyl-hexyl, and cyano bonded phases, and three mobile phases with various solvents, modifiers, and pH compositions. Additionally, a HILIC-CAD method was developed for the quantification of TFA, commonly present in peptide products.

The FAIRification of research in real-world evidence: A practical introduction to reproducible analytic workflows using Git and R.

Transparency and reproducibility are major prerequisites for conducting meaningful real-world evidence (RWE) studies that are fit for decision-making. Many advances have been made in the documentation and reporting of study protocols and results, but the principles for version control and sharing of analytic code in RWE are not yet as established as in other quantitative disciplines like computational biology and health informatics. In this practical tutorial, we aim to give an introduction to distributed version control systems (VCS) tailored toward the FAIR (Findable, Accessible, Interoperable, and Reproducible) implementation of RWE studies. To ease adoption, we provide detailed step-by-step instructions with practical examples on how the Git VCS and R programming language can be implemented into RWE study workflows to facilitate reproducible analyzes. We further discuss and showcase how these tools can be used to track changes, collaborate, disseminate, and archive RWE studies through dedicated project repositories that maintain a complete audit trail of all relevant study documents.

An open-source implementation of tree-based scan statistics.

PURPOSE: We develop an open-source R package to implement tree-based scan statistics (TBSS) analyses. METHODS: TBSS are data mining methods used by the United States Food and Drug Administration and the Centers for Disease Control. They simultaneously screen thousands of hierarchically aggregated outcomes to identify unsuspected adverse effects of drugs or vaccines, accounting for multiple comparisons. The general structure of TBSS is highly adaptable, with four essential components: (1) a hierarchical outcome structure, (2) a test statistic to be computed for each element of the hierarchy, (3) an algorithm to generate data replicates under a null distribution, and (4) observed outcomes at the lower level of the hierarchy. We encode the general TBSS framework in a convenient R package that offers user-friendly functions for the most used TBSS methods. To illustrate the performance of our software, we evaluated two examples of archetypical TBSS analyses previously analyzed using proprietary, closed-source TreeScan™ software. The first considers the risk of congenital malformations associated with first-trimester exposure to valproate, and the second compares exposure to newly prescribed canagliflozin with a dipeptidyl peptidase 4 inhibitor in adults affected by type 2 diabetes. RESULTS: The results of the original studies are replicated. CONCLUSIONS: The diffusion of an open-source implementation of TBSS can enhance innovation of TBSS methods and foster collaborations. We offer an intuitive R package implementing standard TBSS methods with accompanying tutorials. Our unified object-oriented implementation allows expert users to extend the framework, introduce new features, or enhance existing ones.

Emulation of randomized trials of direct oral anticoagulants with claims data and implications for new Factor XI inhibitors.

Direct oral anticoagulants (DOACs) revolutionized the management of thromboembolic disorders. Clinical care may be further improved as Factor XIs undergo large-scale outcome trials. What role can non-randomized database studies play in expediting understanding of these drugs in clinical practice? The RCT-DUPLICATIVE Initiative emulated the design of eight DOAC randomized clinical trials (RCT) using non-randomized claims database studies. RCT study design parameters and measurements were closely emulated by the database studies and produced highly concordant results. The results of the single database study that did not meet all agreement metrics with the specific RCT it was emulating were aligned with a meta-analysis of six trials studying similar questions, suggesting the trial result was an outlier. Well-designed database studies using fit-for-purpose data came to the same conclusions as DOAC trials, illustrating how database studies could complement RCTs for Factor XI inhibitors-by accelerating insights in underrepresented populations, demonstrating effectiveness and safety in clinical practice, and testing broader indications.

Stratified analysis in comparative effectiveness studies that emulate randomized trials.

PURPOSE: For observational cohort studies that employ matching by propensity scores (PS), preliminary stratification by consequential predictors of outcome better emulates stratified randomization and potentially reduces variance and bias through relaxed dependence on modeling assumptions. We assessed the impact of pre-stratification in two real-life examples. For both, prior evidence from placebo-controlled randomized clinical trials (RCTs) suggested small or no risk reduction, but observational analysis suggested protection, presumably the result of confounding bias. STUDY DESIGN AND SETTING: The study populations consisted of Medicare beneficiaries (2014-18) with type 2 diabetes initiating either (i) empagliflozin versus dipeptidyl peptidase-4 inhibitors (DPP-4i) or (ii) empagliflozin versus glucagon-like peptide-1 receptor agonists (GLP-1RA). The outcome was myocardial infarction or stroke. We estimated hazard ratios (HR) and rate differences (RD) after controlling for 143 pre-exposure covariates via 1:1 PS matching after (1) PS estimation in the total cohort (total-cohort PS-matching) and (2) PS estimation separately by baseline cardiovascular disease (stratified PS matching). RESULTS: Stratified PS matching resulted in HRs that exceeded those from total-cohort PS-matching by 13% and 9%, respectively, for the comparisons of empagliflozin to DPP-4i and GLP-1RA. Against both comparators, HRs and RDs after stratified PS matching were closer to the null, with slightly higher variances (2%-3%) than those after total-cohort PS matching. CONCLUSION: Stratified PS matching produced effect estimates closer to the expected trial findings than total-cohort PS matching. The price paid in increased variance was minimal.

A Dynamic Prognostic Model for Identifying Vulnerable COVID-19 Patients at High Risk of Rapid Deterioration.

PURPOSE: We aimed to validate and, if performance was unsatisfactory, update the previously published prognostic model to predict clinical deterioration in patients hospitalized for COVID-19, using data following vaccine availability. METHODS: Using electronic health records of patients ≥18 years, with laboratory-confirmed COVID-19, from a large care-delivery network in Massachusetts, USA, from March 2020 to November 2021, we tested the performance of the previously developed prediction model and updated the prediction model by incorporating data after availability of COVID-19 vaccines. We randomly divided data into development (70%) and validation (30%) cohorts. We built a model predicting worsening in a published severity scale in 24 h by LASSO regression and evaluated performance by c-statistic and Brier score. RESULTS: Our study cohort consisted of 8185 patients (Development: 5730 patients [mean age: 62; 44% female] and Validation: 2455 patients [mean age: 62; 45% female]). The previously published model had suboptimal performance using data after November 2020 (N = 4973, c-statistic = 0.60. Brier score = 0.11). After retraining with the new data, the updated model included 38 predictors including 18 changing biomarkers. Patients hospitalized after Jun 1st, 2021 (when COVID-19 vaccines became widely available in Massachusetts) were younger and had fewer comorbidities than those hospitalized before. The c-statistic and Brier score were 0.77 and 0.13 in the development cohort, and 0.73 and 0.14 in the validation cohort. CONCLUSION: The characteristics of patients hospitalized for COVID-19 differed substantially over time. We developed a new dynamic model for rapid progression with satisfactory performance in the validation set.

Sharing Is Caring? International Society for Pharmacoepidemiology Review and Recommendations for Sharing Programming Code.

PURPOSE: There is increasing recognition of the importance of transparency and reproducibility in scientific research. This study aimed to quantify the extent to which programming code is publicly shared in pharmacoepidemiology, and to develop a set of recommendations on this topic. METHODS: We conducted a literature review identifying all studies published in Pharmacoepidemiology and Drug Safety (PDS) between 2017 and 2022. Data were extracted on the frequency and types of programming code shared, and other key open science practices (clinical codelist sharing, data sharing, study preregistration, and stated use of reporting guidelines and preprinting). We developed six recommendations for investigators who choose to share code and gathered feedback from members of the International Society for Pharmacoepidemiology (ISPE). RESULTS: Programming code sharing by articles published in PDS ranged from 1.8% in 2017 to 9.5% in 2022. It was more prevalent among articles with a methodological focus, simulation studies, and papers which also shared record-level data. CONCLUSION: Programming code sharing is rare but increasing in pharmacoepidemiology studies published in PDS. We recommend improved reporting of whether code is shared and how available code can be accessed. When sharing programming code, we recommend the use of permanent digital identifiers, appropriate licenses, and, where possible, adherence to good software practices around the provision of metadata and documentation, computational reproducibility, and data privacy.

The Oncology QCARD Initiative: Fostering efficient evaluation of initial real-world data proposals.

PURPOSE: The oncology quality, characterization, and assessment of real-world data (Oncology QCARD) Initiative was formed to develop a set of minimum study design and data elements needed to evaluate the fitness of the real-world data (RWD) source(s) proposed in an initial study concept as part of early interaction with scientific reviewers. METHODS: A multidisciplinary executive committee (EC) was established to guide the Oncology QCARD Initiative. The EC conducted a landscape review of published literature, guidances, and guidelines to evaluate relevant dimensions of data quality measurement. Guided by the review and informed by expert feedback, the Oncology QCARD Initial Protocol Characterization (IPC) provides a summary of minimum elements needed to adequately describe an initial clinical study concept that involves RWD and is intended to support decision-making. RESULTS: Fit-for-use data and fit-for-purpose design emerged as themes from the landscape analysis. Data that are fit-for-use are both relevant (sufficiently capturing exposure, outcomes, and covariates) and reliable (understanding data accrual and quality control and whether the data represent the underlying concepts they are intended to represent) to answer a specific research question. A fit-for-purpose design takes appropriate steps to ensure internal and external validity and allows for transparency in reporting. The QCARD-IPC focuses on high-level characteristics of RWD sources and study design domains including data temporality, population, medical product exposure, comparators, and covariates, endpoints, statistical analysis, and data quality assurance plans. CONCLUSIONS: Evaluation of studies including RWD requires understanding the data source, study design, and potential biases to preliminarily evaluate whether selected RWD are fit-for-use for the research question. The Oncology QCARD-IPC provides a structured, transparent approach to facilitate early review and enhanced communication between study sponsors and scientific reviewers of initial study proposals including RWD.

Accessibility and Utility of an Electronic Self-Guided Safety Plan for Adolescents.

OBJECTIVE: Suicidal thoughts and behaviors are a leading cause of death, injury, and hospitalization among adolescents. Few evidence-based interventions exist, and these tend to be inaccessible for most youth. Electronic safety plan interventions represent a new avenue to disseminate an evidence-based suicide prevention tool. However, it is not clear whether youth find electronic, self-guided safety plans helpful, nor whether they will use the resulting safety card when experiencing suicidal thoughts or urges. This study examines adolescents' perceptions and use of an electronic, self-guided safety plan intervention. METHOD: We recruited 322 adolescents with a past-year history of suicidal thoughts or behaviors, ages 13-17 (55.9% white and non-Hispanic; majority reporting diverse sexual orientations and genders) online via social media to participate in this two-part longitudinal study. At baseline, participants completed the electronic safety plan interventions and reported on key aspects of its usefulness and areas of improvement. Participants reported their use and perceptions of the plan one month later. RESULTS: Results suggested that adolescents understood, liked, and believed they would use a self-guided safety plan. One month later, about ⅓ of youth who experienced suicidal thoughts or urges used their safety card. Open-ended responses highlighted several features that adolescents enjoyed, including clarity, ease of use, and privacy. Participants also highlighted key areas for improvement, including formatting and language. CONCLUSION: This study provides initial support for adolescent use of electronic self-guided safety plans. Next steps include updating this intervention based on youth feedback and testing the effectiveness of this tool using gold standard research methods.

Comparative Effectiveness and Safety of Generic Versus Brand-Name Fluticasone-Salmeterol to Treat Chronic Obstructive Pulmonary Disease.

BACKGROUND: In 2019, the U.S. Food and Drug Administration (FDA) approved the first generic maintenance inhaler for asthma and chronic obstructive pulmonary disease (COPD). The inhaler, Wixela Inhub (fluticasone-salmeterol; Viatris), is a substitutable version of the dry powder inhaler Advair Diskus (fluticasone-salmeterol; GlaxoSmithKline). When approving complex generic products like inhalers, the FDA applies a special "weight-of-evidence" approach. In this case, manufacturers were required to perform a randomized controlled trial in patients with asthma but not COPD, although the product received approval for both indications. OBJECTIVE: To compare the effectiveness and safety of generic (Wixela Inhub) and brand-name (Advair Diskus) fluticasone-salmeterol among patients with COPD treated in routine care. DESIGN: A 1:1 propensity score-matched cohort study. SETTING: A large, longitudinal health care database. PATIENTS: Adults older than 40 years with a diagnosis of COPD. MEASUREMENTS: Incidence of first moderate or severe COPD exacerbation (effectiveness outcome) and incidence of first pneumonia hospitalization (safety outcome) in the 365 days after cohort entry. RESULTS: Among 45 369 patients (27 305 Advair Diskus users and 18 064 Wixela Inhub users), 10 012 matched pairs were identified for the primary analysis. Compared with Advair Diskus use, Wixela Inhub use was associated with a nearly identical incidence of first moderate or severe COPD exacerbation (hazard ratio [HR], 0.97 [95% CI, 0.90 to 1.04]) and first pneumonia hospitalization (HR, 0.99 [CI, 0.86 to 1.15]). LIMITATIONS: Follow-up times were short, reflecting real-world clinical practice. The possibility of residual confounding cannot be completely excluded. CONCLUSION: Use of generic and brand-name fluticasone-salmeterol was associated with similar outcomes among patients with COPD treated in routine practice. PRIMARY FUNDING SOURCE: National Heart, Lung, and Blood Institute.

Separation of Plasmid DNA Topological Forms, Messenger RNA, and Lipid Nanoparticle Aggregates Using an Ultrawide Pore Size Exclusion Chromatography Column.

Health authorities have highlighted the need to determine oligonucleotide aggregates. However, existing technologies have limitations that have prevented the reliable analysis of size variants for large nucleic acids and lipid nanoparticles (LNPs). In this work, nucleic acid and LNP aggregation was examined using prototype, low adsorption ultrawide pore size exclusion chromatography (SEC) columns. A preliminary study was conducted to determine the column's physicochemical properties. A large difference in aggregate content (17.8 vs 59.7 %) was found for a model messenger RNA (mRNA) produced by different manufacturers. We further investigated the nature of the aggregates via a heat treatment. Interestingly, thermal stress irreversibly decreased the amount of aggregates from 59.7 to 4.1% and increased the main peak area 3.3-fold. To the best of our knowledge, for the first time, plasmid DNA topological forms and multimers were separated by analytical SEC. The degradation trends were compared to the data obtained with an anion exchange chromatography method. Finally, unconjugated and fragment antigen-binding (Fab)-guided LNPs were analyzed and their elution times were plotted against their sizes as measured by DLS. Multi-angle light scattering (MALS) was coupled to SEC in order to gain further insights on large species eluting before the LNPs, which were later identified as self-associating LNPs. This study demonstrated the utility of ultrawide pore SEC columns in characterizing the size variants of large nucleic acid therapeutics and LNPs.

Process Robustness in Lipid Nanoparticle Production: A Comparison of Microfluidic and Turbulent Jet Mixing.

The recent clinical and commercial success of lipid nanoparticles (LNPs) for nucleic acid delivery has incentivized the development of new technologies to manufacture LNPs. As new technologies emerge, researchers must determine which technologies to assess and how to perform comparative evaluations. In this article, we use a quality-by-design approach to systematically investigate how the mixer technology used to form LNPs influences LNPstructure. Specifically, a coaxial turbulent jet mixer and a staggered herringbone microfluidic mixer were systematically compared via matched formulation and process conditions. A full-factorial design-of-experiments study with three factors and three levels was executed for each mixer to compare process robustness in the production of antisense oligonucleotide (ASO) LNPs. ASO-LNPs generated with the coaxial turbulent jet mixer were consistently smaller, had a narrower particle size distribution, and had a higher ASO encapsulation as compared to the microfluidic mixer, but had a greater variation in internal structure with less ordered cores. A subset of the study was replicated for mRNA-LNPs with comparable trends in particle size and encapsulation, but more frequent bleb features for LNPs produced by the coaxial turbulent jet mixer. The study design used here provides a road map for how researchers may compare different mixer technologies (or process changes more broadly) and how such studies can inform process robustness and manufacturing control strategies.

Klotho Deficiency Causes Heart Aging via Impairing the Nrf2-GR Pathway.

RATIONALE: Cardiac aging is an important contributing factor for heart failure, which affects a large population but remains poorly understood. OBJECTIVE: The purpose of this study is to investigate whether Klotho plays a role in cardiac aging. METHODS AND RESULTS: Heart function declined in old mice (24 months), as evidenced by decreases in fractional shortening, ejection fraction, and cardiac output. Heart size and weight, cardiomyocyte size, and cardiac fibrosis were increased in old mice, indicating that aging causes cardiac hypertrophy and remodeling. Circulating Klotho levels were dramatically decreased in old mice, which prompted us to investigate whether the Klotho decline may cause heart aging. We found that Klotho gene mutation (KL-/-) largely decreased serum klotho levels and impaired heart function. Interestingly, supplement of exogenous secreted Klotho prevented heart failure, hypertrophy, and remodeling in both old mice and KL (-/-) mice. Secreted Klotho treatment inhibited excessive cardiac oxidative stress, senescence and apoptosis in old mice and KL (-/-) mice. Serum phosphate levels in KL (-/-) mice were kept in the normal range, suggesting that Klotho deficiency-induced heart aging is independent of phosphate metabolism. Mechanistically, Klotho deficiency suppressed GR (glutathione reductase) expression and activity in the heart via inhibition of transcription factor Nrf2 (nuclear factor-erythroid 2 p45-related factor 2). Furthermore, cardiac-specific overexpression of GR prevented excessive oxidative stress, apoptosis, and heart failure in both old and KL (-/-) mice. CONCLUSIONS: Klotho deficiency causes cardiac aging via impairing the Nrf2-GR pathway. Supplement of exogenous secreted Klotho represents a promising therapeutic strategy for aging-associated cardiomyopathy and heart failure.

Prospective validation of a dynamic prognostic model for identifying COVID-19 patients at high risk of rapid deterioration.

BACKGROUND: We sought to develop and prospectively validate a dynamic model that incorporates changes in biomarkers to predict rapid clinical deterioration in patients hospitalized for COVID-19. METHODS: We established a retrospective cohort of hospitalized patients aged ≥18 years with laboratory-confirmed COVID-19 using electronic health records (EHR) from a large integrated care delivery network in Massachusetts including >40 facilities from March to November 2020. A total of 71 factors, including time-varying vital signs and laboratory findings during hospitalization were screened. We used elastic net regression and tree-based scan statistics for variable selection to predict rapid deterioration, defined as progression by two levels of a published severity scale in the next 24 h. The development cohort included the first 70% of patients identified chronologically in calendar time; the latter 30% served as the validation cohort. A cut-off point was estimated to alert clinicians of high risk of imminent clinical deterioration. RESULTS: Overall, 3706 patients (2587 in the development and 1119 in the validation cohort) met the eligibility criteria with a median of 6 days of follow-up. Twenty-four variables were selected in the final model, including 16 dynamic changes of laboratory results or vital signs. Area under the ROC curve was 0.81 (95% CI, 0.79-0.82) in the development set and 0.74 (95% CI, 0.71-0.78) in the validation set. The model was well calibrated (slope = 0.84 and intercept = -0.07 on the calibration plot in the validation set). The estimated cut-off point, with a positive predictive value of 83%, was 0.78. CONCLUSIONS: Our prospectively validated dynamic prognostic model demonstrated temporal generalizability in a rapidly evolving pandemic and can be used to inform day-to-day treatment and resource allocation decisions based on dynamic changes in biophysiological factors.

HARmonized Protocol Template to Enhance Reproducibility of hypothesis evaluating real-world evidence studies on treatment effects: A good practices report of a joint ISPE/ISPOR task force.

PROBLEM: Ambiguity in communication of key study parameters limits the utility of real-world evidence (RWE) studies in healthcare decision-making. Clear communication about data provenance, design, analysis, and implementation is needed. This would facilitate reproducibility, replication in independent data, and assessment of potential sources of bias. WHAT WE DID: The International Society for Pharmacoepidemiology (ISPE) and ISPOR-The Professional Society for Health Economics and Outcomes Research (ISPOR) convened a joint task force, including representation from key international stakeholders, to create a harmonized protocol template for RWE studies that evaluate a treatment effect and are intended to inform decision-making. The template builds on existing efforts to improve transparency and incorporates recent insights regarding the level of detail needed to enable RWE study reproducibility. The overarching principle was to reach for sufficient clarity regarding data, design, analysis, and implementation to achieve 3 main goals. One, to help investigators thoroughly consider, then document their choices and rationale for key study parameters that define the causal question (e.g., target estimand), two, to facilitate decision-making by enabling reviewers to readily assess potential for biases related to these choices, and three, to facilitate reproducibility. STRATEGIES TO DISSEMINATE AND FACILITATE USE: Recognizing that the impact of this harmonized template relies on uptake, we have outlined a plan to introduce and pilot the template with key international stakeholders over the next 2 years. CONCLUSION: The HARmonized Protocol Template to Enhance Reproducibility (HARPER) helps to create a shared understanding of intended scientific decisions through a common text, tabular and visual structure. The template provides a set of core recommendations for clear and reproducible RWE study protocols and is intended to be used as a backbone throughout the research process from developing a valid study protocol, to registration, through implementation and reporting on those implementation decisions.