How Do Risk Evaluation and Mitigation Strategies Impact Clinical Practice? A National Survey of Physicians.

The US Food and Drug Administration can require risk evaluation and mitigation strategy (REMS) programs for prescription drugs to ensure the benefits of use outweigh the risks. We conducted a national survey of physicians' experiences prescribing eight REMS-covered drugs: (1) ambrisentan; (2) bosentan; (3) clozapine; (4) isotretinoin; (5-7) the multiple myeloma (MM) drugs lenalidomide, pomalidomide, thalidomide; and (8) sodium oxybate. Between May 2022 and January 2023, we surveyed 5,331 physician prescribers of these drugs, and 1,295 (24%) returned surveys (range: 149 for bosentan to 226 for MM drugs). Although 765 (68%) respondents thought the certification process provided useful drug information, 757 (67%) wanted materials to include benefit data and 944 (84%) non-REMS-related risk data. A majority (704, 63%) thought the safe use requirements facilitated discussion with patients, but a similar number (637, 57%) attributed delayed medication access to these requirements. In multivariable modeling, MM drug and isotretinoin respondents were less likely than sodium oxybate respondents to agree that the certification process provided useful drug information (MM drug: odds ratio (OR) = 0.37, 95% confidence interval (CI) = 0.25-0.55; isotretinoin: OR = 0.39, 95% CI = 0.25-0.61), and isotretinoin, clozapine, and bosetan respondents were more likely than sodium oxybate respondents to agree that the safe use requirements often delayed medication access (isotretinoin: OR = 5.83, 95% CI = 3.70-9.19; clozapine: OR = 1.65, 95% CI = 1.08-2.54; bosentan: OR = 1.78, 95% CI = 1.12-2.85). Most physicians believe REMS programs convey useful drug safety information and facilitate discussion with patients but also seek information on benefits and non-REMS-related risks and better integration of REMS processes into clinical workflows.

Evolution of Cross-Sectional Survey Protocol Quality Over Time: A Case Series of Index U.S. REMS Knowledge Survey Protocols (2007-2020).

INTRODUCTION: Surveys are commonly used to assess effectiveness of FDA-required risk evaluation and mitigation strategies (REMS) for drugs and biologics in the United States. OBJECTIVE: The aim of this study was to assess the scientific rigor of REMS knowledge survey protocols submitted to FDA and compare protocols before and after FDA's 2012 public workshop and 2019 draft guidance. METHOD: A content analysis of index survey protocols submitted to FDA (2007-2020) for single-product REMS with elements to assure safe use (39 programs, 78 protocols) was conducted. Each protocol was scored against 52 core essential elements (CEE), abstracted from FDA's guidance and grouped into six domains: study objective (n = 5), study design (n = 18), survey instrument (n = 9), participant recruitment (n = 7), survey administration (n = 9), and statistical analysis plan (n = 4). Scores were collected by time periods: (A) Oct 2007 to Jul 2012; (B) Aug 2012 to Feb 2019; (C) Mar 2019 to Dec 2020; and compared using logistic generalized linear mixed models adjusting for domain, survey population, vendor, program, and protocol. RESULTS: There were 30 (38.5%), 40 (51.3%), and 8 (10.3%) protocols submitted in time period A, B, and C, respectively. Adjusted marginal means of elements present (on the probability scale) by time period were 0.5816 (SE = 0.0242), 0.6429 (SE = 0.0229), and 0.7543 (SE = 0.0394). The likelihood of missing a CEE declined over time (adjusted p-value = 0.0094, time period A vs C). The statistical analysis plan domain had the most improvement; study design remained the weakest domain with the scientific justification CEE particularly underrepresented. CONCLUSION: The rigor of REMS knowledge survey protocols improved over time consistent with FDA's efforts to advance regulatory science, but gaps remain.

The Risk Evaluation and Mitigation Strategy (REMS) Public Dashboard: Improving Transparency of Regulatory Activities.

This brief paper aims to describe the Risk Evaluation and Mitigation Strategy (REMS) Public Dashboard launched by the US Food and Drug Administration (FDA) in December 2021. The FDA REMS Public Dashboard can be accessed through the REMS@FDA website. The dashboard was developed in Qlik Sense® to support a user-friendly interactive web-based tool that allows healthcare providers, patients, researchers, pharmaceutical companies, and regulators to readily access and visualize REMS information. The dashboard includes eight separate pages to capture information on all REMS, active REMS, REMS with elements to assure safe use, shared system REMS, REMS modifications, REMS revisions, released REMS, and REMS Summary; for REMS programs approved from 2008 to the present. Most of the pages allow users to choose different REMS characteristics to visualize and stratify the data by variables such as REMS approval time, application type, or REMS elements. This interactive platform is intended to allow users to quickly visualize trends over time and locate details of the REMS programs to inform emerging research and regulatory issues in the context of current drug safety. The FDA continues to explore ways to enhance public access of the REMS information in near real-time through the REMS Public Dashboard.

Pragmatic applications of implementation science frameworks to regulatory science: an assessment of FDA Risk Evaluation and Mitigation Strategies (REMS) (2014-2018).

BACKGROUND: A Risk Evaluation and Mitigation Strategy (REMS) is a drug safety program for certain medications with serious safety concerns required by the U.S. Food and Drug Administration (FDA) of manufacturers to implement to help ensure the benefits of the medication outweigh its risks. FDA is encouraging "the research community to develop novel methods for assessing REMS," conveying the unmet need for a standardized evaluation method of these regulatory-mandated healthcare programs. The objective of this research is to evaluate FDA REMS assessment plans using established implementation science frameworks and identify opportunities for strengthening REMS evaluation. METHODS: A content analysis was conducted of publicly available assessment plans for all REMS programs (N = 23) approved 1/1/2014-12/31/2018 for new drug applications (NDAs) and biologics license applications (BLAs) requiring FDA-mandated Elements to Assure Safe Use (ETASU). Blinded reviewers critically appraised REMS assessment measures (n = 674) using three established implementation science frameworks: RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance); PRECEDE-PROCEED (Predisposing, Reinforcing, and Enabling Constructs in Educational/Environmental Diagnosis and Evaluation - Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development); and CFIR (Consolidated Framework for Implementation Research). Framework constructs were mapped to REMS Assessment categories as defined by FDA Guidance for Industry to evaluate congruence. RESULTS: REMS assessment measures demonstrated strong congruence (> 90% mapping rate) with the evaluative constructs of RE-AIM, PRECEDE-PROCEED, and CFIR. Application of the frameworks revealed that REMS assessment measures heavily emphasize implementation and operations, focus less on health outcomes, and do not evaluate program context and design assumptions. CONCLUSIONS: Implementation science frameworks have utility for evaluating FDA-mandated drug safety programs including the selection of primary measures to determine whether REMS goals are being met and of secondary measures to evaluate contextual factors affecting REMS effectiveness in varying organizational settings.

A Multi-modal Approach to Evaluate the Impact of Risk Evaluation and Mitigation Strategy (REMS) Programs.

INTRODUCTION: Since 2007, the US Food and Drug Administration has had the authority to require risk evaluation and mitigation strategy (REMS) programs for certain medications with serious safety concerns to help ensure the benefits of the medication outweigh its risks. Such programs can include requirements for patient monitoring, restrictions on dispensing or administration, and physician and pharmacy training and certification. However, there has been only scattered evidence on the impact of REMS programs on informed decision making, medication access, or patient outcomes. OBJECTIVE: The objective of this article was to describe a study that researchers at Brigham and Women's Hospital and Harvard Medical School will conduct in partnership with the Food and Drug Administration's Office of Surveillance and Epidemiology to investigate systematically how REMS programs have operated in practice. METHODS: Investigations include health insurance claims-based analyses to understand patterns of drug use, adherence to safety requirements, and patient outcomes under REMS programs; surveys and interviews to understand physician and patient experiences with REMS; and REMS program material-based and interview-based analyses to understand the effectiveness of risk communication in REMS programs. CONCLUSIONS: These research activities will evaluate the performance of REMS programs, provide information on the benefits and burdens to patients and healthcare providers, and generate recommendations for actionable steps to improve REMS programs overall.

Adaptation for Regulatory Application: A Content Analysis of FDA Risk Evaluation and Mitigation Strategies Assessment Plans (2014-2018) Using RE-AIM.

Background: Risk Evaluation and Mitigation Strategies (REMS) are safety programs that U.S. Food and Drug Administration can require to ensure a drug's benefits outweigh its risks and can be considered public health interventions. FDA's 2019 draft Guidance for Industry on REMS Assessments encourages the development of "novel methods for assessing REMS [to] help advance the science of post-market assessment of effectiveness of risk mitigation strategies." Objective: To characterize REMS assessment plans using RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework and identify areas for advancing methods for evaluating REMS programs. RE-AIM was selected for its wide application evaluating the translation of scientific advances into practice for public health impact. Methods: A content analysis of REMS assessment plans (N = 18) and measures(n = 540) was conducted for REMS programs approved by FDA between 1/1/2014-12/31/2018. Eligibility criteria were: a new drug application or biologic license application, included FDA-mandated mitigation strategies called elements to assure safe use (ETASU), and represented a single product REMS program. Assessment plans were collected from publicly available regulatory approval letters from REMS@FDA website. Blinded reviewers categorized each REMS assessment measure to a RE-AIM dimension, adjudicated their application (average IRR 75%), and refined the adapted dimensions' definitions. Dimensions were also mapped to REMS Assessment guidance categories. Results: The median number of assessment measures per REMS assessment plan was 31 (IQR: 21-36). Frequency of measures per RE-AIM criteria per REMS program was: Reach (median = 2; IQR: 2-4); Effectiveness (median = 2.5; IQR:1-4); Adoption (median = 3.5; IQR: 2-5); Implementation (median = 18; IQR: 15-24); Maintenance (median = 0; IQR: 0-1). Adoption (among prescriber, health system agents of implementation) was more commonly assessed than Reach (population-attributable number of patients affected). Assessment of heterogeneity of Adoption and Reach was generally absent. Implementation assessment measures were most common among drugs requiring evidence of safe-use conditions before dispensing or administering the drug. Patient-level Effectiveness and Maintenance assessments were most common among drugs requiring patient monitoring. Discussion: Implementation science frameworks, such as RE-AIM, can be applied to characterize REMS assessment measures and identify opportunities for standardizing and strengthening their evaluation. Methods to measure Maintenance are needed to provide real-world evidence of REMS integration into the healthcare system.