The Reporting Recommendations Intended for Pharmaceutical Risk Minimization Evaluation Studies: Standards for Reporting of Implementation Studies Extension (RIMES-SE).

INTRODUCTION: The Reporting recommendations Intended for pharmaceutical risk Minimization Evaluation Studies (RIMES) was developed to improve the quality of reporting of risk minimization program evaluations. In light of continued inadequacies in study reporting, and high-profile program implementation failures, we updated the RIMES Checklist to incorporate additional concepts from the Standards for Reporting of Implementation studies (StaRI). METHODS: The development of the updated checklist, the RIMES-StaRI Extension (RIMES-SE), entailed developing a study protocol and drafting an initial pool of items based on a mapping of the RIMES against the StaRI checklist. A modified e-Delphi exercise was then conducted to determine the importance and understandability of items for checklist inclusion. An expert workshop and an online commentary period for additional feedback followed. RESULTS: The RIMES-SE contains 27 items. It includes two signature features of the StaRI Checklist: 1) a dual strand of items (represented in two columns) describing the risk minimization program (the 'intervention') and the corresponding implementation strategy; and 2) applicable to an array of different research methodologies. CONCLUSIONS: The RIMES-SE Statement and Checklist extends the reporting guidelines set forth in the original RIMES Checklist via inclusion of key implementation science concepts. It is intended to improve the quality and transparency of reporting of risk minimization evaluation studies so as to advance drug safety science.

The use of implementation science theories, models, and frameworks in implementation research for medicinal products: A scoping review.

BACKGROUND: The uptake, adoption and integration of new medicines and treatment regimens within healthcare delivery can take a decade or more. Increasingly, implementation science (IS) research is being used to bridge this gap between the availability of new therapeutic evidence and its actual application in clinical practice. Little is known, however, about the quality of IS research in this area, including the degree to which theories, models and frameworks (TMFs) are being used. The objective of this study was to conduct a scoping review of the use of TMFs in implementation research involving medicinal products. METHODS: A search was conducted for English language abstracts and manuscripts describing the application of TMFs in IS studies for medicinal products. Eligible publications were those published between 1 January 1974 and 12 December 2022. All records were screened at the title and abstract stage; included full-text papers were abstracted using data extraction tables designed for the study. Study quality was appraised using the Implementation Research Development Tool. RESULTS: The initial scoping search identified 2697 publications, of which 9 were ultimately eligible for inclusion in the review. Most studies were published after 2020 and varied in their objectives, design and therapeutic area. Most studies had sample sizes of fewer than 50 participants, and all focused on the post-marketing phase of drug development. The TMF most frequently used was the Consolidated Framework for Implementation Research (CFIR). Although most studies applied all TMF domains, TMF use was limited to instrument development and/or qualitative analysis. Quality appraisals indicated the need for engaging patients and other stakeholders in the implementation research, reporting on the cost of implementation strategies, and evaluating the unintended consequences of implementation efforts. CONCLUSIONS: We found that few IS studies involving medicinal products reported using TMFs. Those that did encompassed a wide variety of therapeutic indications and medicinal products; all were in the post-marketing phase and involved limited application of the TMFs. Researchers should consider conducting IS in earlier phases of drug development and integrating the TMFs throughout the research process. More consistent and in-depth use of TMFs may help advance research in this area.

Balancing benefits and risks in lung cancer therapies: patient preferences for lung cancer treatment alternatives.

Background: In the treatment of Non-Small Cell Lung Cancer (NSCLC) the combination of Immuno- Oncotherapy (IO) and chemotherapy (CT) has been found to be superior to IO or CT alone for patients' survival. Patients and clinicians are confronted with a preference sensitive choice between a more aggressive treatment with a greater negative effect on quality of life versus alternatives that are less effective but have fewer side effects. Objectives: The aims of this study were to: (a) quantify patients' preferences for relevant attributes related to Immuno-Oncotherapy treatment alternatives, and (b) evaluate the maximum acceptable risk (MAR)/Minimum acceptable benefit (MAB) that patients would accept for treatment alternatives. Methods: An online preference survey using discrete-choice experiment (DCE) was completed by NSCLC patients from two hospitals in Italy and Belgium. The survey asked patients' preferences for five patient- relevant treatment attributes. The DCE was developed using a Bayesian D-efficient design. DCE analyses were performed using mixed logit models. Information regarding patient demographics, health literacy, locus of control, and quality of life was also collected. Results: 307 patients (158 Italian, 149 Belgian), stage I to IV, completed the survey. Patients preferred treatments with a higher 5-year survival chance as the most important attribute over all the other attributes. Preference heterogeneity for the attribute weights depended on health literacy, patients' age and locus of control. Patients were willing to accept a substantially increased risks of developing side effects in exchange for the slightest increase (1%) in the chance of surviving at least 5 years from the diagnosis of cancer. Similarly, patients were willing to accept a switch in the mode of administration or complete loss of hair to obtain an increase in survival. Conclusion: In this study, the proportion of respondents who systematically preferred survival over all other treatment attributes was particularly high. Age, objective health literacy and locus of control accounted for heterogeneity in patients' preferences. Evidence on how NSCLC patients trade between survival and other NSCLC attributes can support regulators and other stakeholders on assessing clinical trial evidence and protocols, based on patients' conditions and socio-demographic parameters.

A Road Map for Peer Review of Real-World Evidence Studies on Safety and Effectiveness of Treatments.

The growing acceptance of real-world evidence (RWE) in clinical and regulatory decision-making, coupled with increasing availability of health care data and advances in automated analytic approaches, has contributed to a marked expansion of RWE studies of diabetes and other diseases. However, a recent spate of high-profile retractions highlights the need for improvements in the conduct of RWE research as well as in the associated peer review and editorial processes. We review best pharmacoepidemiologic practices and common pitfalls regarding design, measurement, analysis, data validity, appropriateness, and generalizability of RWE studies. To enhance RWE study assessments, we propose that journal editors require 1) study authors to complete RECORD-PE, a reporting guideline for pharmacoepidemiological studies on routinely collected data, 2) availability of predetermined study protocols and analysis plans, 3) inclusion of pharmacoepidemiologists on the peer review team, and 4) provision of detail on data provenance, characterization, and custodianship to facilitate assessment of the data source. We recognize that none of these steps guarantees a high-quality research study. Collectively, however, they permit an informed assessment of whether the study was adequately designed and conducted and whether the data source used was fit for purpose.

Patients as research partners in preference studies: learnings from IMI-PREFER.

BACKGROUND: There is growing recognition of the importance of patient and public stakeholder involvement (PPI) in patient preference research. However, limited evidence exists regarding the impact, barriers and enablers of PPI in preference studies. The Innovative Medicines Initiative (IMI)-PREFER project conducted a series of preference case studies which incorporated PPI. OBJECTIVE: To describe: (1) how PPI was operationalized in the PREFER case studies, (2) the impact of PPI, and (3) factors that served to impede and facilitate PPI. METHODS: We reviewed the PREFER final study reports to determine how patient partners were involved. We conducted a thematic framework analysis to characterize the impact of PPI and then administered a questionnaire to the PREFER study leads to identify barriers and facilitators to effective PPI. RESULTS: Eight PREFER case studies involved patients as research partners. Patient partners were involved in activities spanning all phases of the patient preference research process, including in study design, conduct and dissemination. However, the type and degree of patient partner involvement varied considerably. Positive impacts of PPI included improvements in the: (1) quality of the research and research process; (2) patient partner empowerment; (3) study transparency and dissemination of results; (4) research ethics, and (5) trust and respect between the research team and the patient community. Of the 13 barriers identified, the 3 most frequently reported were inadequate resources, insufficient time to fully involve patient partners, and uncertainty regarding how to operationalize the role of 'patient partner. Among the 12 facilitators identified, the two most frequently cited were (1) having a clearly stated purpose for involving patients as research partners; and (2) having multiple patient partners involved in the study. CONCLUSION: PPI had many positive impacts on the PREFER studies. Preference study leads with prior PPI experience reported a greater number of positive impacts than those with no such experience. In light of the numerous barriers identified, multi-faceted implementation strategies should be considered to support adoption, integration and sustainment of PPI within preference research. Additional case studies of patient partner involvement in preference research are needed as well to inform best practices in this area.

Research about patients' preferences for medicinal products and treatments is growing. Such research could be improved if patients were involved as 'research partners,' that is, as active members of the study team itself. To date, however, little is known about the actual experience of involving patients as partners in such research. This paper presents learnings from involving patients as partners in 8 case studies conducted as part of IMI-PREFER, a big, European-based project which aimed to develop recommendations about how to conduct preference research. Involving patients as partners led to improvements in the: (1) quality of the research and research process; (2) recruitment of participants; (3) content and design of patient-facing informational materials; and, (4) how and what study results were shared with patient communities. Our findings showed that it is important to plan for patient partners' involvement early on in the design of the preference study so as to ensure that they are fully integrated into the research team and their opportunity to contribute to all stages of the research is optimized. Such planning should address how patient partners will be paid, what their role responsibilities will include, how and when they will be trained and educated, and how they will be supported throughout the course of the study. Having a clearly stated purpose for involving patients as research partners, selecting patient partners who have had prior research experience and relationships with the researchers, and having multiple patient partners on the study team are all also helpful in supporting successful patient involvement. We need more people to share their experiences with involving patient partners in preference research so that we can continue to improve how this is done.

The case for using mixed methods for designing, implementing, and disseminating evidence-based interventions for public health practice.

A well-recognized gap exists between findings from public health research and their use in public health practice to improve outcomes. The World Health Organization (WHO) has identified implementation research as vital to improving the adaptation and dissemination of public health interventions into real-world settings. Implementation research encompasses multimethod evaluation approaches; 'mixed methods', the planned integration of qualitative and quantitative methods, is a key tool. We argue that mixed methods designs are crucial for design and evaluation of public health interventions, provide illustrative case studies, discuss key analytic approaches in mixed methods design, identify resources for mixed methods research and advocate for more training. On behalf of the International Network for Epidemiology in Policy (INEP), an international non-profit organization of 24 member societies that promotes the development of equitable, evidence-based health policies, we call for increased collaboration between qualitative and quantitative research teams to improve the design and evaluation of public health interventions.

Improving the Safety of Medicines via Digital Technology: An Assessment of the Scope and Quality of Risk Minimization Websites in the United States and United Kingdom.

INTRODUCTION: EHealth holds tremendous promise for enhancing drug safety initiatives known as risk minimization programs. Little is known, however, regarding the scope and quality of existing risk minimization websites. METHODS: Two publicly accessible repositories, REMS@FDA [1] and Electronic Medicines Compendium [2], were reviewed to identify all regulatorily approved risk minimization programs in the United States (US) and United Kingdom (UK) with websites. Website quality was evaluated using the Enlight Quality Assessment tool, a psychometrically validated instrument that addresses seven quality domains. RESULTS: Ninety-three websites were identified: 59 for healthcare professionals (7 UK/52 US), and 34 for patients (5 UK/29 US). The websites functioned chiefly as archives for electronic copies of educational materials; a subset (31/93) had additional features. Mean quality ratings for Usability (mean 4.70, SD 0.59), Visual Design (mean 4.03, SD 0.87) and Content (mean 4.31, SD 0.82) were good. General Subjective Evaluation was fair (mean 3.15, SD 1.21). Mean scores for Therapeutic Alliance and Therapeutic Persuasiveness were poor (mean 2.62, SD 1.47; and mean 2.50, SD 1.48, respectively); those for User Engagement were very poor (mean 2.25, SD 1.03). No differences were found by target audience but several were identified based on region. CONCLUSIONS: Risk minimization websites are easy to navigate and well organized. Few, however, incorporate eHealth design elements that facilitate user engagement, build therapeutic alliance and exert therapeutic persuasiveness. Such elements can enhance program uptake and effectiveness. Results highlight opportunities for improving the quality of risk minimization websites and their ability to bridge pharmaceutical and healthcare systems.

A risk minimization program is a type of drug safety measure to ensure that a medicine's benefits outweigh its risks. Electronic versions of these risk minimization programs (websites) offer new ways to reach and educate patients and healthcare professionals. We conducted a study to examine the quality of these websites. We reviewed all approved risk minimization programs in the United States (US) and the United Kingdom (UK) using two publicly available repositories, REMS@FDA and Electronic Medicines Compendium, to identify risk minimization websites. We assessed website quality using the Enlight Quality Assessment tool. We found 93 websites: 59 for healthcare professionals (7 UK/52 US) and 34 for patients (5 UK/29 US). Our analysis showed that the websites were well organized and easy to search. Few, however, used specific electronic design elements that can promote trust in and engagement with the content of the website, and can encourage users to follow the recommended actions for safe and appropriate use of the medicine. In conclusion, there are multiple ways that the design of risk minimization websites could be improved in order to make them more effective as drug safety measures.

Development of a Digital Health Intervention for Rheumatoid Arthritis Symptom Management in a Biotechnology Industry Context: Protocol for the Application of a Human-Centered Design Framework.

BACKGROUND: Involving chronically ill patients in the management of their health is widely recognized as a vital component of high-quality health care. However, to assume the role of informed participants, patients need both access to their health information and assistance in interpreting such data. Smartphone technology with SMS text messaging functionality offers a convenient and minimally demanding mechanism for providing such dual capabilities to patients. To date, a number of similar digital tools have been developed for use in various chronic and progressive disease conditions, including rheumatoid arthritis. OBJECTIVE: This paper aims to describe the development of a research protocol that applies a human-centered design (HCD) approach to develop a mobile health (mHealth) intervention to support symptom management and treatment adherence for rheumatoid arthritis. METHODS: To guide the development of the mHealth intervention for use within a commercial biotechnology context, we selected and applied an HCD framework consisting of three phases: understanding, ideation, and implementation. RESULTS: Leveraging the framework, we mapped the key objectives and research questions to each phase and identified the HCD techniques and methods most suitable for addressing them. In addition, we identified the need to include a fourth phase, one that referred to postimplementation assessment, which would enable evaluation of patient engagement and intervention impact on symptom self-management. CONCLUSIONS: This paper presents a research protocol that applied an HCD framework to guide the development of an mHealth intervention within a commercial biotechnology context. This type of guidance is salient because commercial entities are becoming one of the leading producers of this type of intervention. However, the methodologies used and challenges faced from a research and development perspective are not well-represented in the published research literature to date. Our application of the HCD framework yielded important findings. Each phase of the HCD framework provided important guidance for increasing the likelihood that the final product would be understandable, acceptable, feasible, and engaging to use. Consistent with other researchers in the field of mHealth interventions, we identified the need to add a fourth phase to the HCD framework, one that focused on a postimplementation assessment to guide further improvements to support adoption in real-world settings. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR1-10.2196/16430.

Opportunities and Challenges of Web-Based and Remotely Administered Surveys for Patient Preference Studies in a Vulnerable Population.

The application of web-based and remotely administered surveys is becoming increasingly popular due to the fact that it offers numerous advantages over traditional paper-based or computer-based surveys completed in the presence of the researcher. However, it is unclear whether complex preference elicitation tasks administered online in highly vulnerable patient populations are also feasible. This commentary discusses opportunities and challenges of conducting quantitative patient preference studies in lung cancer patients using web-based modes of data collection. We refer to our recent experience in the context of the Patient Preference in Benefit-Risk Assessments during the Drug Life Cycle (PREFER) project. Among the main advantages were the possibility of reaching a wider and geographically distant population in a shorter timeframe while reducing the financial costs of testing, the greater flexibility offered and the reduced burden on the patients. Some limitations were also identified and should be the object of further research, including the potential lack of inclusiveness of the research, the lack of control over who is completing the survey, a poor comprehension of the study material, and ultimately a lower level of engagement with the study. Despite these limitations, experience from the PREFER project suggests that online quantitative methods for data collection may provide a valuable method to explore preferences in vulnerable patient populations beyond the COVID-19 pandemic.

Key Determinants of Health-Related Quality of Life Among Advanced Lung Cancer Patients: A Qualitative Study in Belgium and Italy.

Background: The lung cancer (LC) treatment landscape has drastically expanded with the arrival of immunotherapy and targeted therapy. This new variety of treatment options, each with its own characteristics, raises uncertainty regarding the key aspects affecting patients' health-related quality of life (HRQL). The present qualitative study aimed to investigate how LC patients perceive their HRQL and the factors that they consider to be most influential in determining their HRQL. Methods: This qualitative research incorporates four focus group discussions, with six LC patients in each group. In total, 24 stage III and IV LC patients were included in the discussions, with Italian (n = 12) and Belgian (n = 12) patients, age range: 42-78, median age = 62 (IQR = 9.3 years), SD = 8.5; 62% men. Using thematic analysis, transcripts and notes from the FGDs were analyzed using NVivo software (edition 12). Results: Three main themes capturing determinants of HRQL were identified. First, patients agreed on the importance of physical aspects (symptoms and side-effects) in determining their HRQL. In particular, skin conditions, nausea, fatigue, risk of infections, sensory abnormalities, pain, and changes in physical appearance were highlighted. Second, patients worried about psychological aspects, negatively impacting their wellbeing such as uncertainties regarding their future health state, and a lower degree of autonomy and independence. Third, patients underlined the importance of social aspects, such as communication with healthcare providers and social interaction with friends, family and peers. Conclusion: This study demonstrates that physical, psychological, and social aspects are key factors driving LC patients' HRQL. Gaining a better understanding of how LC patients perceive their HRQL and how it is affected by their illness and therapy will aid patient-centric decision-making across the drug life cycle, by providing stakeholders (drug developers, regulators, reimbursement bodies, and clinicians) insights about the treatment and disease aspects of importance to LC patients as well as the unmet needs LC patients may have regarding available treatment modalities. Finally, this study underscores a need for individual treatment decision-making that is considerate of uncertainties among LC patients about their future health state, and ways for improving communication between healthcare providers and patients to do so.

Patient Preferences for Lung Cancer Treatments: A Study Protocol for a Preference Survey Using Discrete Choice Experiment and Swing Weighting.

Background: Advanced treatment options for non-small cell lung cancer (NSCLC) consist of immunotherapy, chemotherapy, or a combination of both. Decisions surrounding NSCLC can be considered as preference-sensitive because multiple treatments exist that vary in terms of mode of administration, treatment schedules, and benefit-risk profiles. As part of the IMI PREFER project, we developed a protocol for an online preference survey for NSCLC patients exploring differences in preferences according to patient characteristics (preference heterogeneity). Moreover, this study will evaluate and compare the use of two different preference elicitation methods, the discrete choice experiment (DCE) and the swing weighting (SW) task. Finally, the study explores how demographic (i.e., age, gender, and educational level) and clinical (i.e., cancer stage and line of treatment) information, health literacy, health locus of control, and quality of life may influence or explain patient preferences and the usefulness of a digital interactive tool in providing information on preference elicitation tasks according to patients. Methods: An online survey will be implemented with the aim to recruit 510 NSCLC patients in Belgium and Italy. Participants will be randomized 50:50 to first receive either the DCE or the SW. The survey will also collect information on participants' disease-related status, health locus of control, health literacy, quality of life, and perception of the educational tool. Discussion: This protocol outlines methodological and practical steps to quantitatively elicit and study patient preferences for NSCLC treatment alternatives. Results from this study will increase the understanding of which treatment aspects are most valued by NSCLC patients to inform decision-making in drug development, regulatory approval, and reimbursement. Methodologically, the comparison between the DCE and the SW task will be valuable to gain information on how these preference methods perform against each other in eliciting patient preferences. Overall, this protocol may assist researchers, drug developers, and decision-makers in designing quantitative patient preferences into decision-making along the medical product life cycle.

What Matters Most to Lung Cancer Patients? A Qualitative Study in Italy and Belgium to Investigate Patient Preferences.

Background: The potential value of patient preference studies has been recognized in clinical individual treatment decision-making between clinicians and patients, as well as in upstream drug decision-making. Drug developers, regulators, reimbursement and Health Technology Assessment (HTA) bodies are exploring how the use of patient preference studies could inform drug development, regulatory benefit risk-assessment and reimbursement decisions respectively. Understanding patient preferences may be especially valuable in decisions regarding Non-Small Cell Lung Cancer (NSCLC) treatment options, where a variety of treatment options with different characteristics raise uncertainty about which features are most important to NSCLC patients. As part of the Innovative Medicines Initiative PREFER project, this qualitative study aimed to identify patient-relevant lung cancer treatment characteristics. Methods: This study consisted of a scoping literature review and four focus group discussions, 2 in Italy and 2 in Belgium, with a total of 24 NSCLC patients (Stages III-IV). The focus group discussions sought to identify which treatment characteristics patients find most relevant. The discussions were analyzed thematically using a thematic inductive analysis. Results: Patients highlighted themes reflecting: 1) positive effects or expected gains from treatment such as greater life expectancy and maintenance of daily functioning, 2) negative effects or adverse events related to therapy that negatively impact patients' daily functioning such as fatigue and 3) uncertainty regarding the duration and type of treatment effects. These overarching themes were consistent among patients from Belgium and Italy, suggesting that treatment aspects related to efficacy and safety as well as the psychological impact of lung cancer treatment are common areas of concern for patients, regardless of cultural background or country. Discussion: Our findings illustrate the value of using qualitative methods with patients to identify preferred treatment characteristics for advanced lung cancer. These could inform a subsequent quantitative preference survey that assesses patient trade-offs regarding treatment options.

Patient Preferences for Lung Cancer Treatment: A Qualitative Study Protocol Among Advanced Lung Cancer Patients.

Introduction: Lung cancer is the deadliest and most prevalent cancer worldwide. Lung cancer treatments have different characteristics and are associated with a range of benefits and side effects for patients. Such differences may raise uncertainty among drug developers, regulators, payers, and clinicians regarding the value of these treatment effects to patients. The value of conducting patient preference studies (using qualitative and/or quantitative methods) for benefits and side effects of different treatment options has been recognized by healthcare stakeholders, such as drug developers, regulators, health technology assessment bodies, and clinicians. However, evidence-based guidelines on how and when to conduct and use these studies in drug decision-making are lacking. As part of the Innovative Medicines Initiative PREFER project, we developed a protocol for a qualitative study that aims to understand which treatment characteristics are most important to lung cancer patients and to develop attributes and levels for inclusion in a subsequent quantitative preference survey. Methods: The study protocol specifies a four-phased approach: (i) a scoping literature review of published literature, (ii) four focus group discussions with stage III and IV Non-Small Cell Lung Cancer patients, (iii) two nominal group discussions with stage III and IV Non-Small Cell Lung Cancer patients, and (iv) multi-stakeholder discussions involving clinicians and preference experts. Discussion: This protocol outlines methodological and practical steps as to how qualitative research can be applied to identify and develop attributes and levels for inclusion in patient preference studies aiming to inform decisions across the drug life cycle. The results of this study are intended to inform a subsequent quantitative preference survey that assesses patient trade-offs regarding lung cancer treatment options. This protocol may assist researchers, drug developers, and decision-makers in designing qualitative studies to understand which treatment aspects are most valued by patients in drug development, regulation, and reimbursement.

Risk Management for the 21st Century: Current Status and Future Needs.

Global adoption of risk management principles outlined in the International Conference on Harmonisation (ICH) E2E guideline and the Council for International Organizations of Medical Sciences (CIOMS) Working Group VI guidance introduced greater proactivity and consistency into the practice of pharmacovigilance and benefit-risk management throughout the lifecycle of a drug. However, following the release of these guidelines there have been important advances in the science and practice of risk minimisation itself, especially in terms of how risk minimisation measures (RMMs) are designed, implemented, disseminated and evaluated for effectiveness in real-world healthcare settings. In this article, we describe how the field of design, implementation, dissemination and evaluation of RMMs has advanced in recent years while highlighting current areas of challenge and possible solutions. Where possible we cite global examples to demonstrate how evidence-based approaches have informed the development of RMMs. In this context, while taking into consideration local healthcare system policies and national legislations, we conclude with a call for a global effort to harmonise certain areas that focus on, but are not limited to, standardising certain terms and definitions, consistent application of robust methodologies, and outline of best practices for risk minimisation design, implementation, and dissemination.

Quantitative Benefit-Risk Assessment: State of the Practice Within Industry.

BACKGROUND: Benefit-risk assessments for medicinal products and devices have advanced significantly over the past decade. The purpose of this study was to characterize the extent to which the life sciences industry is utilizing quantitative benefit-risk assessment (qBRA) methods. METHODS: Semi-structured interviews were conducted with a sample of industry professionals working in drug and/or medical device benefit-risk assessments (n = 20). Questions focused on the use, timing, and impact of qBRA; implementation challenges; and future plans. Interviews were recorded, transcribed, and coded for thematic analysis. RESULTS: While most surveyed companies had applied qBRA, application was limited to a small number of assets-primarily to support internal decision-making and regulatory submissions. Positive impacts associated with use included improved team decision-making and communication. Multi-criteria decision analysis and discrete choice experiment were the most frequently utilized qBRA methods. A key challenge of qBRA use was the lack of clarity regarding its value proposition. Championing by senior company leadership and receptivity of regulators to such analyses were cited as important catalysts for successful adoption of qBRA. Investment in qBRA methods, via capability building and pilot studies, was also under way in some instances. CONCLUSION: qBRA application within this sample of life sciences companies was widespread, but concentrated in a small fraction of assets. Its use was primarily for internal decision-making or regulatory submissions. While some companies had plans to build further capacity in this area, others were waiting for further regulatory guidance before doing so.

The utility of real-world evidence for benefit-risk assessment, communication, and evaluation of pharmaceuticals: Case studies.

PURPOSE: In recent years, novel types of real-world evidence (RWE) have played a role in various decision-making processes relating to medicinal products, including regulatory approval, patient access, health technology assessment, safety monitoring, clinical use, and post-approval lifecycle management. We therefore reviewed the potential utility of RWE in the cycle of medicinal product benefit-risk (BR) assessment, communication/risk minimization and evaluation ("BRACE"). METHODS: A convenience sample of illustrative studies was drawn from the published literature and examined. Specifically, we examined the purpose for using RWE, the type of RWE used, its novelty and how it might be integrated with other data and activities of the BRACE cycle, and how it contributed to regulatory decision-making. RESULTS: Eight studies were selected with each illustrating a different activity in the BRACE cycle ranging from BR assessment in the preapproval setting, post-approval assessment of safety or effectiveness, communicating BR information to patients and healthcare professionals, and evaluating the effectiveness of risk minimization initiatives to support a positive BR balance. CONCLUSIONS: RWE has an important role in informing regulatory decision-making regarding the BR management of medicines. With increasing digitalization, facilitating data collection and stakeholder engagement in health, this role is only expected to expand in the future. To reach the full potential of RWE, both regulators and sponsors will need to be familiar with a range of existing and emerging methods for generating and analyzing such evidence appropriately and achieve convergence regarding how different types of RWE can best be used to inform BR management and decision-making.

Quality of Reporting on the Evaluation of Risk Minimization Programs: A Systematic Review.

INTRODUCTION: Risk minimization programs are interventions mandated by regulatory agencies to ensure that benefits of pharmaceutical products outweigh risks. Many regulatory agencies require programs be evaluated for effectiveness; however, the quality of evidence has limited the ability to definitively determine if programs improve drug safety. OBJECTIVE: The aim of this systematic review was to assess and describe the current status of reporting on the effectiveness of pharmaceutical risk management programs. METHODS: Peer-reviewed articles published between January 2012 and December 2018 were selected from three online databases (MEDLINE, PubMed, Embase). Eligible studies reported on effectiveness evaluations of mandated risk minimization measures (beyond labeling) and were written in English. Two reviewers independently examined 2744 titles of articles and 52 full articles were included. Forty-eight sources of gray literature from conference abstract presentations and publicly available regulatory documents were also included. RESULTS: Key opportunities for improvement in reporting included the provision of information regarding (1) selection, design, and testing of risk minimization measures, (2) implementation of programs, (3) process and outcome metrics, including the extent to which programs reached the intended audience, were integrated into the target healthcare settings, or were sustained over time, and (4) burden of the program on the healthcare system and implications for patient access. CONCLUSIONS: Gaps in reporting of risk minimization program evaluation studies were identified. Addressing gaps will help build the evidence base regarding risk minimization initiatives, as well as ensure that programs are maximally effective and minimally burdensome on the healthcare system, and do not unduly interfere with patient access to the medicine.

What is the quality of drug safety information for patients: An analysis of REMS educational materials.

BACKGROUND: Poor-quality patient drug information has been identified as a major cause of preventable medication errors in the United States. The US Food and Drug Administration (FDA) has the authority to require marketing authorization holders of medicinal products to implement risk evaluation and mitigation strategies (REMS) to ensure that the benefits of a drug or biological product outweigh its risks. Aside from medication guides, no research has been conducted to assess the quality of patient-targeted REMS materials, including whether, and to what extent, patients find these materials understandable and actionable. PURPOSE: To describe the readability, understandability, and actionability of patient educational materials in currently approved REMS programs, and to highlight opportunities for improving both the quality and effectiveness of these important drug safety tools. METHODS: Seventy-seven REMS programs were identified from the FDA REMS database. We excluded medication guides (MGs) from our analysis because of the fact that there is a mandatory MG template. Based on this, we identified a total of 27 (non-MG) REMS patient materials on the FDA REMS website for analysis purposes. The materials were tested for readability using the Lexile Measure, the Gunning Fog Index, and Flesch Kincaid and then assessed using the Patient Education Materials Assessment Tool for printable materials, for understandability and actionability. RESULTS: Twenty-three of 77 (30%) REMS programs used educational materials to communicate serious risks to patients, yielding a total of 27 REMS patient materials for analysis. The median readability score for these materials was at a ninth-grade reading level or higher. While most (89%) of these patient education materials met established criteria for being understandable, less than half (49%) were deemed actionable. DISCUSSION: Currently approved REMS patient materials fell short in terms of recommended reading level, and over half did not meet recommended standards for actionability. Developers of these materials should apply plain language principles when design these materials to improve their readability and to assess both understandability and actionability in order to increase the effectiveness when distributed to patients.

The RIMES Statement: A Checklist to Assess the Quality of Studies Evaluating Risk Minimization Programs for Medicinal Products.

INTRODUCTION: Pharmaceutical risk minimization programs involve interventions designed to support safe and appropriate use of medicines. Currently, information regarding the evaluation of these programs is not publicly reported in a standardized and transparent manner. To address this gap, we developed and piloted a quality reporting checklist entitled the Reporting recommendations Intended for pharmaceutical risk Minimization Evaluation Studies (RIMES). METHODS: Checklist development was guided by three sources: (1) a theoretical framework derived from program theory and process evaluation; (2) public health intervention design and evaluation principles; and (3) a review of existing quality reporting checklists. Two raters independently reviewed 10 recently published (2012-2016) risk minimization program evaluation studies using the proposed checklist. Inter-rater reliability of the checklist was assessed using Cohen's Kappa and Gwet's AC1. RESULTS: A 43-item checklist was generated. Results indicated substantial inter-rater reliability overall (κ = 0.65, AC1 = 0.65) and for three (key information, design and evaluation) of the four subscales (κ ≥ 0.64, AC1 ≥ 0.64). The fourth subscale (implementation) showed low reliability based on Cohen's Kappa, but substantial reliability based on the AC1 (κ = 0.17, AC1 = 0.61). CONCLUSIONS: The RIMES statement augments relevant elements from existing quality reporting guidelines with items that address aspects of intervention design, implementation and evaluation specific to pharmaceutical risk minimization programs. Our results show that the RIMES statement reliably measures key dimensions of reporting quality. This tailored checklist is an important first step in improving the reporting quality of risk minimization evaluation studies and may ultimately help to improve the quality of these interventions themselves.