Improving Transparency to Build Trust in Real-World Secondary Data Studies for Hypothesis Testing-Why, What, and How: Recommendations and a Road Map from the Real-World Evidence Transparency Initiative.

Real-world data (RWD) and the derivations of these data into real-world evidence (RWE) are rapidly expanding from informing healthcare decisions at the patient and health system level to influencing major health policy decisions, including regulatory approvals and coverage. Recent examples include the approval of palbociclib in combination with endocrine therapy for male breast cancer and the inclusion of RWE in the label of paliperidone palmitate for schizophrenia. This interest has created an urgency to develop processes that promote trust in the evidence-generation process. Key stakeholders and decision-makers include patients and their healthcare providers; learning health systems; health technology assessment bodies and payers; pharmacoepidemiologists and other clinical reseachers, and policy makers interested in bioethical and regulatory issues. A key to optimal uptake of RWE is transparency of the research process to enable decision-makers to evaluate the quality of the methods used and the applicability of the evidence that results from the RWE studies. Registration of RWE studies-particularly for hypothesis evaluating treatment effectiveness (HETE) studies-has been proposed to improve transparency, trust, and research replicability. Although registration would not guarantee better RWE studies would be conducted, it would encourage the prospective disclosure of study plans, timing, and rationale for modifications. A joint task force of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the International Society for Pharmacoepidemiology (ISPE) recommended that investigators preregister their RWE studies and post their study protocols in a publicly available forum before starting studies to reduce publication bias and improve the transparency of research methods. Recognizing that published recommendations alone are insufficient, especially without accessible registration options and with no incentives, a group of experts gathered on February 25 and 26, 2019, in National Harbor, Maryland, to explore the structural and practical challenges to the successful implementation of the recommendations of the ISPOR/ISPE task force for preregistration. This positioning article describes a plan for making registration of HETE RWE studies routine. The plan includes specifying the rationale for registering HETE RWE studies, the studies that should be registered, where and when these studies should be registered, how and when analytic deviations from protocols should be reported, how and when to publish results, and incentives to encourage registration. Table 1 summarizes the rationale, goals, and potential solutions that increase transparency, in addition to unique concerns about secondary data studies. Definitions of terms used throughout this report are provided in Table 2.

Improving transparency to build trust in real-world secondary data studies for hypothesis testing-Why, what, and how: recommendations and a road map from the real-world evidence transparency initiative.

Real-world data (RWD) and the derivations of these data into real-world evidence (RWE) are rapidly expanding from informing healthcare decisions at the patient and health system level to influencing major health policy decisions, including regulatory approvals and coverage. Recent examples include the approval of palbociclib in combination with endocrine therapy for male breast cancer and the inclusion of RWE in the label of paliperidone palmitate for schizophrenia. This interest has created an urgency to develop processes that promote trust in the evidence-generation process. Key stakeholders and decision-makers include patients and their healthcare providers; learning health systems; health technology assessment bodies and payers; pharmacoepidemiologists and other clinical reseachers, and policy makers interested in bioethical and regulatory issues. A key to optimal uptake of RWE is transparency of the research process to enable decision-makers to evaluate the quality of the methods used and the applicability of the evidence that results from the RWE studies. Registration of RWE studies-particularly for hypothesis evaluating treatment effectiveness (HETE) studies-has been proposed to improve transparency, trust, and research replicability. Although registration would not guarantee better RWE studies would be conducted, it would encourage the prospective disclosure of study plans, timing, and rationale for modifications. A joint task force of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the International Society for Pharmacoepidemiology (ISPE) recommended that investigators preregister their RWE studies and post their study protocols in a publicly available forum before starting studies to reduce publication bias and improve the transparency of research methods. Recognizing that published recommendations alone are insufficient, especially without accessible registration options and with no incentives, a group of experts gathered on February 25 and 26, 2019, in National Harbor, Maryland, to explore the structural and practical challenges to the successful implementation of the recommendations of the ISPOR/ISPE task force for preregistration. This positioning article describes a plan for making registration of HETE RWE studies routine. The plan includes specifying the rationale for registering HETE RWE studies, the studies that should be registered, where and when these studies should be registered, how and when analytic deviations from protocols should be reported, how and when to publish results, and incentives to encourage registration. Table 1 summarizes the rationale, goals, and potential solutions that increase transparency, in addition to unique concerns about secondary data studies. Definitions of terms used throughout this report are provided in Table 2.

Real-World Evidence: Useful in the Real World of US Payer Decision Making? How? When? And What Studies?

OBJECTIVES: To examine how real-world evidence (RWE) is currently perceived and used in managed care environments, especially to inform pharmacy and therapeutic (P&T) committee decisions, to assess which study factors (e.g., data, design, and funding source) contribute to RWE utility in decisions, and to identify barriers to consideration of RWE studies in P&T decision making. METHODS: We conducted focus groups/telephone-based interviews and surveys to understand perceptions of RWE and assess awareness, quality, and relevance of two high-profile examples of published RWE studies. A purposive sample comprised 4 physicians, 15 pharmacists, and 1 researcher representing 18 US health plans and health system organizations. RESULTS: Participants reported that RWE was generally used, or useful, to inform safety monitoring, utilization management, and cost analysis, but less so to guide P&T decisions. Participants were not aware of the two sample RWE studies but considered both studies to be valuable. Relevant research questions and outcomes, transparent methods, study quality, and timely results contribute to the utility of published RWE. Perceived organizational barriers to the use of published RWE included lack of skill, training, and timely study results. CONCLUSIONS: Payers recognize the value of RWE, but use of such studies to inform P&T decisions varies from organization to organization and is limited. Relevance to payers, timeliness, and transparent methods were key concerns with RWE. Participants recognized the need for continuing education on evaluating and using RWE to better understand the study methods, findings, and applicability to their organizations.

PEER-REVIEWED JOURNAL EDITORS' VIEWS ON REAL-WORLD EVIDENCE.

OBJECTIVES: Peer-review publication is a critical step to the translation and dissemination of research results into clinical practice guidelines, health technology assessment (HTA) and payment policies, and clinical care. The objective of this study was to examine current views of journal editors regarding: (i) The value of real-world evidence (RWE) and how it compares with other types of studies; (ii) Education and/or resources journal editors provide to their peer reviewers or perceive as needed for authors, reviewers, and editors related to RWE. METHODS: Journal editors' views on the value of RWE and editorial procedures for RWE manuscripts were obtained through telephone interviews, a survey, and in-person, roundtable discussion. RESULTS: In total, seventy-nine journals were approached, resulting in fifteen telephone interviews, seventeen survey responses and eight roundtable participants. RWE was considered valuable by all interviewed editors (n = 15). Characteristics of high-quality RWE manuscripts included: novelty/relevance, rigorous methodology, and alignment of data to research question. Editors experience challenges finding peer reviewers; however, these challenges persist across all study designs. Journals generally do not provide guidance, assistance, or training for reviewers, including for RWE studies. Health policy/health services research (HSR) editors were more likely than specialty or general medicine editors to participate in this study, potentially indicating that HSR researchers are more comfortable/interested in RWE. CONCLUSIONS: Editors report favorable views of RWE studies provided studies examine important questions and are methodologically rigorous. Improving peer-review processes across all study designs, has the potential to improve the evidence base for decision making, including HTA.

Transparency in evidence evaluation and formulary decision-making: from conceptual development to real-world implementation.

OBJECTIVE: Establishing a better understanding of the relationship between evidence evaluation and formulary decision-making has important implications for patients, payers, and providers. The goal of our study was to develop and test a structured approach to evidence evaluation to increase clarity, consistency, and transparency in formulary decision-making. STUDY DESIGN: The study comprised three phases. First, an expert panel identified key constructs to formulary decision-making and created an evidence-assessment tool. Second, with the use of a balanced incomplete block design, the tool was validated by a large group of decision-makers. Third, the tool was pilot-tested in a real-world P&T committee environment. METHODS: An expert panel identified key factors associated with formulary access by rating the level of access that they would give a drug in various hypothetical scenarios. These findings were used to formulate an evidence-assessment tool that was externally validated by surveying a larger sample of decision-makers. Last, the tool was pilot-tested in a real-world environment where P&T committees used it to review new drugs. RESULTS: Survey responses indicated that a structured approach in the formulary decision-making process could yield greater clarity, consistency, and transparency in decision-making; however, pilot-testing of the structured tool in a real-world P&T committee environment highlighted some of the limitations of our structured approach. CONCLUSION: Although a structured approach to formulary decision-making is beneficial for patients, health care providers, and other stakeholders, this benefit was not realized in a real-world environment. A method to improve clarity, consistency, and transparency is still needed.

Cost-sharing and adherence, clinical outcomes, health care utilization, and costs: A systematic literature review.

BACKGROUND: US health plans are adopting benefit designs that shift greater financial burden to patients through higher deductibles, additional copay tiers, and coinsurance. Prior systematic reviews found that higher cost was associated with reductions in both appropriate and inappropriate medications. However, these reviews were conducted prior to contemporary benefit design and medication utilization. OBJECTIVE: To assess the relationship and factors associated with cost-sharing and (1) medication adherence, (2) clinical outcomes, (3) health care resource utilization (HRU), and (4) costs. METHODS: A systematic review of literature published between January 2010 and August 2020 was conducted to identify the relationship between cost-sharing and medication adherence, clinical outcomes, HRU, and health care costs. Data were extracted using a standardized template and were synthesized by key questions of interest. RESULTS: From 1,995 records screened, 79 articles were included. Most studies, 71 of 79 (90%), reported the relationship between cost-sharing and treatment adherence, persistence and/or discontinuation; 16 (20%) reported data on cost-sharing and HRU or medication initiation, 11 (14%) on costsharing and health care costs, and 6 (8%) on cost-sharing and clinical outcomes. The majority of publications found that, regardless of disease area, increased cost-sharing was associated with worse adherence, persistence, or discontinuation. The aggregate data suggested the greater the magnitude of cost-sharing, the worse the adherence. Among studies examining clinical outcomes, cost-sharing was associated with worse outcomes in 1 study and the remaining 3 found no significant differences. Regarding HRU, higher-cost-sharing trended toward decreased outpatient and increased inpatient utilization. The available evidence suggested higher cost-sharing has an overall neutral to negative impact on total costs. Studies evaluating elimination of copays found either decreased or no impact in total costs. CONCLUSIONS: The published literature shows consistent impacts of higher cost sharing on initiation and continuation of medications, and the greater the cost-sharing, the worse the medication adherence. The evidence is limited regarding the impact of cost-sharing on clinical outcomes, HRU, and costs. Limited evidence suggests increased cost-sharing is associated with more inpatient care and less outpatient care; however, a neutral to no difference was suggested for other outcomes. Although increased costsharing is intended to decrease total costs, studies evaluating reducing or eliminating cost-sharing found that total costs did not rise. Today's growing cost-containment environment should carefully consider the broader impact cost-sharing has on treatment adherence, clinical outcomes, resource use, and total costs. It may be that cost-sharing is a blunt, rather than precise, tool to curb health care costs, affecting both necessary and unnecessary health care use. DISCLOSURES: This study and the development of this article were funded by the National Pharmaceutical Council. Mr Sils is an employee of the National Pharmaceutical Council. Dr Graff is a former employee of the National Pharmaceutical Council. Drs Fusco and Kistler and Ms Ruiz are employees of Xcenda. Xcenda received funding to conduct the literature review.

Payer perceptions and use of value assessment tools in the United States.

BACKGROUND: As the United States transitions toward value-based payment, value assessment tools to measure the value of health care interventions are emerging. As the field evolves, it is important to evaluate how these tools are influencing treatment and coverage decisions. OBJECTIVE: To examine payer perceptions and use of US value assessment tools and identify how these tools inform payer decision-making. METHODS: A double-blind, web-based survey was conducted from June to July 2022 to assess health care payers' perceptions and use of value assessment tools developed by the American Society of Clinical Oncology, Drug Pricing Lab, Institute for Clinical and Economic Review (ICER), Innovation and Value Initiative, and National Comprehensive Cancer Network. RESULTS: 51 respondents completed the survey. 86% of payers were familiar with at least 4 of 5 value assessment tools. Both ICER and National Comprehensive Cancer Network tools are perceived as very useful for informing formulary decisions (57% and 49%, respectively). When selecting a value assessment tool, payers identified the inclusion of appropriate metrics and outcomes (92%), comparative clinical effectiveness information (88%), and reliance on rigorous, unbiased methods (86%) to be very/extremely important. Payers reported the inclusion of the patient, provider, and societal perspectives as lower importance (32%, 31%, and 20% identify these elements as very/extremely important, respectively). Payers reported using ICER evidence reports to both expand and restrict coverage decisions. To advance more useful and relevant value assessment tools, payers identified the need for greater stakeholder awareness of existing tools, and some recommended that value assessors increase the volume of assessments conducted. CONCLUSIONS: US health care payers perceive select value assessment tools to be useful for informing health care decisions. As policy momentum behind value assessment builds, additional examination of value assessment tools is needed to inform appropriate application of value assessment in US health care decision-making. DISCLOSURES: This study was funded by Xcenda/AmerisourceBergen. Ms Buelt, Ms Loo, Ms Westrich, and Drs Hydery and Zheng report employment with Xcenda/AmerisourceBergen. Drs Dharbhamalla and Graff report employment with AMCP.

Prevalence of Avoidable and Bias-Inflicting Methodological Pitfalls in Real-World Studies of Medication Safety and Effectiveness.

Many real-word evidence (RWE) studies that utilize existing healthcare data to evaluate treatment effects incur substantial but avoidable bias from methodologically flawed study design; however, the extent of preventable methodological pitfalls in current RWE is unknown. To characterize the prevalence of avoidable methodological pitfalls with potential for bias in published claims-based studies of medication safety or effectiveness, we conducted an English-language search of PubMed for articles published from January 1, 2010 to May 20, 2019 and randomly selected 75 studies (10 case-control and 65 cohort studies) that evaluated safety or effectiveness of cardiovascular, diabetes, or osteoporosis medications using US health insurance claims. General and methodological study characteristics were extracted independently by two reviewers, and potential for bias was assessed across nine bias domains. Nearly all studies (95%) had at least one avoidable methodological issue known to incur bias, and 81% had potentially at least one of the four issues considered major due to their potential to undermine study validity: time-related bias (57%), potential for depletion of outcome-susceptible individuals (44%), inappropriate adjustment for postbaseline variables (41%), or potential for reverse causation (39%). The median number of major issues per study was 2 (interquartile range (IQR), 1-3) and was lower in cohort studies with a new-user, active-comparator design (median 1, IQR 0-1) than in cohort studies of prevalent users with a nonuser comparator (median 3, IQR 3-4). Recognizing and avoiding known methodological study design pitfalls could substantially improve the utility of RWE and confidence in its validity.

Stakeholders find that step therapy should be evidence-based, flexible, and transparent: assessing appropriateness using a consensus approach.

BACKGROUND: Step therapy, one approach to utilization management, is used by health plans to ensure safe and clinically appropriate care while managing cost. Several patient and provider groups have each developed principles to guide the appropriate use of step therapy; however, no comprehensive multistakeholder informed set of criteria exist. OBJECTIVE: To assess multistakeholder consensus on criteria for the development and implementation of step therapy for pharmaceutical therapies. Stakeholders were asked to (a) assess the appropriateness of step therapy as a utilization management tool; (b) rate specific criteria across 5 domains (development, implementation, communication, appeals, and evaluation) of step therapy; and (c) categorize these criteria as standards or best practices. METHODS: We conducted a multiphase project culminating in a roundtable of experts representing patient, provider, plan, pharmacy, policy, and ethical perspectives. We first reviewed guiding principles, position statements, and legislative activity to draft criteria regarding step therapy protocol development, implementation, communication, and evaluation. To assess consensus across a convenience sample of experts, we employed an iterative 4-step modified Delphi method. Panelists were asked to (a) rate the overall appropriateness of step therapy, (b) rate the appropriateness of specific criteria, and (c) identify each as a standard or best practice. Appropriateness was rated from 1-9 and categorized in terciles (1-3: not appropriate, 4-6: neither, 7-9: appropriate) to assess quantitative agreement, disagreement, and indeterminate agreement. RESULTS: After the second round of voting, roundtable panelists (n = 16) disagreed on the appropriateness of step therapy for utilization management (50% appropriate, 31.25% neither, and 18.75% inappropriate). Agreement was achieved on 21 criteria across 5 themes (clinical criteria as the foundation for protocol development, implementation of protocols, transparency and communication of processes, navigation of the appeals process, and evaluation of health and administrative impact). Fourteen and seven criteria were categorized as standards and best practices, respectively. CONCLUSIONS: The stakeholders in this panel differed in their assessments of the appropriateness of step therapy but agreed regarding how these protocols should be developed, implemented, communicated, and evaluated. Most criteria were rated as standards that can be used by stakeholders when developing, implementing, and assessing step therapy processes today. DISCLOSURES: This study was funded by the National Pharmaceutical Council. Karmarkar was a fellow at the National Pharmaceutical Council and Duke-Margolis Center for Health Policy at the time this study was conducted. Dubois and Graff are employees of the National Pharmaceutical Council. This work was previously presented as a virtual poster during the AMCP 2020 eLearning Days, April 21-24, 2020.

Limited role of patient input in specialty drug coverage policies.

BACKGROUND: Despite increased financial contributions towards care, consumers' role in shaping their insurance benefits is unclear. OBJECTIVE: To examine the role played by patient input when US commercial health plans formulate specialty drug coverage policies, along with the benefits and challenges of considering this input. METHODS: We employed a parallel, mixed-methods approach. First, we reviewed health plans' policy development processes as reported on their websites. Second, we reviewed a data set of private health plan coverage decisions for specialty drugs and examined whether the evidence cited in policies included patient-reported outcomes (eg, health-related quality of life endpoints) and patient-based methodological designs (eg, interviews or surveys of patients). Third, we performed a survey (N = 21 respondents) and interviews (N = 5 interviewees) with plan decision-makers to determine the current role of patient input in plan decision-making, and the benefits and challenges of incorporating this data when formulating specialty drug coverage policies. RESULTS: We found that plans do not commonly solicit patient input when developing coverage policies, with only two instances of limited interaction between plans and patients or members. 1,316 (9%) of the studies plans cited in their specialty drug coverage policies included at least one patient-reported endpoint, and 0.4% (N = 62) used a patient-based methodological design. Of studies with patient-based designs, 40 used interviews, 26 included surveys/questionnaires, and one concerned shared decision-making (design categories not mutually exclusive). Almost half of the survey respondents reported having never engaged with patients or members when developing coverage policies. Among respondents who had engaged with patients or members, most reported doing so only rarely. The survey and interviews highlighted various benefits of soliciting patient input, including the value of obtaining a humanistic perspective, and several challenges, including resource requirements and the quality of obtained information. CONCLUSIONS: We found a notable lack of patient and member engagement by commercial health plans when formulating drug coverage policies. Survey respondents and interviewees identified benefits of accounting for patients' and plan members' values and preferences in specialty drug coverage policies, but also reported a number of important challenges to doing so. DISCLOSURES: National Pharmaceutical Council provided funding for this research.

Online tools to synthesize real-world evidence of comparative effectiveness research to enhance formulary decision making.

Results of randomized controlled trials (RCTs) provide valuable comparisons of 2 or more interventions to inform health care decision making; however, many more comparisons are required than available time and resources to conduct them. Moreover, RCTs have limited generalizability. Comparative effectiveness research (CER) using real-world evidence (RWE) can increase generalizability and is important for decision making, but use of nonrandomized designs makes their evaluation challenging. Several tools are available to assist. In this study, we comparatively characterize 5 tools used to evaluate RWE studies in the context of making health care adoption decision making: (1) Good Research for Comparative Effectiveness (GRACE) Checklist, (2) IMI GetReal RWE Navigator (Navigator), (3) Center for Medical Technology Policy (CMTP) RWE Decoder, (4) CER Collaborative tool, and (5) Real World Evidence Assessments and Needs Guidance (REAdi) tool. We describe each and then compare their features along 8 domains: (1) objective/user/context, (2) development/scope, (3) platform/presentation, (4) user design, (5) study-level internal/external validity of evidence, (6) summarizing body of evidence, (7) assisting in decision making, and (8) sharing results/making improvements. Our summary suggests that the GRACE Checklist aids stakeholders in evaluation of the quality and applicability of individual CER studies. Navigator is a collection of educational resources to guide demonstration of effectiveness, a guidance tool to support development of medicines, and a directory of authoritative resources for RWE. The CMTP RWE Decoder aids in the assessment of relevance and rigor of RWE. The CER Collaborative tool aids in the assessment of credibility and relevance. The REAdi tool aids in refinement of the research question, study retrieval, quality assessment, grading the body of evidence, and prompts with questions to facilitate coverage decisions. All tools specify a framework, were designed with stakeholder input, assess internal validity, are available online, and are easy to use. They vary in their complexity and comprehensiveness. The RWE Decoder, CER Collaborative tool, and REAdi tool synthesize evidence and were specifically designed to aid formulary decision making. This study adds clarity on what the tools provide so that the user can determine which best fits a given purpose. DISCLOSURES: This work was supported by the Health Tech Fund, which was provided to the University of Washington School of Pharmacy by its Corporate Advisory Board. This consortium of pharmaceutical and biotech companies supports the research program of the University of Washington School of Pharmacy across the competitive space. The sponsors seeded the idea for the project and contributed to study design and improvement. The authors had full control of all content development, manuscript drafting, and submission for publication. The REAdi tool was developed by the authors. Chen, Bansal, Barthold, Carlson, Veenstra, Basu, Devine, Yun, Ta, and Beal were supported by a training grant from the University of Washington-Allergan Fellowship, unrelated to this work. Basu reports personal fees from Salutis Consulting, unrelated to this work. Graff is an employee of the National Pharmaceutical Council, which was a partner in the development of the CER Collaborative and funding partner for the CMTP RWE Decoder and the GRACE Checklist. A previous version of this work was presented as an invited workshop at AMCP Nexus 2018; October 22-25, 2018; Orlando, FL.

Underestimating the Value of an Intervention: The Case for Including Productivity in Value Assessments and Formulary Design.

BACKGROUND: Value assessment reports are increasingly being considered in health care coverage decisions. The inputs included and analytic methodologies underlying these reports should include all components of value. OBJECTIVE: To determine whether and how productivity was included in a value assessment, compare the incremental cost per quality-adjusted life-year (cost/QALY) estimates with and without productivity, assess if inclusion of productivity changed the value category and estimate the direction and magnitude of change. METHODS: We reviewed pharmaceutical value assessment reports published between March 2017 and July 2019 by the Institute for Clinical and Economic Review (ICER) to determine whether productivity was included and how it was reported (i.e., co-base case or scenario analysis). Within each report, we identified unique treatment comparisons for which modelers estimated an incremental cost/QALY. We categorized the incremental cost/QALY estimates using ICER's willingness-to-pay (WTP) categories and assessed if inclusion of productivity changed the value category (i.e., < $50,000/QALY). For reports that included 2 numerical estimates, we assessed the direction and magnitude of change when productivity was included. RESULTS: Of the 19 reports that evaluated pharmaceutical treatments, 18 (94.7%) included productivity. Two reports (11.1%) incorporated productivity in a co-base case analysis, and 16 included productivity in a scenario analysis. Across these 18 reports, there were 75 unique comparisons of pharmaceutical interventions. Across the 75 comparisons, 4 (5.3%), 3 (4.0%), 8 (10.6%), and 1 (1.3%) of the coverage decisions would change at the $50,000/QALY, $100,000/QALY, $150,000/QALY, and $500,000/QALY threshold, respectively. Sixty comparisons included 2 numerical cost/QALY estimates. The magnitude of change in the cost/QALY, after including productivity, ranged from -80.1% to 6.8%. The estimated value increased for 54 (72%), decreased for 5 (6.6%), and did not change for 1 (1.7%) of the comparisons. CONCLUSIONS: Value assessment should capture the range of costs and benefits of an intervention. The exclusion of productivity costs can alter, often underestimating, the assessment of value. This may affect coverage decisions-inclusion or exclusion from the insurance benefit-based on these assessments. Value assessment reports intended to be used for health care decision making should include productivity and elevate its visibility by using base-case analyses rather than scenario analyses. DISCLOSURES: This study was funded by the National Pharmaceutical Council. Karmarkar is currently a postdoctoral fellow at the National Pharmaceutical Council. Graff and Westrich are employees of the National Pharmaceutical Council, which provides unrestricted research grants to value assessment bodies including ICER and IVI.

Is Real-World Evidence Used in P&T Monographs and Therapeutic Class Reviews?

BACKGROUND: Payers are faced with making coverage and reimbursement decisions based on the best available evidence. Often these decisions apply to patient populations, provider networks, and care settings not typically studied in clinical trials. Treatment effectiveness evidence is increasingly available from electronic health records, registries, and administrative claims. However, little is known about when and what types of real-world evidence (RWE) studies inform pharmacy and therapeutic (P&T) committee decisions. OBJECTIVE: To evaluate evidence sources cited in P&T committee monographs and therapeutic class reviews and assess the design features and quality of cited RWE studies. METHODS: A convenience sample of representatives from pharmacy benefit management, health system, and health plan organizations provided recent P&T monographs and therapeutic class reviews (or references from such documents). Two investigators examined and grouped references into major categories (published studies, unpublished studies, and other/unknown) and multiple subcategories (e.g., product label, clinical trials, RWE, systematic reviews). Cited comparative RWE was reviewed to assess design features (e.g., population, data source, comparators) and quality using the Good ReseArch for Comparative Effectiveness (GRACE) Checklist. RESULTS: Investigators evaluated 565 references cited in 27 monographs/therapeutic class reviews from 6 managed care organizations. Therapeutic class reviews mostly cited published clinical trials (35.3%, 155/439), while single-product monographs relied most on manufacturer-supplied information (42.1%, 53/126). Published RWE comprised 4.8% (21/439) of therapeutic class review references, and none (0/126) of the monograph references. Of the 21 RWE studies, 12 were comparative and assessed patient care settings and outcomes typically not included in clinical trials (community ambulatory settings [10], long-term safety [8]). RWE studies most frequently were based on registry data (6), conducted in the United States (6), and funded by the pharmaceutical industry (5). GRACE Checklist ratings suggested the data and methods of these comparative RWE studies were of high quality. CONCLUSIONS: RWE was infrequently cited in P&T materials, even among therapeutic class reviews where RWE is more readily available. Although few P&T materials cited RWE, the comparative RWE studies were generally high quality. More research is needed to understand when and what types of real-world studies can more routinely inform coverage and reimbursement decisions. DISCLOSURES: This project was funded by the National Pharmaceutical Council. Hurwitz, Brown, Peters, and Malone have nothing to disclose. Graff is employed by the National Pharmaceutical Council Part of this study was presented as a poster presentation at the AMCP Managed Care & Specialty Pharmacy 2016 Annual Meeting; April 19-22, 2016; San Francisco, CA. Study concept and design were primarily contributed by Malone and Graff, along with Hurwitz and Brown. All authors participated in data collection, and data interpretation was performed by Malone, Hurwitz, and Graff, with assistance from Brown and Peters. The manuscript was written primarily by Hurwitz and Malone, along with Graff, Brown, and Peters, and revised by Malone, Brown, Peters, Hurwitz, and Graff.

Evolution of the AMCP Format for Formulary Submissions.

DISCLOSURES: No funding was required for this project. The authors are or have been members of the Format Executive Committee.

Cost-effectiveness of intensive atorvastatin therapy in secondary cardiovascular prevention in the United Kingdom, Spain, and Germany, based on the Treating to New Targets study.

The Treating to New Targets (TNT) clinical trial found that intensive 80 mg atorvastatin (A80) treatment reduced cardiovascular events by 22% when compared to 10 mg atorvastatin (A10) treatment. We evaluated the cost-effectiveness of intensive A80 vs A10 treatment in the United Kingdom (UK), Spain, and Germany. A lifetime Markov model was developed to predict cardiovascular disease-related events, costs, survival, and quality-adjusted life-years (QALYs). Treatment-specific event probabilities were estimated from the TNT clinical trial. Post-event survival, health-related quality of life, and country-specific medical-care costs were estimated using published sources. Intensive treatment with A80 increased both the per-patient QALYs and corresponding costs of care, when compared to the A10 treatment, in all three countries. The incremental cost per QALY gained was 9,500, 21,000, and 15,000 in the UK, Spain, and Germany, respectively. Intensive A80 treatment is estimated to be cost-effective when compared to A10 treatment in secondary cardiovascular prevention.

Patient-Community Perspectives on Real-World Evidence: Enhancing Engagement, Understanding, and Trust.

BACKGROUND: Healthcare stakeholders have pronounced both enthusiasm and apprehension over the expanding use of real-world evidence (RWE). The patient community-those who benefit from new treatments but are vulnerable to potential safety risks and whose routine medical encounters are used to generate RWE-has been less vocal. Understanding patient perspectives on the use of RWE to guide clinical decision making and inform regulatory decisions and value assessments is imperative. METHODS: We convened a day-long, multi-stakeholder roundtable in Washington D.C., USA, on 31 July 2017 to gather patient-community views on RWE and related concerns and the communications, information and tools needed by patients to understand, trust, and use RWE. Participants included a convenience sample of National Health Council (NHC) members primarily representing patient groups as well as non-patient members with an interest in RWE. Participants were organized into small, pre-assigned groups, ensuring representativeness across stakeholders and patient leadership. Discussions, including storyboards, notes, and illustrative examples were captured and later analyzed thematically by NHC staff. RESULTS: Ten RWE themes emerged: (1) most patients were unaware of RWE and its actual or potential uses, (2) common definitions for real-world data and RWE are needed, (3) patient organizations need RWE skills and tools, (4) patient-scientist partnerships can help differentiate high-quality RWE, (5) RWE should inform decision making, (6) clinician support is needed for RWE uptake in patient decision making, (7) communications to patients should be balanced and empowering, (8) context of use impacts RWE acceptability/trust, (9) privacy/data ownership require clarity, and (10) patient-generated data are also real-world data (RWD). CONCLUSION: Patients see great possibility in using RWE to understand how a treatment works-to find someone that "looks like me" as assurance of how a treatment might benefit them personally. Patient groups will play a critical role in helping to educate constituents on understanding, contributing to, and using RWE. To maximize patient uptake and the co-development and application of RWE, patient groups require education and tools.

Little Consistency In Evidence Cited By Commercial Plans For Specialty Drug Coverage.

We found wide variation in the evidence that US commercial health plans reported reviewing in their specialty drug coverage policies. There was little consistency in the numbers or types of studies cited by health plans. On average, only 15 percent of health plans' coverage policies cited the same study evaluating a specific drug for a specific indication.

Specialty Drug Coverage Varies Across Commercial Health Plans In The US.

We analyzed specialty drug coverage decisions issued by the largest US commercial health plans to examine variation in coverage and the consistency of those decisions with indications approved by the Food and Drug Administration (FDA). Across 3,417 decisions, 16 percent of the 302 drug-indication pairs were covered the same way by all of the health plans, and 48 percent were covered the same way by 75 percent of the plans. Specifically, 52 percent of the decisions were consistent with the FDA label, 9 percent less restrictive, 2 percent mixed (less restrictive in some ways but more restrictive in others), and 33 percent more restrictive, while 5 percent of the pairs were not covered. Health plans restricted coverage of drugs indicated for cancer less often than they did coverage of drugs indicated for other diseases. Using multivariate regression, we found that several drug-related factors were associated with less restrictive coverage, including indications for orphan diseases or pediatric populations, absence of safety warnings, time on the market, lack of alternatives, and expedited FDA review. Variations in coverage have implications for patients' access to treatment and health system costs.

Does a One-Size-Fits-All Cost-Sharing Approach Incentivize Appropriate Medication Use? A Roundtable on the Fairness and Ethics Associated with Variable Cost Sharing.

BACKGROUND: Tiered formularies, in which patients pay copays or coinsurance out-of-pocket (OOP), are used to manage costs and encourage more efficient health care resource use. Formulary tiers are typically based on the cost of treatment rather than the medical appropriateness for the patient. Cost sharing may have unintended consequences on treatment adherence and health outcomes. Use of higher-cost, higher-tier medications can be due to a variety of factors, including unsuccessful treatment because of lack of efficacy or side effects, patient clinical or genetic characteristics, patient preferences to avoid potential side effects, or patient preferences based on the route of administration. For example, patients with rheumatoid arthritis may be required to fail low-cost generic treatments before obtaining coverage for a higher-tier tumor necrosis factor alpha inhibitor for which they would have a larger financial burden. Little is known about stakeholders' views on the acceptability of greater patient cost sharing if the individual patient characteristics lead to the higher-cost treatments. OBJECTIVE: To identify and discuss the trade-offs associated with variable cost sharing in pharmacy benefits. METHODS: To discuss the trade-offs associated with variable cost sharing in pharmacy benefits, we convened an expert roundtable of patient, payer, and employer representatives (panelists). Panelists reviewed background white papers, including an ethics framework; actuarial analysis; legal review; and stakeholder perspectives representing health plan, employer, and patient views. Using case studies, panelists were asked to consider (a) when it would be more (or less) acceptable to require higher cost sharing; (b) the optimal distribution of financial burdens across patients, all plan members, and employers; and (c) the existing barriers and potential solutions to align OOP costs with medically appropriate treatments. RESULTS: Panelists felt it was least acceptable for patients to have greater OOP costs if the use of the higher-cost treatment was due to biological reasons such as step therapy (6 = unacceptable, 9 = neutral, 2 = acceptable) or diagnostic results (5 = unacceptable, 10 = neutral, and 2 = acceptable). In contrast, panelists felt it was more acceptable for patients to pay greater OOP costs when treatment choice was based on preferences to avoid a side-effect risk (1 = unacceptable, 3 = neutral, and 13 = acceptable) or the route/frequency of administration (1 = unacceptable, 1 = neutral, and 15 = acceptable). Five guiding principles emerged from the discussion: When patients have tried lower-cost therapies unsuccessfully, the benefits of higher-cost treatments were certain and significant, the cost difference between treatments was aligned with improved benefits, and penalties due to bad luck were mitigated, then cost-sharing differences should be minimized but not eliminated. CONCLUSIONS: Patient OOP costs can affect the use of both inappropriate and appropriate medications. This study identified 5 guiding principles to determine when it was more (or less) acceptable for patients with the same or similar conditions to have different OOP costs. Barriers that hinder the alignment of care and patient cost sharing exist. Policies that facilitate the alignment of patient cost sharing with appropriate care are needed. DISCLOSURES: Funding for this roundtable was provided by the National Pharmaceutical Council (NPC). Graff and Dubois are employed by the NPC. Shih was employed by the NPC at the time of this study. Barker, Dieguez, Sherman, and Larson received consulting fees for participation in this study. Larson also reports receiving grants and other payment from multiple major pharmaceutical manufacturers outside of this study. The NPC employees developed the study design and chose the case studies in collaboration with the white paper authors. The roundtable was facilitated by Dubois, and the meeting summary and manuscript were written by Graff and Shih, with revisions by all roundtable participants. The abstract for this article was previously presented as a poster at the following meetings: Stakeholder perspectives on balancing patient-centeredness and drug costs in the design of pharmacy benefits. Presented at: Academy of Managed Care Pharmacy 27th Annual Meeting & Expo; San Diego, California; April 8, 2015. Considering efficiency and fairness in the design of prescription drug benefits: seeking a balanced approach to improve patient access to medically appropriate medication and manage drug costs. Presented at: AcademyHealth Annual Research Meeting; Minneapolis, Minnesota; June 15, 2015. Study concept and design were contributed by Shih, Dubois, and Graff, along with Barker and Dieguez. Barker and Dieguez took the lead in data collection, assisted by Graff, Shih, and Dubois. Data interpretation was performed by Shih, Larson, Sherman, and Graff, with assistance from Dubois. The manuscript was written and revised by Graff and Shih, with assistance from the other authors.

Is Real-World Evidence Used in P&T Monographs and Therapeutic Class Reviews?

BACKGROUND: Payers are faced with making coverage and reimbursement decisions based on the best available evidence. Often these decisions apply to patient populations, provider networks, and care settings not typically studied in clinical trials. Treatment effectiveness evidence is increasingly available from electronic health records, registries, and administrative claims. However, little is known about when and what types of real-world evidence (RWE) studies inform pharmacy and therapeutic (P&T) committee decisions. OBJECTIVE: To evaluate evidence sources cited in P&T committee monographs and therapeutic class reviews and assess the design features and quality of cited RWE studies. METHODS: A convenience sample of representatives from pharmacy benefit management, health system, and health plan organizations provided recent P&T monographs and therapeutic class reviews (or references from such documents). Two investigators examined and grouped references into major categories (published studies, unpublished studies, and other/unknown) and multiple subcategories (e.g., product label, clinical trials, RWE, systematic reviews). Cited comparative RWE was reviewed to assess design features (e.g., population, data source, comparators) and quality using the Good ReseArch for Comparative Effectiveness (GRACE) Checklist. RESULTS: Investigators evaluated 565 references cited in 27 monographs/therapeutic class reviews from 6 managed care organizations. Therapeutic class reviews mostly cited published clinical trials (35.3%, 155/439), while single-product monographs relied most on manufacturer-supplied information (42.1%, 53/126). Published RWE comprised 4.8% (21/439) of therapeutic class review references, and none (0/126) of the monograph references. Of the 21 RWE studies, 12 were comparative and assessed patient care settings and outcomes typically not included in clinical trials (community ambulatory settings [10], long-term safety [8]). RWE studies most frequently were based on registry data (6), conducted in the United States (6), and funded by the pharmaceutical industry (5). GRACE Checklist ratings suggested the data and methods of these comparative RWE studies were of high quality. CONCLUSIONS: RWE was infrequently cited in P&T materials, even among therapeutic class reviews where RWE is more readily available. Although few P&T materials cited RWE, the comparative RWE studies were generally high quality. More research is needed to understand when and what types of real-world studies can more routinely inform coverage and reimbursement decisions. DISCLOSURES: This project was funded by the National Pharmaceutical Council. Hurwitz, Brown, Peters, and Malone have nothing to disclose. Graff is employed by the National Pharmaceutical Council Part of this study was presented as a poster presentation at the AMCP Managed Care & Specialty Pharmacy 2016 Annual Meeting; April 19-22, 2016; San Francisco, CA. Study concept and design were primarily contributed by Malone and Graff, along with Hurwitz and Brown. All authors participated in data collection, and data interpretation was performed by Malone, Hurwitz, and Graff, with assistance from Brown and Peters. The manuscript was written primarily by Hurwitz and Malone, along with Graff, Brown, and Peters, and revised by Malone, Brown, Peters, Hurwitz, and Graff.