Real-World Evidence: Useful in the Real World of US Payer Decision Making? How? When? And What Studies?

OBJECTIVES: To examine how real-world evidence (RWE) is currently perceived and used in managed care environments, especially to inform pharmacy and therapeutic (P&T) committee decisions, to assess which study factors (e.g., data, design, and funding source) contribute to RWE utility in decisions, and to identify barriers to consideration of RWE studies in P&T decision making. METHODS: We conducted focus groups/telephone-based interviews and surveys to understand perceptions of RWE and assess awareness, quality, and relevance of two high-profile examples of published RWE studies. A purposive sample comprised 4 physicians, 15 pharmacists, and 1 researcher representing 18 US health plans and health system organizations. RESULTS: Participants reported that RWE was generally used, or useful, to inform safety monitoring, utilization management, and cost analysis, but less so to guide P&T decisions. Participants were not aware of the two sample RWE studies but considered both studies to be valuable. Relevant research questions and outcomes, transparent methods, study quality, and timely results contribute to the utility of published RWE. Perceived organizational barriers to the use of published RWE included lack of skill, training, and timely study results. CONCLUSIONS: Payers recognize the value of RWE, but use of such studies to inform P&T decisions varies from organization to organization and is limited. Relevance to payers, timeliness, and transparent methods were key concerns with RWE. Participants recognized the need for continuing education on evaluating and using RWE to better understand the study methods, findings, and applicability to their organizations.

Integrative medicine primary care: assessing the practice model through patients' experiences.

BACKGROUND: The University of Arizona Integrative Health Center (UAIHC) was an innovative integrative medicine (IM) adult primary care clinic in Phoenix, Arizona. UAIHC used a hybrid payment model to deliver comprehensive healthcare that includes conventional and complementary medical treatments. METHODS: Fidelity measures were collected to evaluate how well the IM care delivery process matched ideals for IM. Patient experiences are presented here. Patients visiting UAIHC on 1 of 10 randomly selected days between September 2013 and February 2015 were surveyed. Patients were asked about their experience with: holistic care; promotion of health, self-care, and well-being; relationship and communication with practitioners; and overall satisfaction. RESULTS: Eighty-three patients completed surveys. Based on patient-reported experiences, UAIHC delivered IM care as defined by the practice model. CONCLUSIONS: Patients received holistic care, established positive caring relationships with providers who promoted their self-care and well-being, and reported high overall satisfaction with UAIHC.

Patient knowledge and use of acetaminophen in over-the-counter medications.

OBJECTIVES To evaluate patient knowledge of over-the-counter (OTC) products containing acetaminophen and to determine patients' accuracy in dosing adult, child, and infant formulations. DESIGN Cross-sectional study. SETTING Six community pharmacies in Tucson, AZ, between February and May 2011. PARTICIPANTS 88 adults aged 19 to 89 years. INTERVENTION Investigator-administered, semistructured interviews. MAIN OUTCOME MEASURES Patient knowledge of and ability to safely use OTC products containing acetaminophen, including understanding risks, identifying products, and dosing different formulations. RESULTS Although most (86%) participants heard of acetaminophen, only 68% understood at least one of its uses and only 9% knew the abbreviation APAP. Virtually all knew that consuming too much acetaminophen in 1 day could be harmful, but only 17% and 35% knew that overdoses could result in death or liver damage, respectively. On average, participants correctly identified 80% (range 27-100%) of products with and without acetaminophen from a lineup of 11 OTC products. Although 38% (n = 84) of participants correctly measured both the child and infant doses of acetaminophen, doses ranged from one-half to twice the amount of the labeled child dose and one-third of the labeled infant dose. Findings from the regression analysis suggested that on average, women and those with college degrees had higher overall scores, while participants' age or parent status were nonsignificant predictors. CONCLUSION Many patients remain confused about using acetaminophen safely, signaling the need for greater patient education to prevent unintentional harm. The results further specify common misunderstandings to address during patient contact, which also includes replacing "APAP" with "acetaminophen" on any prescription bottle labels or patient-directed information.

Insights into insurance coverage for digital therapeutics: A qualitative study of US payer perspectives.

BACKGROUND: In the last decade there has been an increase in the development and marketing of digital therapeutic (DTx) products aiming to prevent, manage, or treat a medical disorder or disease. Health insurance coverage for these products is not well established, and payers are facing increasing pressure to include these products as a covered benefit. OBJECTIVE: To examine factors and characteristics that could drive health insurance coverage of DTx products from US payers' and coverage decision-makers' perspectives. METHODS: This was a qualitative noninterventional, cross-sectional study conducted from August 2022 to October 2022. Virtual focus group meetings with pharmacy benefit managers/directors or medical directors representing a range of health insurance organizations were held following a semistructured interview guide. Convenience and snowball sampling techniques were used to identify participants. Transcripts were coded and analyzed with Atlas.ti software to identify common themes and subthemes. RESULTS: Five focus group meetings and 1 individual interview were held from August to October 2022. Participants (n = 22) were mostly pharmacists (n = 18, 85%) with more than 15 years of experience (n = 18, 85%). Some participants indicated that DTx products for diabetes (n = 6, 29%), mental/behavioral health (n = 3, 14%), and substance abuse disorders (n = 3, 14%) were already covered by their organizations. The topics generating the most comments grouped by code were issues around the evidence for DTx (67 unique comments) and barriers for coverage (60 unique comments). Participants indicated they want to have evidence of effectiveness that is similar to traditional pharmaceutical products. Barriers for coverage included a need to revise benefit policies, exclusion of nonprescription products, and mechanisms for billing. DTx products with an indication for mental/behavioral health were viewed as most likely to be reimbursed. Coverage of DTx products may occur under either the pharmacy or medical benefit. CONCLUSIONS: Health care payers stated that evidence of effectiveness was a necessary condition for health insurance coverage of DTx products. Given these are relatively new in health care, payers had more questions than answers regarding how these products will be integrated into health benefits.

Cost of illness of stage IV non-small cell lung cancer (NSCLC) positive for programmed cell death ligand 1 (PD-L1) in the US.

BACKGROUND: Non-small cell lung cancer (NSCLC) positive for programmed cell death ligand 1 (PD-L1) may be eligible for targeted immunotherapies. Literature does not currently estimate direct costs associated with this population. We aimed to identify the total direct costs associated with PD-L1 positive stage IV NSCLC treated with immunotherapy. METHODS: Using progression-free survival, overall survival, treatment-related serious adverse events leading to hospitalization, and end-of-life resource use, we estimated costs for one year of treatment in this incidence-based study. Data were obtained from online databases, guideline recommendations, clinical trials, and proprietary market share data. We summed the costs of PD-L1 immunohistochemistry (IHC) tests, drugs, hospitalizations, and deaths associated with treatment, estimating the overall cost-of-illness for stage IV NSCLC in the United States in 2021. RESULTS: An estimated 22,711 patients in the US had stage IV NSCLC treated with PD-L1 immunotherapy in 2021. Total 2021 costs were estimated at $3.01 billion. Drugs (including immunotherapy, second-line chemotherapy, and other oncology drugs) accounted for nearly 97% ($2.91 billion) of the total. CONCLUSIONS: PD-L1 positive stage IV NSCLC treatment is a costly condition with annual direct medical costs of $3.01 billion. The primary cost driver was immunotherapy, making up 74.6% of the total cost.

Cost-Effectiveness of PD-L1 Testing in Non-Small Cell Lung Cancer (NSCLC) Using In Vitro Diagnostic (IVD) Versus Laboratory-Developed Test (LDT).

INTRODUCTION: Accurate PD-L1 testing for non-small cell lung cancer (NSCLC) maximizes the benefits of immune checkpoint inhibitor (ICI) drugs like pembrolizumab. False negative test results deny ICI treatments to eligible patients, worsening clinical and economic outcomes, while false positives increase costs by using ICI treatments without their benefits. This study evaluates the cost-effectiveness of PD-L1 testing with an in vitro diagnostic (IVD) compared to a laboratory-developed test (LDT) for allocating patients with NSCLC to treatment with either pembrolizumab or chemotherapy using the German healthcare system as a model. METHODS: We developed a decision analytical model to evaluate the cost-effectiveness of PD-L1 testing with a regulatory body approved IVD compared to an LDT from the national German healthcare payer (statutory health insurance system) perspective. Accuracy of PD-L1 testing was based on data from two independent proficiency testing programs. The 1-year model was based on outcomes data from the KEYNOTE-024 clinical trial and treatment patterns reflecting current German practices. RESULTS: IVDs produced accurate PD-L1 testing results in 93% (752/811) of tested cases compared to 73% (492/672) with LDTs. Most misclassifications concerned false negatives, occurring in 21% of LDTs vs 7% of IVDs. Total costs of the IVD group (48,878 €) were 196 € higher than the LDT group (48,682 €). These costs incorporate testing, first- and second-line therapy, managing treatment-related grade 3+ adverse events (AEs), and end-of-life costs for those who died within the year. Total effectiveness (percentage of patients successfully diagnosed and prescribed the correct therapy per German treatment guidelines) was 19 percentage points higher for the IVD group (88%) compared to the LDT group (69%). These differences in costs and effects lead to an incremental cost-effectiveness ratio (ICER) of 1057 €. CONCLUSION: Compared to LDT technology, on-label IVD use for PD-L1 testing is only slightly more costly and substantially more effective for aligning patients with PD-L1-positive NSCLC with ICI therapy according to German practice guidelines. Given these findings, changes to testing and reimbursement policies may be considered to maximize patient outcomes in NSCLC.

Shared Decision-Making for Drug-Drug Interactions: Formative Evaluation of an Anticoagulant Drug Interaction.

BACKGROUND: Warnings about drug-drug interactions (DDIs) between warfarin and nonsteroidal anti-inflammatory drugs (NSAIDs) within electronic health records indicate potential harm but fail to account for contextual factors and preferences. We developed a tool called DDInteract to enhance and support shared decision-making (SDM) between patients and physicians when both warfarin and NSAIDs are used concurrently. DDInteract was designed to be integrated into electronic health records using interoperability standards. OBJECTIVE: The purpose of this study was to conduct a formative evaluation of a DDInteract that incorporates patient and product contextual factors to estimate the risk of bleeding. METHODS: A randomized formative evaluation was conducted to compare DDInteract to usual care (UC) using physician-patient dyads. Using case vignettes, physicians and patients on warfarin participated in simulated virtual clinical encounters where they discussed the use of taking ibuprofen and warfarin concurrently and determined an appropriate therapeutic plan based on the patient's individualized risk. Dyads were randomized to either DDInteract or UC. Participants completed a postsession interview and survey of the SDM process. This included the 9-item Shared Decision-Making Questionnaire (SDM-Q-9), tool usability and workload National Aeronautics and Space Administration (NASA) Task Load Index, Unified Theory of Acceptance and Use of Technology (UTAUT), Perceived Behavioral Control (PBC) scale, System Usability Scale (SUS), and Decision Conflict Scale (DCS). They also were interviewed after the session to obtain perceptions on DDInteract and UC resources for DDIs. RESULTS: Twelve dyad encounters were performed using virtual software. Most (n=11, 91.7%) patients were over 50 years of age, and 9 (75%) had been taking warfarin for more than 2 years (75%). Regarding scores on the SDM-Q-9, participants rated DDInteract higher than UC for questions pertaining to helping patients clarify the decision (P=.03), involving patients in the decision (P=.01), displaying treatment options (P<.001), identifying advantages and disadvantages (P=.01), and facilitating patient understanding (P=.01) and discussion of preferences (P=.01). Five of the 8 UTAUT constructs showed differences between the 2 groups, favoring DDInteract (P<.05). Usability ratings from the SUS were significantly higher (P<.05) for physicians using DDInteract compared to those in the UC group but showed no differences from the patient's perspective. No differences in patient responses were observed between groups using the DCS. During the session debrief, physicians indicated little concern for the additional time or workload entailed by DDInteract use. Both clinicians and patients indicated that the tool was beneficial in simulated encounters to understand and mitigate the risk of harm from this DDI. CONCLUSIONS: Overall, DDInteract may improve encounters where there is a risk of bleeding due to a potential drug-drug interaction involving anticoagulants. Participants rated DDInteract as logical and useful for enhancing SDM. They reported that they would be willing to use the tool for an interaction involving warfarin and NSAIDs.

Cost-effectiveness of once-daily vs twice-daily tacrolimus among Hispanic and Black kidney transplant recipients.

BACKGROUND: Tacrolimus is a first-line immunosuppressive therapy to prevent rejection and graft failure in kidney transplant recipients. Once-daily extended-release tacrolimus tablets (LCPT) have been shown to be efficacious, particularly for Hispanic and Black patient subpopulations who are rapid metabolizers, but is more costly than twice-daily immediate-release tacrolimus (IR-Tac). OBJECTIVE: To evaluate the cost-effectiveness of LCPT during the first year of treatment vs IR-Tac in kidney transplant recipients who are Hispanic or Black. METHODS: A decision analytic model from a US payer perspective was developed using (1) subgroup outcomes data pooled from two phase 3 clinical trials that compared LCPT and IR-Tac, and (2) direct costs from real-world data sources (ie, costs of LCPT and IR-Tac treatments, biopsy-proven acute rejection, treatment-related serious adverse events [SAEs], graft failure, and consequent dialysis). The primary outcome was cost per successfully treated patient, defined as having a functioning graft after 1 year and without treatment-related SAEs. Probabilistic sensitivity analyses established distributions for cost and outcomes estimates, and a series of one-way sensitivity analyses identified parameters that had the most effect on results. RESULTS: Total overall cost for the Hispanic group was $14,765 for LCPT and $12,416 for IR-Tac, and total cost in the Black group was $16,626 for LCPT and $9,871 for IR-Tac. Total overall effectiveness of LCPT and IR-Tac was 88.32% and 84.75% in the Hispanic group and 93.24% and 85.78% in the Black group, respectively. The incremental cost-effectiveness ratio (ICER) for using LCPT over IR-Tac during the first year of treatment in the Hispanic group was $65,643 per additional successfully treated patient. The ICER for the Black group was $90,458. The single parameter having the most impact on results in both groups was the probability of a treatment-related SAE in IR-Tac, which accounted for 49% of variation in results in the Hispanic group and 46% in the Black group. CONCLUSIONS: Overall results for both groups show that LCPT is incrementally more costly and more effective compared with IR-Tac, indicating a trade-off scenario. LCPT is a cost-effective strategy if a decision makers' willingness to pay for 1 additional successfully treated patient exceeds the ICER and must be weighed against the costs of graft loss, continuing dialysis, and potential retransplant. This study provides a foundation for further research to update and expand inputs as more data become available to improve real-world relevance and decision making. DISCLOSURES: This study was funded by Veloxis Pharmaceuticals, Inc., which provided clinical trial file data and nonbinding feedback on the model structure, data interpretation, clinical expertise, manuscript review, and areas of publication interest (ie, managed care). Hurwitz, Grizzle, Villa Zapata, and Malone received grant funding from Veloxis Pharmaceuticals, Inc., through University of Arizona to conduct research and analysis for this study. Tyler is employed by Veloxis Pharmaceuticals, Inc. Some of the data reported and used in this research were available from the US Renal Data System, the US Bureau of Labor Statistics, and the Agency for Healthcare Research and Quality's Healthcare Cost and Utility Project. The interpretation and reporting of these data are the responsibility of the authors and in no way should be seen as an official policy or interpretation of the US government.

A Shared Decision-making Tool for Drug Interactions Between Warfarin and Nonsteroidal Anti-inflammatory Drugs: Design and Usability Study.

BACKGROUND: Exposure to life-threatening drug-drug interactions (DDIs) occurs despite the widespread use of clinical decision support. The DDI between warfarin and nonsteroidal anti-inflammatory drugs is common and potentially life-threatening. Patients can play a substantial role in preventing harm from DDIs; however, the current model for DDI decision-making is clinician centric. OBJECTIVE: This study aims to design and study the usability of DDInteract, a tool to support shared decision-making (SDM) between a patient and provider for the DDI between warfarin and nonsteroidal anti-inflammatory drugs. METHODS: We used an SDM framework and user-centered design methods to guide the design and usability of DDInteract-an SDM electronic health record app to prevent harm from clinically significant DDIs. The design involved iterative prototypes, qualitative feedback from stakeholders, and a heuristic evaluation. The usability evaluation included patients and clinicians. Patients participated in a simulated SDM discussion using clinical vignettes. Clinicians were asked to complete eight tasks using DDInteract and to assess the tool using a survey adapted from the System Usability Scale. RESULTS: The designed DDInteract prototype includes the following features: a patient-specific risk profile, dynamic risk icon array, patient education section, and treatment decision tree. A total of 4 patients and 11 clinicians participated in the usability study. After an SDM session where patients and clinicians review the tool concurrently, patients generally favored pain treatments with less risk of gastrointestinal bleeding. Clinicians successfully completed the tasks with a mean of 144 (SD 74) seconds and rated the usability of DDInteract as 4.32 (SD 0.52) of 5. CONCLUSIONS: This study expands the use of SDM to DDIs. The next steps are to determine if DDInteract can improve shared decision-making quality and to implement it across health systems using interoperable technology.

Is Real-World Evidence Used in P&T Monographs and Therapeutic Class Reviews?

BACKGROUND: Payers are faced with making coverage and reimbursement decisions based on the best available evidence. Often these decisions apply to patient populations, provider networks, and care settings not typically studied in clinical trials. Treatment effectiveness evidence is increasingly available from electronic health records, registries, and administrative claims. However, little is known about when and what types of real-world evidence (RWE) studies inform pharmacy and therapeutic (P&T) committee decisions. OBJECTIVE: To evaluate evidence sources cited in P&T committee monographs and therapeutic class reviews and assess the design features and quality of cited RWE studies. METHODS: A convenience sample of representatives from pharmacy benefit management, health system, and health plan organizations provided recent P&T monographs and therapeutic class reviews (or references from such documents). Two investigators examined and grouped references into major categories (published studies, unpublished studies, and other/unknown) and multiple subcategories (e.g., product label, clinical trials, RWE, systematic reviews). Cited comparative RWE was reviewed to assess design features (e.g., population, data source, comparators) and quality using the Good ReseArch for Comparative Effectiveness (GRACE) Checklist. RESULTS: Investigators evaluated 565 references cited in 27 monographs/therapeutic class reviews from 6 managed care organizations. Therapeutic class reviews mostly cited published clinical trials (35.3%, 155/439), while single-product monographs relied most on manufacturer-supplied information (42.1%, 53/126). Published RWE comprised 4.8% (21/439) of therapeutic class review references, and none (0/126) of the monograph references. Of the 21 RWE studies, 12 were comparative and assessed patient care settings and outcomes typically not included in clinical trials (community ambulatory settings [10], long-term safety [8]). RWE studies most frequently were based on registry data (6), conducted in the United States (6), and funded by the pharmaceutical industry (5). GRACE Checklist ratings suggested the data and methods of these comparative RWE studies were of high quality. CONCLUSIONS: RWE was infrequently cited in P&T materials, even among therapeutic class reviews where RWE is more readily available. Although few P&T materials cited RWE, the comparative RWE studies were generally high quality. More research is needed to understand when and what types of real-world studies can more routinely inform coverage and reimbursement decisions. DISCLOSURES: This project was funded by the National Pharmaceutical Council. Hurwitz, Brown, Peters, and Malone have nothing to disclose. Graff is employed by the National Pharmaceutical Council Part of this study was presented as a poster presentation at the AMCP Managed Care & Specialty Pharmacy 2016 Annual Meeting; April 19-22, 2016; San Francisco, CA. Study concept and design were primarily contributed by Malone and Graff, along with Hurwitz and Brown. All authors participated in data collection, and data interpretation was performed by Malone, Hurwitz, and Graff, with assistance from Brown and Peters. The manuscript was written primarily by Hurwitz and Malone, along with Graff, Brown, and Peters, and revised by Malone, Brown, Peters, Hurwitz, and Graff.

Real-World Evidence from the Integrative Medicine Primary Care Trial (IMPACT): Assessing Patient-Reported Outcomes at Baseline and 12-Month Follow-Up.

PURPOSE: The University of Arizona Integrative Health Center (UAIHC) was an innovative membership-supported integrative medicine (IM) adult primary care clinic in Phoenix, Arizona. UAIHC delivered healthcare using an integrative medicine model that combined conventional and complementary medical treatments, including nutrition, mind-body medicine, acupuncture, manual medicine, health coaching, educational classes, and groups. Results from pre-post evaluation of patient-reported outcomes on several standardized measures are presented here. METHODS: UAIHC patients completed surveys at baseline and after 12 months of continuous integrative primary care. Patients reported on perceived changes in health outcomes as measured by Short-Form Health Survey (SF-12 general, mental, and physical health), Perceived Stress Scale (PSS4), Work Productivity and Activity Impairment Questionnaire (WPAI), World Health Organization Well-Being Index (WHO-5), Pain Visual Analog Scale (VAS), Fatigue Severity Scale (VAS; FSS), Generalized Anxiety Disorder Scale (GAD2), Patient Health Questionnaire for depression (PHQ2), Pittsburgh Sleep Quality Index (PSQI) global rating of sleep quality, and the Behavioral Risk Factor Surveillance System (BRFSS; nutrition, exercise, and physical activity). Overall differences between time points were assessed for statistical significance. Patient demographics are also described. RESULTS: 177 patients completed baseline and follow-up outcome measures. Patients were predominantly white, female, college-educated, and employed. Baseline to one-year follow-up results indicate statistically significant improvements (p <.05) on all but perceived stress (PSS-4) and work absenteeism (WPAI). Clinical impact and/or practical effects are reported as percent change or standardized effect sizes whenever possible. Other demographic and descriptive information is summarized. CONCLUSIONS: Following one year of IM primary care at UAIHC, patient-reported outcomes indicated positive impacts in several areas of patients' lives: mental, physical, and overall health; work productivity; sleep quality; pain; fatigue; overall well-being; and physical activity.

Is Real-World Evidence Used in P&T Monographs and Therapeutic Class Reviews?

BACKGROUND: Payers are faced with making coverage and reimbursement decisions based on the best available evidence. Often these decisions apply to patient populations, provider networks, and care settings not typically studied in clinical trials. Treatment effectiveness evidence is increasingly available from electronic health records, registries, and administrative claims. However, little is known about when and what types of real-world evidence (RWE) studies inform pharmacy and therapeutic (P&T) committee decisions. OBJECTIVE: To evaluate evidence sources cited in P&T committee monographs and therapeutic class reviews and assess the design features and quality of cited RWE studies. METHODS: A convenience sample of representatives from pharmacy benefit management, health system, and health plan organizations provided recent P&T monographs and therapeutic class reviews (or references from such documents). Two investigators examined and grouped references into major categories (published studies, unpublished studies, and other/unknown) and multiple subcategories (e.g., product label, clinical trials, RWE, systematic reviews). Cited comparative RWE was reviewed to assess design features (e.g., population, data source, comparators) and quality using the Good ReseArch for Comparative Effectiveness (GRACE) Checklist. RESULTS: Investigators evaluated 565 references cited in 27 monographs/therapeutic class reviews from 6 managed care organizations. Therapeutic class reviews mostly cited published clinical trials (35.3%, 155/439), while single-product monographs relied most on manufacturer-supplied information (42.1%, 53/126). Published RWE comprised 4.8% (21/439) of therapeutic class review references, and none (0/126) of the monograph references. Of the 21 RWE studies, 12 were comparative and assessed patient care settings and outcomes typically not included in clinical trials (community ambulatory settings [10], long-term safety [8]). RWE studies most frequently were based on registry data (6), conducted in the United States (6), and funded by the pharmaceutical industry (5). GRACE Checklist ratings suggested the data and methods of these comparative RWE studies were of high quality. CONCLUSIONS: RWE was infrequently cited in P&T materials, even among therapeutic class reviews where RWE is more readily available. Although few P&T materials cited RWE, the comparative RWE studies were generally high quality. More research is needed to understand when and what types of real-world studies can more routinely inform coverage and reimbursement decisions. DISCLOSURES: This project was funded by the National Pharmaceutical Council. Hurwitz, Brown, Peters, and Malone have nothing to disclose. Graff is employed by the National Pharmaceutical Council Part of this study was presented as a poster presentation at the AMCP Managed Care & Specialty Pharmacy 2016 Annual Meeting; April 19-22, 2016; San Francisco, CA. Study concept and design were primarily contributed by Malone and Graff, along with Hurwitz and Brown. All authors participated in data collection, and data interpretation was performed by Malone, Hurwitz, and Graff, with assistance from Brown and Peters. The manuscript was written primarily by Hurwitz and Malone, along with Graff, Brown, and Peters, and revised by Malone, Brown, Peters, Hurwitz, and Graff.

Accepting Medication Therapy Management Recommendations to Add ACEIs or ARBs in Diabetes Care.

BACKGROUND: National guidelines and initiatives have promoted the use of angiotensin-converting enzyme inhibitors (ACEIs) and angiotensin receptor blockers (ARBs) for patients with diabetes. The University of Arizona Medication Management Center (UA-MMC) is contracted by Medicare health plans, pharmacy benefit managers (PBMs), and multiple commercial health insurance plans to provide medication therapy management (MTM) services for plan members. As part of the MTM program, recommendations have been made for those patients who may benefit from the addition of an ACEI/ARB. Although the intervention benefits and guidelines for using ACEIs/ARBs are clear, real-world evidence is needed to understand and potentially increase uptake of guideline interventions among eligible patients. OBJECTIVES: To (a) identify patient characteristics that predict acceptance of guideline recommendations to add ACEI/ARB medications to diabetic treatment via MTM services and (b) examine how well different case characteristics (i.e., patient age and sex, type and number of recommendation attempts, type of health care plan) predict the odds of adding ACEI/ARB medications to diabetic regimens when recommended through an MTM call center. METHODS: This was a retrospective analysis of secondary data provided by the UA-MMC. The de-identified national data included adult plan members with diabetes who the UA-MMC recommended adding an ACEI/ARB prescription based on 2012 national guidelines. The UA-MMC made recommendations by either patient letters, patient phone calls, physician faxes, or any combination thereof. We conducted a binary logistic regression analysis to assess the impact of case characteristics on the likelihood of accepting recommendations to add ACEI/ARB medications. The outcome variable was recommendation acceptance (yes/no), defined as new prescription claims for an ACEI/ARB within 120 days following the recommendation. Five predictor variables were assessed: (1) patient's age quartile; (2) method of communicating recommendations (letter, phone call, fax, or some combination thereof); (3) whether recommendations were made once or twice on separate dates; (4) patient's sex; and (5) type of health care plan. RESULTS: Recommendations were made for 31,495 members of health plans or PBMs that contracted with the UA-MMC. Patients' ages ranged from 19-90 (Mean =72.01; SD =10.21), with females comprising 56% of the sample. The recommendation to add ACEI/ARB medications was accepted for 14.5% (4,559) of patients. In most cases (73%), recommendations occurred via a letter to patients together with a fax to their providers. The fitted model, containing 3 predictor variables (age quartile, type of contact to communicate the recommendations, and whether recommendation contacts were made twice), was statistically significant, χ(2) (10; N = 31,495) = 112.82 (P < 0.001), indicating that the model was able to distinguish between those who did and did not accept UA-MMC's recommendations to add ACEI/ARB medications. The likelihood of recommendation acceptance decreased as patient age increased compared with patients in the first age quartile (ages 19-67; P ≤ 0.005 at all levels). Compared with sending only a provider fax, patients who received all 3 types of contact (provider fax with patient phone call and letter) were estimated to be 1.34 times more likely (34% increase) to have recommendation acceptance ( P = 0.004; 95% CI = 1.10-1.63). Similarly, patients who received only letters were also 1.32 times more likely (32% increase) than provider faxes alone to result in recommendation acceptance ( P = 0.003; 95% CI = 1.10-1.59). Patients for whom recommendations were made twice were less likely to have recommendation acceptance than for those contacted once, controlling for all other predictor variables in the model ( P < 0.001; OR = 0.77; 95% CI = 0.69-0.86). CONCLUSIONS: Recommendations to add an ACEI/ARB to diabetic regimens are more likely to be accepted for younger patients and those who receive recommendations through all 3 communication types (provider fax combined with patient phone call and letter) or just letters than provider faxes alone. Further research is needed to understand why prescribers are not accepting MTM recommendations.

Ability and Use of Comparative Effectiveness Research by P&T Committee Members and Support Staff: A 1-Year Follow-up.

BACKGROUND: In recent years, comparative effectiveness tools and methods have evolved to assist health care decision makers in identifying optimal therapies. In-person training programs on comparative effectiveness research may be helpful in understanding and applying this information. OBJECTIVE: To provide a follow-up assessment of the use of comparative effectiveness research (CER) in the pharmacy and therapeutics (P&T) committee decision-making process, using information collected from participants 1 year after attending a live continuing education program, in which participants were taught about CER designs and how to access available CER resources through the Agency for Healthcare Research and Quality (AHRQ) Effective Health Care (EHC) Program. METHODS: A retrospective, cross-sectional questionnaire was developed and mailed to 2 groups of individuals: CER workshop attendees and interested nonattendees (expressing an interest in attending a workshop but did not attend for unknown reasons). The questionnaire asked respondents to indicate personal and organizational use of CER in the decision-making process. Participants were asked to indicate whether their knowledge, ability, and use of CER studies increased since participating in the program. Data were analyzed using nonparametric tests to compare the responses of attendees and nonattendees, as well as overall reliability of the instrument. RESULTS: A total of 164 respondents completed the questionnaire (63 attendees and 101 nonattendees; overall response rate = 44%). The majority of respondents were pharmacists (n = 157, 95.7%) and were affiliated with a hospital (n = 106, 64.6%). Proportions of attendees and nonattendees differed significantly in the use of EHC research reviews/reports (45% and 28%, respectively; P = 0.02) and EHC executive summaries of research reviews/reports (48% and 29%, respectively; P = 0.01). At 1-year follow-up, the majority of attendees reported an increase ("somewhat" or "very much") in knowledge of CER (91.5%), ability to use CER (83.0%), and use of CER studies (58.7%). CONCLUSIONS: Health professionals attending a continuing education CER program reported higher use of EHC CER materials compared with nonattendees. Additionally, attendees reported increased use of CER in clinical decision making. A continuing education program such as this may provide an effective avenue for introducing CER methods and resources to the P&T committee and clinical decision-making processes. DISCLOSURES: This project was supported by grant number R18HS019220 from the Agency of Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency of Healthcare Research and Quality. Augustine has stock in Pfizer. The authors declared no other potential conflicts of interest associated with this study. Study concept and design were primarily contributed by Malone, along with Warholak, Hines, Brown, Hurwitz, and Taylor. Warholak, Hines, Brown, Hurwitz, and Taylor collected the data, assisted by Malone, Brixner, Cobaugh, and Schlaifer. Data interpretation was performed by Malone and Augustine, with assistance from the other authors. The manuscript was written primarily by Augustine, with assistance from Malone, Sun, Warholak, Hines, Brown, Hurwitz, and Taylor. Malone revised the manuscript, assisted by Warholak, Hines, Brown, Hurwitz, Taylor, Brixner, Cobaugh, and Schlaifer.

Size and consistency of problem-solving consultation outcomes: an empirical analysis.

In this study, we analyzed extant data to evaluate the variability and magnitude of students' behavior change outcomes (academic, social, and behavioral) produced by consultants through problem-solving consultation with teachers. Research questions were twofold: (a) Do consultants produce consistent and sizeable positive student outcomes across their cases as measured through direct and frequent assessment? and (b) What proportion of variability in student outcomes is attributable to consultants? Analyses of extant data collected from problem-solving consultation outcome studies that used single-case, time-series AB designs with multiple participants were analyzed. Four such studies ultimately met the inclusion criteria for the extant data, comprising 124 consultants who worked with 302 school teachers regarding 453 individual students. Consultants constituted the independent variable, while the primary dependent variable was a descriptive effect size based on student behavior change as measured by (a) curriculum-based measures, (b) permanent products, or (c) direct observations. Primary analyses involved visual and statistical evaluation of effect size magnitude and variability observed within and between consultants and studies. Given the nested nature of the data, multilevel analyses were used to assess consultant effects on student outcomes. Results suggest that consultants consistently produced positive effect sizes on average across their cases, but outcomes varied between consultants. Findings also indicated that consultants, teachers, and the corresponding studies accounted for a significant proportion of variability in student outcomes. This investigation advances the use of multilevel and integrative data analyses to evaluate consultation outcomes and extends research on problem-solving consultation, consultant effects, and meta-analysis of case study AB designs. Practical implications for evaluating consultation service delivery in school settings are also discussed.

Evaluation of an educational program on deciphering heterogeneity for medical coverage decisions.

BACKGROUND: It is increasingly important for decision makers, such as medical and pharmacy managers (or pharmacy therapeutics committee members and staff), to understand the variation and diversity in treatment response as decisions shift from an individual patient perspective to optimizing care for populations of patients. OBJECTIVE: To assess the effectiveness of an instructional program on heterogeneity designed for medical and pharmacy managers. METHODS: A live educational program was offered to members of the Academy of Managed Care Pharmacy at the fall 2012 educational meeting and also to medical directors and managers attending a national payer roundtable meeting in October 2012. Participants completed a retrospective pretest-posttest assessment of their knowledge, attitudes, and self-efficacy immediately following the program. Participants were offered the opportunity to participate in a follow-up assessment 6 months later. Willing participants for the follow-up assessment were contacted via e-mail and telephone. Rasch rating scale models were used to compare pre- and postscores measuring participants' knowledge about and attitude towards heterogeneity. RESULTS: A total of 49 individuals completed the retrospective pretest-posttest assessment and agreed to be a part of the program evaluation. Fifty percent (n = 25) of participants had heard of the phrase "heterogeneity of treatment effect," and 36 (72%) were familiar with the phrase "individualized treatment effect" prior to the live program. Participants reported a significant improvement in knowledge of heterogeneity (P less than 0.01) and attitudes about heterogeneity (P less than 0.01) immediately after attending the program. At the time of the educational program, participants had either never considered heterogeneity (26%) or reported not knowing (28%) whether their organizations considered it when determining basic coverage. Participants were more likely to report "sometimes" considering heterogeneity for determining necessity for individual appeals, prior authorization, tier placement for pharmaceutical therapies, and other types of medical management. At the 6-month follow-up, 21 of the 49 willing participants (43% response rate) completed the evaluation; participants continued to have a good understanding of heterogeneity, but there was no significant difference in attitudes towards heterogeneity between pre- and 6-month follow-up. CONCLUSION: A live educational program was effective in improving participants' immediate knowledge and attitudes regarding the topic of heterogeneity. Participating managed care pharmacists and medical managers indicated that heterogeneity of treatment effect was likely to be used in determining prior authorizations and determining necessity.

Health care decision makers' use of comparative effectiveness research: report from a series of focus groups.

BACKGROUND: Comparative effectiveness research (CER) is a helpful approach to improve health outcomes by developing and disseminating evidence-based information to patients, clinicians, and other decision makers about the most effective interventions. OBJECTIVES: To (a) identify the factors necessary to increase the use of the Agency for Healthcare Research and Quality's (AHRQ) CER reviews in hospitals and managed care organizations; (b) assess current awareness and implementation of CER materials in these facilities and organizations; and (c) inform development of content for a workshop on CER. METHODS: Pharmacy and therapeutics (PT) committee members and supportive personnel were recruited to participate in focus groups conducted at national health professional meetings. Prior to the sessions, each participant completed a prefocus group questionnaire evaluating the organization and process of the respondent's PT committee, as well as the respondent's role in the PT committee and awareness of AHRQ CER reports. Each session consisted of a focused discussion about CER and sources of evidence for PT monographs, and each participant completed a ballot to rank topics of importance for inclusion in a CER workshop for health care professionals involved in the PT process. Overarching themes were later identified using qualitative analysis of the transcripts of the focus group sessions. RESULTS: Thirty-nine (68%) pharmacists and 18 (32%) physicians involved in the PT process participated in 1 of 7 focus groups. Almost half of the participants had 6-15 years experience with the PT process. Participants represented health plans, hospitals, and health care systems. Two-thirds indicated they were aware of AHRQ's Effective Health Care Program's CER reviews, yet only 26% reported using the reviews in their organizations. The overarching themes reflected the need for timely and conclusive CER information; the role of the pharmacist as central to evidence synthesis for the PT process; and the need for educational programs in online formats that are designed primarily for pharmacists. CONCLUSION: Health care decision makers identified timeliness as a key factor for facilitating the use of AHRQ CER reviews and guides in hospitals and managed care organizations. To facilitate integration of CER into the decision-making process, it is imperative that key stakeholders have access to comprehensive and timely information. While the majority of participants indicated that they were aware of AHRQ CER reviews, few had used them in the PT process.