Real-World Evidence: Useful in the Real World of US Payer Decision Making? How? When? And What Studies?

OBJECTIVES: To examine how real-world evidence (RWE) is currently perceived and used in managed care environments, especially to inform pharmacy and therapeutic (P&T) committee decisions, to assess which study factors (e.g., data, design, and funding source) contribute to RWE utility in decisions, and to identify barriers to consideration of RWE studies in P&T decision making. METHODS: We conducted focus groups/telephone-based interviews and surveys to understand perceptions of RWE and assess awareness, quality, and relevance of two high-profile examples of published RWE studies. A purposive sample comprised 4 physicians, 15 pharmacists, and 1 researcher representing 18 US health plans and health system organizations. RESULTS: Participants reported that RWE was generally used, or useful, to inform safety monitoring, utilization management, and cost analysis, but less so to guide P&T decisions. Participants were not aware of the two sample RWE studies but considered both studies to be valuable. Relevant research questions and outcomes, transparent methods, study quality, and timely results contribute to the utility of published RWE. Perceived organizational barriers to the use of published RWE included lack of skill, training, and timely study results. CONCLUSIONS: Payers recognize the value of RWE, but use of such studies to inform P&T decisions varies from organization to organization and is limited. Relevance to payers, timeliness, and transparent methods were key concerns with RWE. Participants recognized the need for continuing education on evaluating and using RWE to better understand the study methods, findings, and applicability to their organizations.

Integrative medicine primary care: assessing the practice model through patients' experiences.

BACKGROUND: The University of Arizona Integrative Health Center (UAIHC) was an innovative integrative medicine (IM) adult primary care clinic in Phoenix, Arizona. UAIHC used a hybrid payment model to deliver comprehensive healthcare that includes conventional and complementary medical treatments. METHODS: Fidelity measures were collected to evaluate how well the IM care delivery process matched ideals for IM. Patient experiences are presented here. Patients visiting UAIHC on 1 of 10 randomly selected days between September 2013 and February 2015 were surveyed. Patients were asked about their experience with: holistic care; promotion of health, self-care, and well-being; relationship and communication with practitioners; and overall satisfaction. RESULTS: Eighty-three patients completed surveys. Based on patient-reported experiences, UAIHC delivered IM care as defined by the practice model. CONCLUSIONS: Patients received holistic care, established positive caring relationships with providers who promoted their self-care and well-being, and reported high overall satisfaction with UAIHC.

Insights into insurance coverage for digital therapeutics: A qualitative study of US payer perspectives.

BACKGROUND: In the last decade there has been an increase in the development and marketing of digital therapeutic (DTx) products aiming to prevent, manage, or treat a medical disorder or disease. Health insurance coverage for these products is not well established, and payers are facing increasing pressure to include these products as a covered benefit. OBJECTIVE: To examine factors and characteristics that could drive health insurance coverage of DTx products from US payers' and coverage decision-makers' perspectives. METHODS: This was a qualitative noninterventional, cross-sectional study conducted from August 2022 to October 2022. Virtual focus group meetings with pharmacy benefit managers/directors or medical directors representing a range of health insurance organizations were held following a semistructured interview guide. Convenience and snowball sampling techniques were used to identify participants. Transcripts were coded and analyzed with Atlas.ti software to identify common themes and subthemes. RESULTS: Five focus group meetings and 1 individual interview were held from August to October 2022. Participants (n = 22) were mostly pharmacists (n = 18, 85%) with more than 15 years of experience (n = 18, 85%). Some participants indicated that DTx products for diabetes (n = 6, 29%), mental/behavioral health (n = 3, 14%), and substance abuse disorders (n = 3, 14%) were already covered by their organizations. The topics generating the most comments grouped by code were issues around the evidence for DTx (67 unique comments) and barriers for coverage (60 unique comments). Participants indicated they want to have evidence of effectiveness that is similar to traditional pharmaceutical products. Barriers for coverage included a need to revise benefit policies, exclusion of nonprescription products, and mechanisms for billing. DTx products with an indication for mental/behavioral health were viewed as most likely to be reimbursed. Coverage of DTx products may occur under either the pharmacy or medical benefit. CONCLUSIONS: Health care payers stated that evidence of effectiveness was a necessary condition for health insurance coverage of DTx products. Given these are relatively new in health care, payers had more questions than answers regarding how these products will be integrated into health benefits.

Cost of illness of stage IV non-small cell lung cancer (NSCLC) positive for programmed cell death ligand 1 (PD-L1) in the US.

BACKGROUND: Non-small cell lung cancer (NSCLC) positive for programmed cell death ligand 1 (PD-L1) may be eligible for targeted immunotherapies. Literature does not currently estimate direct costs associated with this population. We aimed to identify the total direct costs associated with PD-L1 positive stage IV NSCLC treated with immunotherapy. METHODS: Using progression-free survival, overall survival, treatment-related serious adverse events leading to hospitalization, and end-of-life resource use, we estimated costs for one year of treatment in this incidence-based study. Data were obtained from online databases, guideline recommendations, clinical trials, and proprietary market share data. We summed the costs of PD-L1 immunohistochemistry (IHC) tests, drugs, hospitalizations, and deaths associated with treatment, estimating the overall cost-of-illness for stage IV NSCLC in the United States in 2021. RESULTS: An estimated 22,711 patients in the US had stage IV NSCLC treated with PD-L1 immunotherapy in 2021. Total 2021 costs were estimated at $3.01 billion. Drugs (including immunotherapy, second-line chemotherapy, and other oncology drugs) accounted for nearly 97% ($2.91 billion) of the total. CONCLUSIONS: PD-L1 positive stage IV NSCLC treatment is a costly condition with annual direct medical costs of $3.01 billion. The primary cost driver was immunotherapy, making up 74.6% of the total cost.

Cost-Effectiveness of PD-L1 Testing in Non-Small Cell Lung Cancer (NSCLC) Using In Vitro Diagnostic (IVD) Versus Laboratory-Developed Test (LDT).

INTRODUCTION: Accurate PD-L1 testing for non-small cell lung cancer (NSCLC) maximizes the benefits of immune checkpoint inhibitor (ICI) drugs like pembrolizumab. False negative test results deny ICI treatments to eligible patients, worsening clinical and economic outcomes, while false positives increase costs by using ICI treatments without their benefits. This study evaluates the cost-effectiveness of PD-L1 testing with an in vitro diagnostic (IVD) compared to a laboratory-developed test (LDT) for allocating patients with NSCLC to treatment with either pembrolizumab or chemotherapy using the German healthcare system as a model. METHODS: We developed a decision analytical model to evaluate the cost-effectiveness of PD-L1 testing with a regulatory body approved IVD compared to an LDT from the national German healthcare payer (statutory health insurance system) perspective. Accuracy of PD-L1 testing was based on data from two independent proficiency testing programs. The 1-year model was based on outcomes data from the KEYNOTE-024 clinical trial and treatment patterns reflecting current German practices. RESULTS: IVDs produced accurate PD-L1 testing results in 93% (752/811) of tested cases compared to 73% (492/672) with LDTs. Most misclassifications concerned false negatives, occurring in 21% of LDTs vs 7% of IVDs. Total costs of the IVD group (48,878 €) were 196 € higher than the LDT group (48,682 €). These costs incorporate testing, first- and second-line therapy, managing treatment-related grade 3+ adverse events (AEs), and end-of-life costs for those who died within the year. Total effectiveness (percentage of patients successfully diagnosed and prescribed the correct therapy per German treatment guidelines) was 19 percentage points higher for the IVD group (88%) compared to the LDT group (69%). These differences in costs and effects lead to an incremental cost-effectiveness ratio (ICER) of 1057 €. CONCLUSION: Compared to LDT technology, on-label IVD use for PD-L1 testing is only slightly more costly and substantially more effective for aligning patients with PD-L1-positive NSCLC with ICI therapy according to German practice guidelines. Given these findings, changes to testing and reimbursement policies may be considered to maximize patient outcomes in NSCLC.

Shared Decision-Making for Drug-Drug Interactions: Formative Evaluation of an Anticoagulant Drug Interaction.

BACKGROUND: Warnings about drug-drug interactions (DDIs) between warfarin and nonsteroidal anti-inflammatory drugs (NSAIDs) within electronic health records indicate potential harm but fail to account for contextual factors and preferences. We developed a tool called DDInteract to enhance and support shared decision-making (SDM) between patients and physicians when both warfarin and NSAIDs are used concurrently. DDInteract was designed to be integrated into electronic health records using interoperability standards. OBJECTIVE: The purpose of this study was to conduct a formative evaluation of a DDInteract that incorporates patient and product contextual factors to estimate the risk of bleeding. METHODS: A randomized formative evaluation was conducted to compare DDInteract to usual care (UC) using physician-patient dyads. Using case vignettes, physicians and patients on warfarin participated in simulated virtual clinical encounters where they discussed the use of taking ibuprofen and warfarin concurrently and determined an appropriate therapeutic plan based on the patient's individualized risk. Dyads were randomized to either DDInteract or UC. Participants completed a postsession interview and survey of the SDM process. This included the 9-item Shared Decision-Making Questionnaire (SDM-Q-9), tool usability and workload National Aeronautics and Space Administration (NASA) Task Load Index, Unified Theory of Acceptance and Use of Technology (UTAUT), Perceived Behavioral Control (PBC) scale, System Usability Scale (SUS), and Decision Conflict Scale (DCS). They also were interviewed after the session to obtain perceptions on DDInteract and UC resources for DDIs. RESULTS: Twelve dyad encounters were performed using virtual software. Most (n=11, 91.7%) patients were over 50 years of age, and 9 (75%) had been taking warfarin for more than 2 years (75%). Regarding scores on the SDM-Q-9, participants rated DDInteract higher than UC for questions pertaining to helping patients clarify the decision (P=.03), involving patients in the decision (P=.01), displaying treatment options (P<.001), identifying advantages and disadvantages (P=.01), and facilitating patient understanding (P=.01) and discussion of preferences (P=.01). Five of the 8 UTAUT constructs showed differences between the 2 groups, favoring DDInteract (P<.05). Usability ratings from the SUS were significantly higher (P<.05) for physicians using DDInteract compared to those in the UC group but showed no differences from the patient's perspective. No differences in patient responses were observed between groups using the DCS. During the session debrief, physicians indicated little concern for the additional time or workload entailed by DDInteract use. Both clinicians and patients indicated that the tool was beneficial in simulated encounters to understand and mitigate the risk of harm from this DDI. CONCLUSIONS: Overall, DDInteract may improve encounters where there is a risk of bleeding due to a potential drug-drug interaction involving anticoagulants. Participants rated DDInteract as logical and useful for enhancing SDM. They reported that they would be willing to use the tool for an interaction involving warfarin and NSAIDs.

Cost-effectiveness of once-daily vs twice-daily tacrolimus among Hispanic and Black kidney transplant recipients.

BACKGROUND: Tacrolimus is a first-line immunosuppressive therapy to prevent rejection and graft failure in kidney transplant recipients. Once-daily extended-release tacrolimus tablets (LCPT) have been shown to be efficacious, particularly for Hispanic and Black patient subpopulations who are rapid metabolizers, but is more costly than twice-daily immediate-release tacrolimus (IR-Tac). OBJECTIVE: To evaluate the cost-effectiveness of LCPT during the first year of treatment vs IR-Tac in kidney transplant recipients who are Hispanic or Black. METHODS: A decision analytic model from a US payer perspective was developed using (1) subgroup outcomes data pooled from two phase 3 clinical trials that compared LCPT and IR-Tac, and (2) direct costs from real-world data sources (ie, costs of LCPT and IR-Tac treatments, biopsy-proven acute rejection, treatment-related serious adverse events [SAEs], graft failure, and consequent dialysis). The primary outcome was cost per successfully treated patient, defined as having a functioning graft after 1 year and without treatment-related SAEs. Probabilistic sensitivity analyses established distributions for cost and outcomes estimates, and a series of one-way sensitivity analyses identified parameters that had the most effect on results. RESULTS: Total overall cost for the Hispanic group was $14,765 for LCPT and $12,416 for IR-Tac, and total cost in the Black group was $16,626 for LCPT and $9,871 for IR-Tac. Total overall effectiveness of LCPT and IR-Tac was 88.32% and 84.75% in the Hispanic group and 93.24% and 85.78% in the Black group, respectively. The incremental cost-effectiveness ratio (ICER) for using LCPT over IR-Tac during the first year of treatment in the Hispanic group was $65,643 per additional successfully treated patient. The ICER for the Black group was $90,458. The single parameter having the most impact on results in both groups was the probability of a treatment-related SAE in IR-Tac, which accounted for 49% of variation in results in the Hispanic group and 46% in the Black group. CONCLUSIONS: Overall results for both groups show that LCPT is incrementally more costly and more effective compared with IR-Tac, indicating a trade-off scenario. LCPT is a cost-effective strategy if a decision makers' willingness to pay for 1 additional successfully treated patient exceeds the ICER and must be weighed against the costs of graft loss, continuing dialysis, and potential retransplant. This study provides a foundation for further research to update and expand inputs as more data become available to improve real-world relevance and decision making. DISCLOSURES: This study was funded by Veloxis Pharmaceuticals, Inc., which provided clinical trial file data and nonbinding feedback on the model structure, data interpretation, clinical expertise, manuscript review, and areas of publication interest (ie, managed care). Hurwitz, Grizzle, Villa Zapata, and Malone received grant funding from Veloxis Pharmaceuticals, Inc., through University of Arizona to conduct research and analysis for this study. Tyler is employed by Veloxis Pharmaceuticals, Inc. Some of the data reported and used in this research were available from the US Renal Data System, the US Bureau of Labor Statistics, and the Agency for Healthcare Research and Quality's Healthcare Cost and Utility Project. The interpretation and reporting of these data are the responsibility of the authors and in no way should be seen as an official policy or interpretation of the US government.

A Shared Decision-making Tool for Drug Interactions Between Warfarin and Nonsteroidal Anti-inflammatory Drugs: Design and Usability Study.

BACKGROUND: Exposure to life-threatening drug-drug interactions (DDIs) occurs despite the widespread use of clinical decision support. The DDI between warfarin and nonsteroidal anti-inflammatory drugs is common and potentially life-threatening. Patients can play a substantial role in preventing harm from DDIs; however, the current model for DDI decision-making is clinician centric. OBJECTIVE: This study aims to design and study the usability of DDInteract, a tool to support shared decision-making (SDM) between a patient and provider for the DDI between warfarin and nonsteroidal anti-inflammatory drugs. METHODS: We used an SDM framework and user-centered design methods to guide the design and usability of DDInteract-an SDM electronic health record app to prevent harm from clinically significant DDIs. The design involved iterative prototypes, qualitative feedback from stakeholders, and a heuristic evaluation. The usability evaluation included patients and clinicians. Patients participated in a simulated SDM discussion using clinical vignettes. Clinicians were asked to complete eight tasks using DDInteract and to assess the tool using a survey adapted from the System Usability Scale. RESULTS: The designed DDInteract prototype includes the following features: a patient-specific risk profile, dynamic risk icon array, patient education section, and treatment decision tree. A total of 4 patients and 11 clinicians participated in the usability study. After an SDM session where patients and clinicians review the tool concurrently, patients generally favored pain treatments with less risk of gastrointestinal bleeding. Clinicians successfully completed the tasks with a mean of 144 (SD 74) seconds and rated the usability of DDInteract as 4.32 (SD 0.52) of 5. CONCLUSIONS: This study expands the use of SDM to DDIs. The next steps are to determine if DDInteract can improve shared decision-making quality and to implement it across health systems using interoperable technology.

Is Real-World Evidence Used in P&T Monographs and Therapeutic Class Reviews?

BACKGROUND: Payers are faced with making coverage and reimbursement decisions based on the best available evidence. Often these decisions apply to patient populations, provider networks, and care settings not typically studied in clinical trials. Treatment effectiveness evidence is increasingly available from electronic health records, registries, and administrative claims. However, little is known about when and what types of real-world evidence (RWE) studies inform pharmacy and therapeutic (P&T) committee decisions. OBJECTIVE: To evaluate evidence sources cited in P&T committee monographs and therapeutic class reviews and assess the design features and quality of cited RWE studies. METHODS: A convenience sample of representatives from pharmacy benefit management, health system, and health plan organizations provided recent P&T monographs and therapeutic class reviews (or references from such documents). Two investigators examined and grouped references into major categories (published studies, unpublished studies, and other/unknown) and multiple subcategories (e.g., product label, clinical trials, RWE, systematic reviews). Cited comparative RWE was reviewed to assess design features (e.g., population, data source, comparators) and quality using the Good ReseArch for Comparative Effectiveness (GRACE) Checklist. RESULTS: Investigators evaluated 565 references cited in 27 monographs/therapeutic class reviews from 6 managed care organizations. Therapeutic class reviews mostly cited published clinical trials (35.3%, 155/439), while single-product monographs relied most on manufacturer-supplied information (42.1%, 53/126). Published RWE comprised 4.8% (21/439) of therapeutic class review references, and none (0/126) of the monograph references. Of the 21 RWE studies, 12 were comparative and assessed patient care settings and outcomes typically not included in clinical trials (community ambulatory settings [10], long-term safety [8]). RWE studies most frequently were based on registry data (6), conducted in the United States (6), and funded by the pharmaceutical industry (5). GRACE Checklist ratings suggested the data and methods of these comparative RWE studies were of high quality. CONCLUSIONS: RWE was infrequently cited in P&T materials, even among therapeutic class reviews where RWE is more readily available. Although few P&T materials cited RWE, the comparative RWE studies were generally high quality. More research is needed to understand when and what types of real-world studies can more routinely inform coverage and reimbursement decisions. DISCLOSURES: This project was funded by the National Pharmaceutical Council. Hurwitz, Brown, Peters, and Malone have nothing to disclose. Graff is employed by the National Pharmaceutical Council Part of this study was presented as a poster presentation at the AMCP Managed Care & Specialty Pharmacy 2016 Annual Meeting; April 19-22, 2016; San Francisco, CA. Study concept and design were primarily contributed by Malone and Graff, along with Hurwitz and Brown. All authors participated in data collection, and data interpretation was performed by Malone, Hurwitz, and Graff, with assistance from Brown and Peters. The manuscript was written primarily by Hurwitz and Malone, along with Graff, Brown, and Peters, and revised by Malone, Brown, Peters, Hurwitz, and Graff.

Real-World Evidence from the Integrative Medicine Primary Care Trial (IMPACT): Assessing Patient-Reported Outcomes at Baseline and 12-Month Follow-Up.

PURPOSE: The University of Arizona Integrative Health Center (UAIHC) was an innovative membership-supported integrative medicine (IM) adult primary care clinic in Phoenix, Arizona. UAIHC delivered healthcare using an integrative medicine model that combined conventional and complementary medical treatments, including nutrition, mind-body medicine, acupuncture, manual medicine, health coaching, educational classes, and groups. Results from pre-post evaluation of patient-reported outcomes on several standardized measures are presented here. METHODS: UAIHC patients completed surveys at baseline and after 12 months of continuous integrative primary care. Patients reported on perceived changes in health outcomes as measured by Short-Form Health Survey (SF-12 general, mental, and physical health), Perceived Stress Scale (PSS4), Work Productivity and Activity Impairment Questionnaire (WPAI), World Health Organization Well-Being Index (WHO-5), Pain Visual Analog Scale (VAS), Fatigue Severity Scale (VAS; FSS), Generalized Anxiety Disorder Scale (GAD2), Patient Health Questionnaire for depression (PHQ2), Pittsburgh Sleep Quality Index (PSQI) global rating of sleep quality, and the Behavioral Risk Factor Surveillance System (BRFSS; nutrition, exercise, and physical activity). Overall differences between time points were assessed for statistical significance. Patient demographics are also described. RESULTS: 177 patients completed baseline and follow-up outcome measures. Patients were predominantly white, female, college-educated, and employed. Baseline to one-year follow-up results indicate statistically significant improvements (p <.05) on all but perceived stress (PSS-4) and work absenteeism (WPAI). Clinical impact and/or practical effects are reported as percent change or standardized effect sizes whenever possible. Other demographic and descriptive information is summarized. CONCLUSIONS: Following one year of IM primary care at UAIHC, patient-reported outcomes indicated positive impacts in several areas of patients' lives: mental, physical, and overall health; work productivity; sleep quality; pain; fatigue; overall well-being; and physical activity.

Is Real-World Evidence Used in P&T Monographs and Therapeutic Class Reviews?

BACKGROUND: Payers are faced with making coverage and reimbursement decisions based on the best available evidence. Often these decisions apply to patient populations, provider networks, and care settings not typically studied in clinical trials. Treatment effectiveness evidence is increasingly available from electronic health records, registries, and administrative claims. However, little is known about when and what types of real-world evidence (RWE) studies inform pharmacy and therapeutic (P&T) committee decisions. OBJECTIVE: To evaluate evidence sources cited in P&T committee monographs and therapeutic class reviews and assess the design features and quality of cited RWE studies. METHODS: A convenience sample of representatives from pharmacy benefit management, health system, and health plan organizations provided recent P&T monographs and therapeutic class reviews (or references from such documents). Two investigators examined and grouped references into major categories (published studies, unpublished studies, and other/unknown) and multiple subcategories (e.g., product label, clinical trials, RWE, systematic reviews). Cited comparative RWE was reviewed to assess design features (e.g., population, data source, comparators) and quality using the Good ReseArch for Comparative Effectiveness (GRACE) Checklist. RESULTS: Investigators evaluated 565 references cited in 27 monographs/therapeutic class reviews from 6 managed care organizations. Therapeutic class reviews mostly cited published clinical trials (35.3%, 155/439), while single-product monographs relied most on manufacturer-supplied information (42.1%, 53/126). Published RWE comprised 4.8% (21/439) of therapeutic class review references, and none (0/126) of the monograph references. Of the 21 RWE studies, 12 were comparative and assessed patient care settings and outcomes typically not included in clinical trials (community ambulatory settings [10], long-term safety [8]). RWE studies most frequently were based on registry data (6), conducted in the United States (6), and funded by the pharmaceutical industry (5). GRACE Checklist ratings suggested the data and methods of these comparative RWE studies were of high quality. CONCLUSIONS: RWE was infrequently cited in P&T materials, even among therapeutic class reviews where RWE is more readily available. Although few P&T materials cited RWE, the comparative RWE studies were generally high quality. More research is needed to understand when and what types of real-world studies can more routinely inform coverage and reimbursement decisions. DISCLOSURES: This project was funded by the National Pharmaceutical Council. Hurwitz, Brown, Peters, and Malone have nothing to disclose. Graff is employed by the National Pharmaceutical Council Part of this study was presented as a poster presentation at the AMCP Managed Care & Specialty Pharmacy 2016 Annual Meeting; April 19-22, 2016; San Francisco, CA. Study concept and design were primarily contributed by Malone and Graff, along with Hurwitz and Brown. All authors participated in data collection, and data interpretation was performed by Malone, Hurwitz, and Graff, with assistance from Brown and Peters. The manuscript was written primarily by Hurwitz and Malone, along with Graff, Brown, and Peters, and revised by Malone, Brown, Peters, Hurwitz, and Graff.

Accepting Medication Therapy Management Recommendations to Add ACEIs or ARBs in Diabetes Care.

BACKGROUND: National guidelines and initiatives have promoted the use of angiotensin-converting enzyme inhibitors (ACEIs) and angiotensin receptor blockers (ARBs) for patients with diabetes. The University of Arizona Medication Management Center (UA-MMC) is contracted by Medicare health plans, pharmacy benefit managers (PBMs), and multiple commercial health insurance plans to provide medication therapy management (MTM) services for plan members. As part of the MTM program, recommendations have been made for those patients who may benefit from the addition of an ACEI/ARB. Although the intervention benefits and guidelines for using ACEIs/ARBs are clear, real-world evidence is needed to understand and potentially increase uptake of guideline interventions among eligible patients. OBJECTIVES: To (a) identify patient characteristics that predict acceptance of guideline recommendations to add ACEI/ARB medications to diabetic treatment via MTM services and (b) examine how well different case characteristics (i.e., patient age and sex, type and number of recommendation attempts, type of health care plan) predict the odds of adding ACEI/ARB medications to diabetic regimens when recommended through an MTM call center. METHODS: This was a retrospective analysis of secondary data provided by the UA-MMC. The de-identified national data included adult plan members with diabetes who the UA-MMC recommended adding an ACEI/ARB prescription based on 2012 national guidelines. The UA-MMC made recommendations by either patient letters, patient phone calls, physician faxes, or any combination thereof. We conducted a binary logistic regression analysis to assess the impact of case characteristics on the likelihood of accepting recommendations to add ACEI/ARB medications. The outcome variable was recommendation acceptance (yes/no), defined as new prescription claims for an ACEI/ARB within 120 days following the recommendation. Five predictor variables were assessed: (1) patient's age quartile; (2) method of communicating recommendations (letter, phone call, fax, or some combination thereof); (3) whether recommendations were made once or twice on separate dates; (4) patient's sex; and (5) type of health care plan. RESULTS: Recommendations were made for 31,495 members of health plans or PBMs that contracted with the UA-MMC. Patients' ages ranged from 19-90 (Mean =72.01; SD =10.21), with females comprising 56% of the sample. The recommendation to add ACEI/ARB medications was accepted for 14.5% (4,559) of patients. In most cases (73%), recommendations occurred via a letter to patients together with a fax to their providers. The fitted model, containing 3 predictor variables (age quartile, type of contact to communicate the recommendations, and whether recommendation contacts were made twice), was statistically significant, χ(2) (10; N = 31,495) = 112.82 (P < 0.001), indicating that the model was able to distinguish between those who did and did not accept UA-MMC's recommendations to add ACEI/ARB medications. The likelihood of recommendation acceptance decreased as patient age increased compared with patients in the first age quartile (ages 19-67; P ≤ 0.005 at all levels). Compared with sending only a provider fax, patients who received all 3 types of contact (provider fax with patient phone call and letter) were estimated to be 1.34 times more likely (34% increase) to have recommendation acceptance ( P = 0.004; 95% CI = 1.10-1.63). Similarly, patients who received only letters were also 1.32 times more likely (32% increase) than provider faxes alone to result in recommendation acceptance ( P = 0.003; 95% CI = 1.10-1.59). Patients for whom recommendations were made twice were less likely to have recommendation acceptance than for those contacted once, controlling for all other predictor variables in the model ( P < 0.001; OR = 0.77; 95% CI = 0.69-0.86). CONCLUSIONS: Recommendations to add an ACEI/ARB to diabetic regimens are more likely to be accepted for younger patients and those who receive recommendations through all 3 communication types (provider fax combined with patient phone call and letter) or just letters than provider faxes alone. Further research is needed to understand why prescribers are not accepting MTM recommendations.

Size and consistency of problem-solving consultation outcomes: an empirical analysis.

In this study, we analyzed extant data to evaluate the variability and magnitude of students' behavior change outcomes (academic, social, and behavioral) produced by consultants through problem-solving consultation with teachers. Research questions were twofold: (a) Do consultants produce consistent and sizeable positive student outcomes across their cases as measured through direct and frequent assessment? and (b) What proportion of variability in student outcomes is attributable to consultants? Analyses of extant data collected from problem-solving consultation outcome studies that used single-case, time-series AB designs with multiple participants were analyzed. Four such studies ultimately met the inclusion criteria for the extant data, comprising 124 consultants who worked with 302 school teachers regarding 453 individual students. Consultants constituted the independent variable, while the primary dependent variable was a descriptive effect size based on student behavior change as measured by (a) curriculum-based measures, (b) permanent products, or (c) direct observations. Primary analyses involved visual and statistical evaluation of effect size magnitude and variability observed within and between consultants and studies. Given the nested nature of the data, multilevel analyses were used to assess consultant effects on student outcomes. Results suggest that consultants consistently produced positive effect sizes on average across their cases, but outcomes varied between consultants. Findings also indicated that consultants, teachers, and the corresponding studies accounted for a significant proportion of variability in student outcomes. This investigation advances the use of multilevel and integrative data analyses to evaluate consultation outcomes and extends research on problem-solving consultation, consultant effects, and meta-analysis of case study AB designs. Practical implications for evaluating consultation service delivery in school settings are also discussed.