Comparison of EHR Data-Completeness in Patients with Different Types of Medical Insurance Coverage in the United States.

Prior studies have demonstrated that misclassification of study variables due to electronic health record (EHR)-discontinuity can be mitigated by restricting EHR-based analyses to subjects with high predicted EHR-continuity based on a simple algorithm. In this study, we compared EHR continuity in populations covered by Medicare, Medicaid, or commercial insurance. Using claims-linked EHRs from a multicenter network in Massachusetts, including Medicare (MA EHR-Medicare cohort) and Medicaid (MA EHR-Medicaid cohort) claims data; and TriNetX (TriNetX cohort) claims-linked EHR data from 11 US-based healthcare organizations, we assessed (1) EHR-continuity quantified by proportion of encounters captured by EHR (capture proportion (CP)); (2) area under receiver operating curve (AUROC) of previously validated model to identify patients with high EHR-continuity (CP ≥ 0.6); (3) misclassification of 40 patient characteristics, quantified by average standardized absolute mean difference (ASAMD). Study participants were ≥ 65 years (Medicare) or ≥ 18 years (Medicaid, TriNetX) with ≥ 365 days of continuous insurance enrollment overlapping with an EHR encounter. We found that the mean CP was 0.30, 0.18, and 0.19 and AUROC of the prediction model to identify patients with high EHR-continuity was 0.92, 0.89, and 0.77 in the MA EHR-Medicare, MA EHR-Medicaid, and TriNetX cohorts, respectively. Restricting to patients with predicted EHR-continuity percentile of top 20%, 50%, and 50% in MA EHR-Medicare, MA EHR-Medicaid, and TriNetX cohorts resulted in acceptable levels of misclassification (ASAMD < 0.1). Using a prediction model to identify cohorts with high EHR-continuity can improve validity, but cutoffs to achieve this goal vary by population.

The Impact of Longitudinal Data-Completeness of Electronic Health Record Data on the Prediction Performance of Clinical Risk Scores.

The impact of electronic health record (EHR) discontinuity (i.e., receiving care outside of a given EHR system) on EHR-based risk prediction is unknown. We aimed to assess the impact of EHR-continuity on the performance of clinical risk scores. The study cohort consisted of patients aged ≥ 65 years with ≥ 1 EHR encounter in the 2 networks in Massachusetts (MA; 2007/1/1-2017/12/31, internal training and validation dataset), and one network in North Carolina (NC; 2007/1/1-2016/12/31, external validation dataset) that were linked with Medicare claims data. Risk scores were calculated using EHR data alone vs. linked EHR-claims data (not subject to misclassification due to EHR-discontinuity): (i) combined comorbidity score (CCS), (ii) claim-based frailty score (CFI), (iii) CHAD2 DS2 -VASc, and (iv) Hypertension, Abnormal renal/liver function, Stroke, Bleeding, Labile, Elderly, and Drugs (HAS-BLED). We assessed the performance of CCS and CFI predicting death, CHAD2 DS2 -VASc predicting ischemic stroke, and HAS-BLED predicting bleeding by area under receiver operating characteristic curve (AUROC), stratified by quartiles of predicted EHR-continuity (Q1-4). There were 319,740 patients in the MA systems and 125,380 in the NC system. In the external validation dataset, AUROC for EHR-based CCS predicting 1-year risk of death was 0.583 in Q1 (lowest) EHR-continuity group, which increased to 0.739 in Q4 (highest) EHR-continuity group. The corresponding improvement in AUROC was 0.539 to 0.647 for CFI, 0.556 to 0.637 for CHAD2 DS2 -VASc, and 0.517 to 0.556 for HAS-BLED. The AUROC in Q4 EHR-continuity group based on EHR alone approximates that based on EHR-claims data. The prediction performance of four clinical risk scores was substantially worse in patients with lower vs. high EHR-continuity.

The Aetion Coalition to Advance Real-World Evidence through Randomized Controlled Trial Emulation Initiative: Oncology.

Legislative and technological advancements over the past decade have given rise to the proliferation of healthcare data generated from routine clinical practice, often referred to as real-world data (RWD). These data have piqued the interest of healthcare stakeholders due to their potential utility in generating evidence to support clinical and regulatory decision making. In the oncology setting, studies leveraging RWD offer distinct advantages that are complementary to randomized controlled trials (RCTs). They also permit the conduct of investigations that may not be possible through prospective designs due to ethics or feasibility. Despite its promise, the use of RWD for the generation of clinical evidence remains controversial due to concerns of unmeasured confounding and other sources of bias that must be carefully addressed in the study design and analysis. To facilitate a better understanding of when RWD can provide reliable conclusions on drug effectiveness, we seek to conduct 10 RWD-based studies that emulate RCTs in oncology using a systematic, protocol-driven approach described herein. Results of this investigation will help inform clinical, scientific, and regulatory stakeholders on the applications of RWD in the context of product labeling expansion, drug safety, and comparative effectiveness in oncology.

Effectiveness research in oncology with electronic health record data: A retrospective cohort study emulating the PALOMA-2 trial.

PURPOSE: Oncology electronic health record (EHR) databases have increased in quality and availability over the past decade, yet it remains unclear whether these clinical practice data can be used to conduct reliable comparative effectiveness studies. We sought to emulate a clinical trial with EHR data in the advanced breast cancer population and compare our results against the trial. METHODS: This cohort study used EHR data from US oncology practices. All elements of the study were defined to mimic the PALOMA-2 trial as closely as possible. Patients with hormone-positive, HER-2 negative metastatic breast cancer with no prior treatment for metastatic disease were included. Patients initiating palbociclib and letrozole on the same day following the earliest record of metastasis were compared to those initiating letrozole only. The primary associational measure was the conditional hazard ratio for time-to-next treatment (TTNT). TTNT is well-measured in our data source and amenable for calibration against the randomized study results of the PALOMA-2 trial. We used multiple imputation for several patient characteristics with missing values. RESULTS: There were 3836 study-eligible women with advanced breast cancer. The hazard ratio for TTNT in the observational study (HR: 0.62; 95% CI: 0.56-0.68) was closely aligned with that of the randomized trial (HR: 0.64; 95% CI: 0.52-0.78). CONCLUSIONS: Under our assumptions on missing data and comparability of the two study populations, results from our non-randomized study closely matched that of the randomized trial. Further studies are needed to determine whether EHR data can yield reliable conclusions on treatment effects in oncology.

Advancing an Algorithm for the Identification of Patients with High Data-Continuity in Electronic Health Records.

Background: Identifying high data-continuity patients in an electronic health record (EHR) system may facilitate selecting cohorts with a lower degree of variable misclassification and promote study validity. We updated a previously developed algorithm for identifying patients with high EHR data-completeness by adding demographic and health utilization factors to improve adaptability to networks serving patients of diverse backgrounds. We also expanded the algorithm to accommodate data in the ICD-10 era. Methods: We used Medicare claims linked with EHR data to identify individuals aged ≥65 years. EHR-continuity was defined as the proportion of encounters captured in EHR data relative to claims. We compared the model with additional demographic factors and their interaction terms with other predictors with the original algorithm and assessed the performance by area under the ROC curve (AUC) and net reclassification index (NRI). Results: The study cohort consisted of 264,099 subjects. The updated prediction model had an AUC of 0.93 in the validation set. Compared to the previous model, the new model had an NRI of 37.4% (p<0.001) for EHR-continuity classification. Interaction terms between demographic variables and other predictors did not improve the performance. Patients within the top 20% of predicted EHR-continuity had four times less misclassification of key variables compared to the remaining population. Conclusion: Adding demographic and healthcare utilization variables significantly improved the model performance. Patients with high predicted EHR-continuity had less misclassification of study variables compared to the remaining population in both ICD-9 and 10 eras.

Oncology Drug Effectiveness from Electronic Health Record Data Calibrated Against RCT Evidence: The PARSIFAL Trial Emulation.

Background: The use of electronic health records (EHR) data to assess drug effectiveness in clinical oncology practice is of great interest to regulators, clinicians, and payers. However, the utility of EHR data in clinical effectiveness studies may be limited by missing data, unmeasured confounding, and imperfect outcome surveillance. This study sought to emulate and compare the results of a randomized controlled trial investigating the efficacy of palbociclib with fulvestrant vs letrozole in advanced breast cancer. Methods: This was a cohort study using longitudinal EHR data derived from outpatient oncology practices in the United States. Eligibility criteria from the PARSIFAL trial were emulated as closely as possible. Patients were included if they had hormone-positive, human epidermal growth factor receptor - 2 (HER-2) negative metastatic breast cancer and had no record of prior treatment for metastatic disease. Patients initiating first-line treatment with palbociclib and fulvestrant following their first record of metastasis were compared to those initiating palbociclib and letrozole on the same day. Treatments were ascertained by oncology medication ordering records in the data source. The primary outcome was death as recorded in the oncologists' EHR systems. Results: There were 1886 eligible women in the study cohort. Although the 3-year survival was meaningfully lower in clinical practice (59%) compared to the randomized trial (78%), the relative effect size was a hazard ratio (HR) of 1.07 (95% CI: 0.86-1.35), similar to the randomized trial (HR = 1.00; 95% CI: 0.68-1.48). Conclusion: Despite common challenges encountered in EHR-based studies, it is possible to achieve similar conclusions to emulated randomized trials with the application of analytic approaches that address missing data, confounding, and selection bias. This is a promising finding in light of other emulations and ongoing efforts to improve data from clinical practice and causal analytics.

Real-World Evidence Prediction of a Phase IV Oncology Trial: Comparative Degarelix vs Leuprolide Safety.

BACKGROUND: Medical and regulatory communities are increasingly interested in the utility of real-world evidence (RWE) for answering questions pertaining to drug safety and effectiveness but concerns about validity remain. A principled approach to conducting RWE studies may alleviate concerns and increase confidence in findings. This study sought to predict the findings from the PRONOUNCE trial using a principled approach to generating RWE. METHODS: This propensity-score (PS) matched observational cohort study utilized 3 claims databases to compare the occurrence of major adverse cardiovascular events (MACE) among initiators of degarelix vs. leuprolide. Patients were included if they had history of prostate cancer and atherosclerotic cardiovascular disease. Subjects were excluded if they didn't have continuous database enrollment in the year prior to treatment initiation, were exposed to androgen deprivation therapy or experienced an acute cardiovascular event within 30 days prior to treatment initiation, or had a history or risk factors of QT prolongation. RESULTS: There were 12,448 leuprolide and 1,969 degarelix study-eligible patients before matching, with 1,887 in each arm after PS-matching. The results for MACE comparing degarelix to leuprolide in the observational analysis (hazard ratio= 1.35; 95% confidence interval = 0.94-1.93) was consistent with the subsequently released PRONOUNCE result (hazard ratio = 1.28; 95% confidence interval = 0.59-2.79). CONCLUSIONS: This study successfully predicted the result of a comparative cardiovascular safety trial in the oncology setting. Although the findings are encouraging, limitations of measuring cancer stage and tumor progression are representative of challenges in attempting to generalize whether claims-based RWE can be used as actionable evidence.

An algorithm to predict data completeness in oncology electronic medical records for comparative effectiveness research.

INTRODUCTION: Electronic health record (EHR) discontinuity (missing out-of-network encounters) can lead to information bias. We sought to construct an algorithm that identifies high EHR-continuity among oncology patients. METHODS: Using a linked Medicare-EHR database and regression, we sought to 1) measure how often Medicare claims for outpatient encounters were substantiated by visits recorded in the EHR, and 2) predict continuity ratio, defined as the yearly proportion of outpatient encounters reported to Medicare that were captured by EHR data. The prediction model...s performance was evaluated with the coefficient of determination and Spearman...s correlation. We quantified variable misclassification by decile of continuity ratio using standardized difference and sensitivity. RESULTS: A total of 79,678 subjects met all eligibility criteria. Predicted and observed continuity was highly correlated (σSpearman=0.86). On average across all variables measured, MSD was reduced by a factor of 1/7th and sensitivity was improved 35-fold comparing subjects in the highest vs. lowest decile of CR. CONCLUSION: In the oncology population, restricting EHR-based study cohorts to subjects with high continuity may reduce misclassification without greatly impacting representativeness. Further work is needed to elucidate the best manner of implementing continuity prediction rules in cohort studies.

Outpatient Reduction Mammaplasty Offers Significantly Lower Costs with Comparable Outcomes: A Propensity Score-Matched Analysis of 18,780 Cases.

BACKGROUND: Breast reduction mammaplasty is a common plastic surgery operation. Although many contemporary surgeons provide breast reduction mammaplasty as an outpatient procedure, roughly 15 percent of patients are still observed postoperatively. The authors hypothesize that observation confers no safety benefit but engenders significant cost. METHODS: The authors reviewed cases of breast reduction mammaplasty in a commercial database and formulated three propensity score-matched cohorts: inpatient, 23-hour observation, and outpatient. Comparisons were made between inpatients and outpatients and between 23-hour observation patients and outpatients. The primary outcome variable was 14-day re-presentation rate to the emergency department or readmission. Financial data were also collected. RESULTS: Comparison of inpatients and outpatients included 1237 patients each (n = 2474 total patients). The 23-hour observation-outpatient comparison included 8153 patients each (n = 16,306 total patients). For inpatients versus outpatients, the 14-day re-presentation rate was 1.4 percent for inpatients and 0.3 percent for outpatients (p < 0.01). The overall surgical complication rate was higher for inpatients (7.8 percent) than for outpatients (4.9 percent) (p < 0.01). Comparing outpatients to 23-hour observation patients, the 14-day re-presentation rate was similar (0.5 percent observation versus 0.3 percent outpatient; p = 0.10). The complication rate was higher for 23-hour observation patients (4.8 percent) than for outpatients (3.2 percent) (p < 0.01). When compared with outpatients (median, $9077), inpatients (median, $19,975) generated $10,898 more in costs. Similarly, 23-hour observation patients (median, $12,451) generated $4050 more in costs than outpatients (median, $8401) (p < 0.01). CONCLUSIONS: Outpatient breast reduction mammaplasty is equally safe when compared to observation or admission. Non-outpatient breast reduction mammaplasty had median costs of 148 to 220 percent that of outpatient breast reduction mammaplasty. In an era of cost consciousness, ambulatory reduction mammaplasty may offer a relatively simple method of decreasing expenditures. CLINICAL QUESTION/LEVEL OF EVIDENCE: Therapeutic, III.

Initial Opioid Prescriptions Predict Continued Narcotic Use: Analysis of 24,594 Reduction Mammaplasty Patients.

BACKGROUND: The objective of this study was to establish whether a particular perioperative opioid regimen was associated with a higher risk of opioid refilling in the reduction mammaplasty patient population. METHODS: A retrospective cohort study was conducted on subjects that underwent bilateral reduction mammaplasty with no history of opioid use in the year before surgery. Patients were followed for a period of 4 months after surgery. Multivariable logistic regression was used to establish factors associated with prescription refilling. Predictive probabilities of opioid refilling, given various perioperative opioid regimens, were explored. RESULTS: A total of 24,594 subjects met all criteria for inclusion in the study, 13 percent of whom demonstrated continued opioid use following surgery. The probability of refill in patients receiving 15 to 59 daily morphine milligram equivalents perioperatively was significantly lower than in those receiving less than 15 daily morphine milligram equivalents (e.g., 15 mg of codeine every 4 hours) or more than 60 daily morphine milligram equivalents (e.g., 10 mg of oxycodone every 6 hours). CONCLUSIONS: The probability of refilling prescription opioids is increased by nonmodifiable risk factors, such as age. However, modifiable risk factors exist as well, most notably the amount of narcotic prescribed during the perioperative period. Overprescribing was found to be common in the reduction mammaplasty patient population. Limiting outpatient opioid exposure to a range of 15 to 59 daily morphine milligram equivalents (e.g., 5 mg of hydrocodone every 6 hours) during the perisurgical period may reduce the probability that the surgeon will need to prescribe further narcotics. Further studies are needed to validate our findings. CLINICAL QUESTION/LEVEL OF EVIDENCE: Risk, III.

The Effect of Performance Transparency on Adherence to Barcode Scanning During Order Preparation in an Adult Inpatient Satellite Pharmacy.

Purpose: The aim of this study was to investigate the effect of performance transparency and individualized feedback on pharmacy technician compliance with barcode verification technology during inpatient order preparation. Methods: Following the incorporation of barcode scanning technology into the workflow of pharmacy staff, a multiphasic intervention was employed to promote its use. The intervention included verbal feedback and publically posting performance metrics to increase accountability. An interrupted time-series analysis was conducted to ascertain trends and levels in the percent of orders that were dispensed using barcode verification, before and after the study intervention. Analyses were conducted by shift and overall for pharmacy workers in a single satellite pharmacy. Results: A significant increase in percent scanned orders was observed immediately following the intervention in our analysis of all pharmacy workers (+14.4%; P = .045; 95% confidence interval [CI]: 0.35-28.4). In the analysis of each shift, statistically significant increases in percent scanned orders were observed immediately following the intervention for both the evening shift (+5.1%; P = .024; 95% CI: 0.70-9.6) and the night shift (+29.9%; P = .025; 95% CI: 3.9-55.9), but not the day shift (+2.6; P = .707; 95% CI: -11.1 to 16.2). Conclusion: Increasing transparency of individual and team performance metrics in conjunction with targeted feedback is an effective intervention to improve compliance with barcode scanning technology.

Current Practices of Intraventricular Antibiotic Therapy in the Treatment of Meningitis and Ventriculitis: Results from a Multicenter Retrospective Cohort Study.

BACKGROUND: Central nervous system (CNS) infections are particularly prevalent in the adult neurocritical care patient population and are associated with significant morbidity and mortality. Factors relevant to the nature of CNS infections pose significant challenges to clinicians treating afflicted patients. Intraventricular (IVT) administration of antibiotics may offer several benefits over systemic therapy; however, the outcomes and current practices of such treatments are poorly described in the literature. OBJECTIVE: To describe current practices and outcomes of patients receiving intraventricular antibiotic treatment for CNS infections in neurological intensive care units of academic medical centers nationwide. METHODS: A retrospective cohort study was conducted on patients admitted to intensive care units who received IVT antibiotic treatment at participating centers in the USA between January 01, 2003, and December 31, 2013. Clinical and laboratory parameters, microbiology, surgical and antimicrobial management, and treatment outcomes were collected and described. RESULTS: Of the 105 patients included, all received systemic antimicrobial therapy along with at least one dose of IVT antimicrobial agents. Intraventricular vancomycin was used in 52.4% of patients. The average dose was 12.2 mg/day for a median duration of 5 days. Intraventricular aminoglycosides were used in 47.5% of the patients, either alone or in combination with IVT vancomycin. The average dose of gentamicin/tobramycin was 6.7 mg/day with a median duration of 6 days. Overall mortality was 18.1%. Cerebrospinal fluid (CSF) culture sterilization occurred in 88.4% of the patients with a rate of recurrence or persistence of positive cultures of 9.5%. CONCLUSION: Intraventricular antimicrobial agents resulted in a high CSF sterilization rate. Contemporary use of this route typically results in a treatment duration of less than a week. Prospective studies are needed to establish the optimal patient population, as well as the efficacy and safety of this route of administration.

Costs Associated with Productivity Loss Among U.S. Patients Newly Diagnosed with Multiple Myeloma Receiving Oral Versus Injectable Chemotherapy.

BACKGROUND: The use of novel drug agents in the treatment of multiple myeloma (MM) has been associated with improved therapeutic outcomes and survival; however, MM continues to pose a significant economic burden on patients and health care systems. Evaluating economic implications of therapies can provide key points of distinctions between available treatment options. Patients with MM may experience productivity loss, including lost days from work or inability to work due to MM symptoms or to undergoing treatment. Although direct costs of illness have been well described in the literature, indirect costs associated with MM are understudied. OBJECTIVE: To compare the extent of disability benefit use and resultant workplace productivity loss among U.S. adult patients with newly diagnosed MM who received oral versus injectable MM therapy. METHODS: A retrospective cohort study was conducted using the Truven Health Analytics MarketScan Commercial Claims and Encounters, Medicare Supplemental Coordination of Benefits, and Health and Productivity Management databases (2008-2015). Workplace absenteeism, as measured by disability benefit use, was evaluated 1 year before and 1 year after first MM diagnosis. Patients receiving only oral chemotherapy were compared with those who received injectable therapy. Absenteeism days and associated costs were compared among study groups using multivariable zero-inflated Poisson regression. RESULTS: The final study cohort included 299 patients with newly diagnosed MM, of whom 73 received oral therapy only and 226 received injectable therapy. Treatment type was a significant predictor of disability benefit use. Patients who received injectable therapy missed an average of 110 work days in the 1 year after diagnosis, compared with 87 for patients receiving only oral therapy (difference of 23 days, 95% CI = 19-26, P < 0.001). Treatment type was also a significant predictor of costs associated with lost productivity. Patients who received injectable therapy experienced productivity loss valued at $18,315, compared with patients who only received oral drug therapy ($14,429). The difference between these estimates was statistically significant ($3,886, 95% CI = $3,540-$4,231, P < 0.001). CONCLUSIONS: Patients newly diagnosed with MM face significant losses in productivity. Patients receiving injectable MM therapy use significantly more disability benefits and incur higher productivity costs, compared with those receiving oral MM therapy. Further studies elucidating the nature of the differences between injectable and noninjectable chemotherapy users are needed. DISCLOSURES: This study was funded by Millennium Pharmaceuticals, a wholly owned subsidiary of Takeda Pharmaceutical Company. Yong and Noga are employees of Millennium Pharmaceuticals. Merola reports personal fees from Millennium Pharmaceuticals during the time of this study.

Impact of Displaying Inpatient Pharmaceutical Costs at the Time of Order Entry: Lessons From a Tertiary Care Center.

BACKGROUND: A lack of cost-conscious medication use is a major contributor to excessive healthcare expenditures in the inpatient setting. Expensive medicines are often utilized when there are comparable alternatives available at a lower cost. Increasing prescriber awareness of medication cost at the time of ordering may help promote cost-conscious use of medications in the hospital. OBJECTIVE: To evaluate the impact of cost messaging on the ordering of 9 expensive medications. DESIGN: Retrospective analysis of an institutional cost-transparency initiative. SETTING: A 1145-bed, tertiary care, academic medical center. PARTICIPANTS: Prescribers who ordered medications through the computerized provider order entry system at the Johns Hopkins Hospital. METHODS: Interrupted time series and segmented regression models were used to examine prescriber ordering before and after implementation of cost messaging for 9 highcost medications. RESULTS: Following the implementation of cost messaging, no significant changes were observed in the number of orders or ordering trends for intravenous (IV) formulations of eculizumab, calcitonin, levetiracetam, linezolid, mycophenolate, ribavirin, and levothyroxine. An immediate and sustained reduction in medication utilization was seen in 2 drugs that underwent a policy change during our study, IV pantoprazole and oral voriconazole. IV pantoprazole became restricted at our facility due to a national shortage (-985 orders per 10,000 patient days; 𝑃 < 0.001), and oral voriconazole was replaced with an alternative antifungal in oncology order sets (-110 orders per 10,000 patient days; 𝑃 = 0.001). CONCLUSIONS: Prescriber cost transparency alone did not significantly influence medication utilization at our institution. Active strategies to reduce ordering resulted in dramatic reductions in ordering.