Validation of an international classification of disease, tenth revision, clinical modification (ICD-10-CM) algorithm in identifying severe hypoglycaemia events for real-world studies.

AIM: The transition to the ICD-10-CM coding system has reduced the utility of hypoglycaemia algorithms based on ICD-9-CM diagnosis codes in real-world studies of antidiabetic drugs. We mapped a validated ICD-9-CM hypoglycaemia algorithm to ICD-10-CM codes to create an ICD-10-CM hypoglycaemia algorithm and assessed its performance in identifying severe hypoglycaemia. MATERIALS AND METHODS: We assembled a cohort of Medicare patients with DM and linked electronic health record (EHR) data to the University of North Carolina Health System and identified candidate severe hypoglycaemia events from their Medicare claims using the ICD-10-CM hypoglycaemia algorithm. We confirmed severe hypoglycaemia by EHR review and computed a positive predictive value (PPV) of the algorithm to assess its performance. We refined the algorithm by removing poor performing codes (PPV ≤0.5) and computed a Cohen's κ statistic to evaluate the agreement of the EHR reviews. RESULTS: The algorithm identified 642 candidate severe hypoglycaemia events, and we confirmed 455 as true severe hypoglycaemia events, PPV of 0.709 (95% confidence interval: 0.672, 0.744). When we refined the algorithm, the PPV increased to 0.893 (0.862, 0.918) and missed <2.42% (<11) true severe hypoglycaemia events. Agreement between reviewers was high, κ = 0.93 (0.89, 0.97). CONCLUSIONS: We translated an ICD-9-CM hypoglycaemia algorithm to an ICD-10-CM version and found its performance was modest. The performance of the algorithm improved by removing poor performing codes at the trade-off of missing very few severe hypoglycaemia events. The algorithm has the potential to be used to identify severe hypoglycaemia in real-world studies of antidiabetic drugs.

Combining meta-analysis with multiple imputation for one-step, privacy-protecting estimation of causal treatment effects in multi-site studies.

Missing data complicates statistical analyses in multi-site studies, especially when it is not feasible to centrally pool individual-level data across sites. We combined meta-analysis with within-site multiple imputation for one-step estimation of the average causal effect (ACE) of a target population comprised of all individuals from all data-contributing sites within a multi-site distributed data network, without the need for sharing individual-level data to handle missing data. We considered two orders of combination and three choices of weights for meta-analysis, resulting in six approaches. The first three approaches, denoted as RR + metaF, RR + metaR and RR + std, first combined results from imputed data sets within each site using Rubin's rules and then meta-analyzed the combined results across sites using fixed-effect, random-effects and sample-standardization weights, respectively. The last three approaches, denoted as metaF + RR, metaR + RR and std + RR, first meta-analyzed results across sites separately for each imputation and then combined the meta-analysis results using Rubin's rules. Simulation results confirmed very good performance of RR + std and std + RR under various missing completely at random and missing at random settings. A direct application of the inverse-variance weighted meta-analysis based on site-specific ACEs can lead to biased results for the targeted network-wide ACE in the presence of treatment effect heterogeneity by site, demonstrating the need to clearly specify the target population and estimand and properly account for potential site heterogeneity in meta-analyses seeking to draw causal interpretations. An illustration using a large administrative claims database is presented.

Automatable Distributed Regression Analysis of Vertically Partitioned Data Facilitated by PopMedNet: Feasibility and Enhancement Study.

BACKGROUND: In clinical research, important variables may be collected from multiple data sources. Physical pooling of patient-level data from multiple sources often raises several challenges, including proper protection of patient privacy and proprietary interests. We previously developed an SAS-based package to perform distributed regression-a suite of privacy-protecting methods that perform multivariable-adjusted regression analysis using only summary-level information-with horizontally partitioned data, a setting where distinct cohorts of patients are available from different data sources. We integrated the package with PopMedNet, an open-source file transfer software, to facilitate secure file transfer between the analysis center and the data-contributing sites. The feasibility of using PopMedNet to facilitate distributed regression analysis (DRA) with vertically partitioned data, a setting where the data attributes from a cohort of patients are available from different data sources, was unknown. OBJECTIVE: The objective of the study was to describe the feasibility of using PopMedNet and enhancements to PopMedNet to facilitate automatable vertical DRA (vDRA) in real-world settings. METHODS: We gathered the statistical and informatic requirements of using PopMedNet to facilitate automatable vDRA. We enhanced PopMedNet based on these requirements to improve its technical capability to support vDRA. RESULTS: PopMedNet can enable automatable vDRA. We identified and implemented two enhancements to PopMedNet that improved its technical capability to perform automatable vDRA in real-world settings. The first was the ability to simultaneously upload and download multiple files, and the second was the ability to directly transfer summary-level information between the data-contributing sites without a third-party analysis center. CONCLUSIONS: PopMedNet can be used to facilitate automatable vDRA to protect patient privacy and support clinical research in real-world settings.

Distributed Regression Analysis Application in Large Distributed Data Networks: Analysis of Precision and Operational Performance.

BACKGROUND: A distributed data network approach combined with distributed regression analysis (DRA) can reduce the risk of disclosing sensitive individual and institutional information in multicenter studies. However, software that facilitates large-scale and efficient implementation of DRA is limited. OBJECTIVE: This study aimed to assess the precision and operational performance of a DRA application comprising a SAS-based DRA package and a file transfer workflow developed within the open-source distributed networking software PopMedNet in a horizontally partitioned distributed data network. METHODS: We executed the SAS-based DRA package to perform distributed linear, logistic, and Cox proportional hazards regression analysis on a real-world test case with 3 data partners. We used PopMedNet to iteratively and automatically transfer highly summarized information between the data partners and the analysis center. We compared the DRA results with the results from standard SAS procedures executed on the pooled individual-level dataset to evaluate the precision of the SAS-based DRA package. We computed the execution time of each step in the workflow to evaluate the operational performance of the PopMedNet-driven file transfer workflow. RESULTS: All DRA results were precise (<10-12), and DRA model fit curves were identical or similar to those obtained from the corresponding pooled individual-level data analyses. All regression models required less than 20 min for full end-to-end execution. CONCLUSIONS: We integrated a SAS-based DRA package with PopMedNet and successfully tested the new capability within an active distributed data network. The study demonstrated the validity and feasibility of using DRA to enable more privacy-protecting analysis in multicenter studies.

Identification of potential drug name confusion errors in the Sentinel System.

PURPOSE: In July 2015, the US Food and Drug Administration (FDA) published a drug safety communication regarding errors in prescribing and dispensing of the antidepressant Brintellix (vortioxetine) and the antiplatelet Brilinta (ticagrelor) that arose due to proprietary drug name confusion. Brintellix is indicated for major depressive disorder; Brilinta is indicated to reduce cardiovascular death, myocardial infarction, and stroke in patients with acute coronary syndrome or history of myocardial infarction. Brintellix was renamed to Trintellix in May 2016. Using Brilinta and Brintellix as a proof-of-concept feasibility use case, we assessed whether drug name confusion errors between the pair could be identified in electronic health care data via the combination of a claims-based algorithm and limited manual claims data review. METHODS: Using data from the Sentinel System, we defined potential errors as Brintellix users without an on- or off-label indication for Brintellix, without a dispensing for a drug with the same indications as Brintellix, and with an on- or off-label indication for Brilinta between -365 and +30 days after index Brintellix dispensing; the reverse was done for Brilinta. We manually reviewed claims profiles of potential cases. RESULTS: We identified 27 (0.1%) potential errors among 21 208 Brintellix users; 16 appeared to be likely errors based on claims profile review. Fifty-one (0.3%) of the 16 779 Brilinta users were identified as potential errors, and four appeared to be likely errors. CONCLUSIONS: A claims-based algorithm combined with manual review of claims profiles could identify potential drug name confusion errors, and narrow down likely errors that warrant further investigation.

Sound-Alike Look-Alike Confusion and Matching Medication Product Attributes: Simulated Case-Control Studies.

Background: Sound-alike look-alike (SALA) medication confusions harm as many as 250 000 Americans annually. Most preventive strategies focus on medication name similarities. Objectives: To evaluate the association between matching medication product attributes and SALA confusion. Methods: We simulated 20 000 case-control studies using the Institute for Safe Medication Practices' List of Confused Drug Names as case pairs and 4 randomly selected control pairs per case pair from the First DataBank MedKnowledge (FDBM) database. We extracted 7 product attributes for each medication from the FDBM database. We used logistic regression models to estimate the associations between matching product attributes and SALA confusion. The models included a series of univariate (unadjusted) models, a model that adjusted for all product attributes, and a model that further adjusted for medication name similarity measures. Results: Medications with a matching product attribute had increased odds of SALA confusion in the univariate analyses (odds ratio [OR] = 4.2 to 55.5). When we simultaneously adjusted for all attributes, the associations of matching package unit, package unit size, formulation, strength, therapeutic class, and manufacturer with SALA confusion were attenuated but remained elevated (OR = 1.5 to 26.5), whereas the direction of association between matching route and SALA confusion reversed (OR = 0.8). These associations persisted on adjustment for medication name similarity measures. Conclusions and Relevance: This study is the first to evaluate the association between matching medication product attributes and SALA confusion with a control group. Having matching product attributes increased the odds of SALA confusion. SALA risk reduction strategies should consider product attributes.

A Query Workflow Design to Perform Automatable Distributed Regression Analysis in Large Distributed Data Networks.

INTRODUCTION: Patient privacy and data security concerns often limit the feasibility of pooling patient-level data from multiple sources for analysis. Distributed data networks (DDNs) that employ privacy-protecting analytical methods, such as distributed regression analysis (DRA), can mitigate these concerns. However, DRA is not routinely implemented in large DDNs. OBJECTIVE: We describe the design and implementation of a process framework and query workflow that allow automatable DRA in real-world DDNs that use PopMedNet™, an open-source distributed networking software platform. METHODS: We surveyed and catalogued existing hardware and software configurations at all data partners in the Sentinel System, a PopMedNet-driven DDN. Key guiding principles for the design included minimal disruptions to the current PopMedNet query workflow and minimal modifications to data partners' hardware configurations and software requirements. RESULTS: We developed and implemented a three-step process framework and PopMedNet query workflow that enables automatable DRA: 1) assembling a de-identified patient-level dataset at each data partner, 2) distributing a DRA package to data partners for local iterative analysis, and 3) iteratively transferring intermediate files between data partners and analysis center. The DRA query workflow is agnostic to statistical software, accommodates different regression models, and allows different levels of user-specified automation. DISCUSSION: The process framework can be generalized to and the query workflow can be adopted by other PopMedNet-based DDNs. CONCLUSION: DRA has great potential to change the paradigm of data analysis in DDNs. Successful implementation of DRA in Sentinel will facilitate adoption of the analytic approach in other DDNs.

Early impact of the ICD-10-CM transition on selected health outcomes in 13 electronic health care databases in the United States.

PURPOSE: To describe the consistency in the frequency of 5 health outcomes across the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) and Tenth Revision, Clinical Modification (ICD-10-CM) eras in the US. METHODS: We examined the incidence of 3 acute conditions (acute myocardial infarction [AMI], angioedema, ischemic stroke) and the prevalence of 2 chronic conditions (diabetes, hypertension) during the final 5 years of the ICD-9-CM era (January 2010-September 2015) and the first 15 months of the ICD-10-CM era (October 2015-December 2016) in 13 electronic health care databases in the Sentinel System. For each health outcome reviewed during the ICD-10-CM era, we evaluated 4 definitions, including published algorithms derived from other countries, as well as simple-forward, simple-backward, and forward-backward mapping using the General Equivalence Mappings. For acute conditions, we also compared the incidence between April to December 2014 and April to December 2016. RESULTS: The analyses included data from approximately 172 million health plan members. While the incidence or prevalence of AMI and hypertension performed similarly across the 2 eras, the other 3 outcomes did not demonstrate consistent trends for some or all the ICD-10-CM definitions assessed. CONCLUSIONS: When using data from both the ICD-9-CM and ICD-10-CM eras, or when using results from ICD-10-CM data to compare to results from ICD-9-CM data, researchers should test multiple ICD-10-CM outcome definitions as part of sensitivity analysis. Ongoing assessment of the impact of ICD-10-CM transition on identification of health outcomes in US electronic health care databases should occur as more data accrue.

Assessing the impact of the new ICD-10-CM coding system on pharmacoepidemiologic studies-An application to the known association between angiotensin-converting enzyme inhibitors and angioedema.

PURPOSE: To replicate the well-established association between angiotensin-converting enzyme inhibitors versus beta blockers and angioedema in the International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) era. METHODS: We conducted a retrospective, inception cohort study in a large insurance database formatted to the Sentinel Common Data Model. We defined study periods spanning the ICD-9-CM era only, ICD-10-CM era only, and ICD-9-CM and ICD-10-CM era and conducted simple-forward mapping (SFM), simple-backward mapping (SBM), and forward-backward mapping (FBM) referencing the General Equivalence Mappings to translate the outcome (angioedema) and covariates from ICD-9-CM to ICD-10-CM. We performed propensity score (PS)-matched and PS-stratified Cox proportional hazards regression to estimate hazard ratios (HRs) and 95% confidence intervals (CIs). RESULTS: In the ICD-9-CM and ICD-10-CM eras spanning April 1 to September 30 of 2015 and 2016, there were 152 017 and 145 232 angiotensin-converting enzyme inhibitor initiators and 115 073 and 116 652 beta-blocker initiators, respectively. The PS-matched HR was 4.19 (95% CI, 2.82-6.23) in the ICD-9-CM era, 4.37 (2.92-6.52) in the ICD-10-CM era using SFM, and 4.64 (3.05-7.07) in the ICD-10-CM era using SBM and FBM. The PS-matched HRs from the mixed ICD-9-CM and ICD-10-CM eras ranged from 3.91 (2.69-5.68) to 4.35 (3.33-5.70). CONCLUSION: The adjusted HRs across different diagnostic coding eras and the use of SFM versus SBM and FBM produced numerically different but clinically similar results. Additional investigations as ICD-10-CM data accumulate are warranted.

Medication-related clinical decision support alert overrides in inpatients.

Objective: To define the types and numbers of inpatient clinical decision support alerts, measure the frequency with which they are overridden, and describe providers' reasons for overriding them and the appropriateness of those reasons. Materials and Methods: We conducted a cross-sectional study of medication-related clinical decision support alerts over a 3-year period at a 793-bed tertiary-care teaching institution. We measured the rate of alert overrides, the rate of overrides by alert type, the reasons cited for overrides, and the appropriateness of those reasons. Results: Overall, 73.3% of patient allergy, drug-drug interaction, and duplicate drug alerts were overridden, though the rate of overrides varied by alert type (P < .0001). About 60% of overrides were appropriate, and that proportion also varied by alert type (P < .0001). Few overrides of renal- (2.2%) or age-based (26.4%) medication substitutions were appropriate, while most duplicate drug (98%), patient allergy (96.5%), and formulary substitution (82.5%) alerts were appropriate. Discussion: Despite warnings of potential significant harm, certain categories of alert overrides were inappropriate >75% of the time. The vast majority of duplicate drug, patient allergy, and formulary substitution alerts were appropriate, suggesting that these categories of alerts might be good targets for refinement to reduce alert fatigue. Conclusion: Almost three-quarters of alerts were overridden, and 40% of the overrides were not appropriate. Future research should optimize alert types and frequencies to increase their clinical relevance, reducing alert fatigue so that important alerts are not inappropriately overridden.

Adaptation and Validation of the Combined Comorbidity Score for ICD-10-CM.

BACKGROUND: The combined comorbidity score, which merges the Charlson and Elixhauser comorbidity indices, uses the ninth revision of the International Classification of Diseases, Clinical Modification (ICD-9-CM). In October 2015, the United States adopted the 10th revision (ICD-10-CM). OBJECTIVE: The objective of this study is to examine different coding algorithms for the ICD-10-CM combined comorbidity score and compare their performance to the original ICD-9-CM score. METHODS: Four ICD-10-CM coding algorithms were defined: 2 using General Equivalence Mappings (GEMs), one based on ICD-10-CA (Canadian modification) codes for Charlson and Elixhauser measures, and one including codes from all 3 algorithms. We used claims data from the Clinfomatics Data Mart to identify 2 cohorts. The ICD-10-CM cohort comprised patients who had a hospitalization between January 1, 2016 and March 1, 2016. The ICD-9-CM cohort comprised patients who had a hospitalization between January 1, 2015 and March 1, 2015. We used logistic regression models to predict 30-day hospital readmission for the original score in the ICD-9-CM cohort and for each ICD-10-CM algorithm in the ICD-10-CM cohort. RESULTS: Distributions of each version of the score were similar. The algorithm based on ICD-10-CA codes [c-statistic, 0.646; 95% confidence interval (CI), 0.640-0.653] had the most similar discrimination for readmission to the ICD-9-CM version (c, 0.646; 95% CI, 0.639-0.653), but combining all identified ICD-10-CM codes had the highest c-statistic (c, 0.651; 95% CI, 0.644-0.657). CONCLUSIONS: We propose an ICD-10-CM version of the combined comorbidity score that includes codes identified by ICD-10-CA and GEMs. Compared with the original score, it has similar performance in predicting readmission in a population of United States commercially insured individuals.

Review of Nonformulary Medication Approvals in an Academic Medical Center.

BACKGROUND: The Joint Commission requires hospitals to formally review formulary medications at least annually based on new clinical information. Although review of nonformulary medication (NFM) use is not required, frequent and inappropriate use of NFMs has the potential to increase hospital costs, negatively affect quality of care, and increase medication errors. Limited resources may restrict an institution's ability to review NFM use in addition to the required annual formulary review. NFM use at Brigham and Women's Hospital (BWH) was reviewed to provide insight on how to best direct an NFM review that is both effective and efficient. How an NFM review may negatively affect cost, quality of care, and medication errors is also inferred. METHODS: All approved NFM requests between 2009 and 2012 from Brigham and Women's Hospital's computerized provider order entry system were extracted and categorized according to the American Hospital Formulary Service (AHFS) Pharmacologic-Therapeutic Classification System. RESULTS: Of the 15,356,016 new medication orders, there were 223,266 NFM approvals for 433 unique NFMs. NFMs were categorized into 91 AHFS, 14 combination, and 4 "Others" classes. Twenty-five AHFS Classes accounted for approximately the top 90% of all NFM approvals, and the top 2 NFMs in each class accounted for a majority of the NFM approvals. CONCLUSION: Only a few classes of medications and a few medications within each class accounted for most of the NFM use at BWH. Targeting review of the most frequently used NFMs in each class may be a feasible strategy to reviewing NFMs annually that is both effective and efficient in optimizing formulary benefits.

A cross-sectional observational study of high override rates of drug allergy alerts in inpatient and outpatient settings, and opportunities for improvement.

OBJECTIVES: To evaluate how often and why providers overrode drug allergy alerts in both the inpatient and outpatient settings. DESIGN: A cross-sectional observational study of drug allergy alerts generated over a 3-year period between 1 January 2009 and 31 December 2011. SETTING: A 793-bed tertiary care teaching affiliate of Harvard Medical School and 36 primary care practices. PARTICIPANTS: Drug allergy alerts were displayed for a total of 29 420 patients across both settings. MAIN OUTCOME MEASURES: Proportion of drug allergy alerts displayed and overridden, proportion of appropriate overrides, proportion of overrides in each medication class, different reasons for overriding and types of reactions overridden. RESULTS: A total of 158 023 drug allergy alerts were displayed, 131 615 (83%) in the inpatient setting and 26 408 (17%) in the outpatient setting; 128 157 (81%) of which were overridden. A random sample of inpatient (n=200, 0.19%) and outpatient (n=50, 0.25%) alert overrides were screened for appropriateness, with >96% considered appropriate. Alerts for some drug classes, such as 'non-antibiotic sulfonamides', were overridden for >81% of prescriptions in both settings. The most common override reason was patient has taken previously without allergic reaction. In the inpatient setting alone, 70.9% of alerts that warned against the risk of anaphylaxis were overridden. CONCLUSIONS: The information contained in patients' drug allergy lists needs to be regularly updated. Most of the drug allergy alerts were overridden, with the majority of alert overrides in the subsample considered appropriate. Some of the rules for these alerts should be carefully reviewed and modified, or removed. Further research is needed to understand providers' overriding of alerts that warned against the risk of 'anaphylaxis', which are more concerning with respect to patient safety.

The frequency of inappropriate nonformulary medication alert overrides in the inpatient setting.

BACKGROUND: Experts suggest that formulary alerts at the time of medication order entry are the most effective form of clinical decision support to automate formulary management. OBJECTIVE: Our objectives were to quantify the frequency of inappropriate nonformulary medication (NFM) alert overrides in the inpatient setting and provide insight on how the design of formulary alerts could be improved. METHODS: Alert overrides of the top 11 (n = 206) most-utilized and highest-costing NFMs, from January 1 to December 31, 2012, were randomly selected for appropriateness evaluation. Using an empirically developed appropriateness algorithm, appropriateness of NFM alert overrides was assessed by 2 pharmacists via chart review. Appropriateness agreement of overrides was assessed with a Cohen's kappa. We also assessed which types of NFMs were most likely to be inappropriately overridden, the override reasons that were disproportionately provided in the inappropriate overrides, and the specific reasons the overrides were considered inappropriate. RESULTS: Approximately 17.2% (n = 35.4/206) of NFM alerts were inappropriately overridden. Non-oral NFM alerts were more likely to be inappropriately overridden compared to orals. Alerts overridden with "blank" reasons were more likely to be inappropriate. The failure to first try a formulary alternative was the most common reason for alerts being overridden inappropriately. CONCLUSION: Approximately 1 in 5 NFM alert overrides are overridden inappropriately. Future research should evaluate the impact of mandating a valid override reason and adding a list of formulary alternatives to each NFM alert; we speculate these NFM alert features may decrease the frequency of inappropriate overrides.

Development of an algorithm to assess appropriateness of overriding alerts for nonformulary medications in a computerized prescriber-order-entry system.

PURPOSE: An algorithm for assessing the appropriateness of physician overrides of clinical decision support alerts triggered by nonformulary medication (NFM) requests is described. METHODS: Data on a random sample of 5000 NFM alert overrides at Brigham and Women's Hospital over a four-year period (2009-12) were extracted from the hospital's computerized prescriber-order-entry (CPOE) system. Through an iterative process, a scheme for categorizing the reasons given by prescribers for alert overrides was developed. A pharmacist and a physician used the categorization scheme to classify and group alert override reasons, and the resultant data guided the development of an algorithm for assessing alert overrides. RESULTS: In free-text comments written in response to NFM alerts, prescribers provided more than 1150 unique reasons to justify formulary deviation. The compiled reasons were analyzed and grouped into nine categories through the iterative process, with a high degree of interrater agreement (κ = 0.989; 95% confidence interval, 0.985-0.992). An initially developed 30-item "NFM alert override appropriateness algorithm" was simplified to create an 8-question algorithm that was presented to an interdisciplinary team for evaluation, with subsequent refinements for enhanced clinical creditability. The final algorithm can be used by researchers and formulary managers to develop strategies for limiting NFM alert overrides and to avoid the labor-intensive task of creating appropriateness criteria for each NFM. CONCLUSION: A multistep process was used to develop a generalized algorithm for categorizing the appropriateness of reasons given for NFM alert overrides in a CPOE system.

Impact of incorporating pharmacy claims data into electronic medication reconciliation.

PURPOSE: The potential value of adding pharmacy claims data to the medication history in the electronic health record (EHR) to improve the accuracy of medication reconciliation was studied. METHODS: Three medication history sources were used for this evaluation: a gold-standard preadmission medication list (PAML) created by reviewing all available medication history information, an EHR-generated PAML, and pharmacy claims data. The study population consisted of patients from the Partners Medication Reconciliation Study with medication history information available from all three medication history sources. The aggregated medication list from each medication history source was compared with the gold-standard PAML to identify and categorize missing medications, additional medications, and discrepancies in the various attributes of a medication order, including dose, route, and frequency. McNemar's test was used to compare paired proportions of medication entries across each source to the gold-standard PAMLs. RESULTS: Fifteen patients had medication histories in all three medication history sources. Medication entries across all three sources included 169 from the gold- standard PAMLs, 158 from the EHR-PAMLs, and 351 from pharmacy claims data. The EHR-PAMLs and pharmacy claims data correctly reflected 52.1% and 43.2% of the gold-standard PAMLs, respectively. Combining the EHR-PAMLs and pharmacy claims resulted in 69.2% of the gold-standard PAMLs correctly reflected (p < 0.0001). Combining these two data sources increased the accuracy of medication history by 17.1%. CONCLUSION: Combining the EHR-PAML and pharmacy claims data resulted in a significant increase in the number of medications correctly reflected in the gold-standard PAML compared with the EHR-PAML or claims data separately.