Causal fairness assessment of treatment allocation with electronic health records.

OBJECTIVE: Healthcare continues to grapple with the persistent issue of treatment disparities, sparking concerns regarding the equitable allocation of treatments in clinical practice. While various fairness metrics have emerged to assess fairness in decision-making processes, a growing focus has been on causality-based fairness concepts due to their capacity to mitigate confounding effects and reason about bias. However, the application of causal fairness notions in evaluating the fairness of clinical decision-making with electronic health record (EHR) data remains an understudied domain. This study aims to address the methodological gap in assessing causal fairness of treatment allocation with electronic health records data. In addition, we investigate the impact of social determinants of health on the assessment of causal fairness of treatment allocation. METHODS: We propose a causal fairness algorithm to assess fairness in clinical decision-making. Our algorithm accounts for the heterogeneity of patient populations and identifies potential unfairness in treatment allocation by conditioning on patients who have the same likelihood to benefit from the treatment. We apply this framework to a patient cohort with coronary artery disease derived from an EHR database to evaluate the fairness of treatment decisions. RESULTS: Our analysis reveals notable disparities in coronary artery bypass grafting (CABG) allocation among different patient groups. Women were found to be 4.4%-7.7% less likely to receive CABG than men in two out of four treatment response strata. Similarly, Black or African American patients were 5.4%-8.7% less likely to receive CABG than others in three out of four response strata. These results were similar when social determinants of health (insurance and area deprivation index) were dropped from the algorithm. These findings highlight the presence of disparities in treatment allocation among similar patients, suggesting potential unfairness in the clinical decision-making process. CONCLUSION: This study introduces a novel approach for assessing the fairness of treatment allocation in healthcare. By incorporating responses to treatment into fairness framework, our method explores the potential of quantifying fairness from a causal perspective using EHR data. Our research advances the methodological development of fairness assessment in healthcare and highlight the importance of causality in determining treatment fairness.

Use of Recommended Neurodiagnostic Evaluation Among Patients With Drug-Resistant Epilepsy.

Importance: Interdisciplinary practice parameters recommend that patients with drug-resistant epilepsy (DRE) undergo comprehensive neurodiagnostic evaluation, including presurgical assessment. Reporting from specialized centers suggests long delays to referral and underuse of surgery; however, longitudinal data are limited to characterize neurodiagnostic evaluation among patients with DRE in more diverse US settings and populations. Objective: To examine the rate and factors associated with neurodiagnostic studies and comprehensive evaluation among patients with DRE within 3 US cohorts. Design, Setting, and Participants: A retrospective cross-sectional study was conducted using the Observational Medical Outcomes Partnership Common Data Model including US multistate Medicaid data, commercial claims data, and Columbia University Medical Center (CUMC) electronic health record data. Patients meeting a validated computable phenotype algorithm for DRE between January 1, 2015, and April 1, 2020, were included. No eligible participants were excluded. Exposure: Demographic and clinical variables were queried. Main Outcomes and Measures: The proportion of patients receiving a composite proxy for comprehensive neurodiagnostic evaluation, including (1) magnetic resonance or other advanced brain imaging, (2) video electroencephalography, and (3) neuropsychological evaluation within 2 years of meeting the inclusion criteria. Results: A total of 33 542 patients with DRE were included in the Medicaid cohort, 22 496 in the commercial insurance cohort, and 2741 in the CUMC database. A total of 31 516 patients (53.6%) were women. The proportion of patients meeting the comprehensive evaluation main outcome in the Medicaid cohort was 4.5% (n = 1520); in the commercial insurance cohort, 8.0% (n = 1796); and in the CUMC cohort, 14.3% (n = 393). Video electroencephalography (24.9% Medicaid, 28.4% commercial, 63.2% CUMC) and magnetic resonance imaging of the brain (35.6% Medicaid, 43.4% commercial, 52.6% CUMC) were performed more regularly than neuropsychological evaluation (13.0% Medicaid, 16.6% commercial, 19.2% CUMC) or advanced imaging (3.2% Medicaid, 5.4% commercial, 13.1% CUMC). Factors independently associated with greater odds of evaluation across all 3 data sets included the number of inpatient and outpatient nonemergency epilepsy visits and focal rather than generalized epilepsy. Conclusions and Relevance: The findings of this study suggest there is a gap in the use of diagnostic studies to evaluate patients with DRE. Care setting, insurance type, frequency of nonemergency visits, and epilepsy type are all associated with evaluation. A common data model can be used to measure adherence with best practices across a variety of observational data sources.

Clinical comparison between trial participants and potentially eligible patients using electronic health record data: A generalizability assessment method.

OBJECTIVE: To present a generalizability assessment method that compares baseline clinical characteristics of trial participants (TP) to potentially eligible (PE) patients as presented in their electronic health record (EHR) data while controlling for clinical setting and recruitment period. METHODS: For each clinical trial, a clinical event was defined to identify patients of interest using available EHR data from one clinical setting during the trial's recruitment timeframe. The trial's eligibility criteria were then applied and patients were separated into two mutually exclusive groups: (1) TP, which were patients that participated in the trial per trial enrollment data; (2) PE, the remaining patients. The primary outcome was standardized differences in clinical characteristics between TP and PE per trial. A standardized difference was considered prominent if its absolute value was greater than or equal to 0.1. The secondary outcome was the difference in mean propensity scores (PS) between TP and PE per trial, in which the PS represented prediction for a patient to be in the trial. Three diverse trials were selected for illustration: one focused on hepatitis C virus (HCV) patients receiving a liver transplantation; one focused on leukemia patients and lymphoma patients; and one focused on appendicitis patients. RESULTS: For the HCV trial, 43 TP and 83 PE were found, with 61 characteristics evaluated. Prominent differences were found among 69% of characteristics, with a mean PS difference of 0.13. For the leukemia/lymphoma trial, 23 TP and 23 PE were found, with 39 characteristics evaluated. Prominent differences were found among 82% of characteristics, with a mean PS difference of 0.76. For the appendicitis trial, 123 TP and 242 PE were found, with 52 characteristics evaluated. Prominent differences were found among 52% of characteristics, with a mean PS difference of 0.15. CONCLUSIONS: Differences in clinical characteristics were observed between TP and PE among all three trials. In two of the three trials, not all of the differences necessarily compromised trial generalizability and subsets of PE could be considered similar to their corresponding TP. In the remaining trial, lack of generalizability appeared present, but may be a result of other factors such as small sample size or site recruitment strategy. These inconsistent findings suggest eligibility criteria alone are sometimes insufficient in defining a target group to generalize to. With caveats in limited scalability, EHR data quality, and lack of patient perspective on trial participation, this generalizability assessment method that incorporates control for temporality and clinical setting promise to better pinpoint clinical patterns and trial considerations.

Baseline phenotype and 30-day outcomes of people tested for COVID-19: an international network cohort including >3.32 million people tested with real-time PCR and >219,000 tested positive for SARS-CoV-2 in South Korea, Spain and the United States.

Early identification of symptoms and comorbidities most predictive of COVID-19 is critical to identify infection, guide policies to effectively contain the pandemic, and improve health systems' response. Here, we characterised socio-demographics and comorbidity in 3,316,107persons tested and 219,072 persons tested positive for SARS-CoV-2 since January 2020, and their key health outcomes in the month following the first positive test. Routine care data from primary care electronic health records (EHR) from Spain, hospital EHR from the United States (US), and claims data from South Korea and the US were used. The majority of study participants were women aged 18-65 years old. Positive/tested ratio varied greatly geographically (2.2:100 to 31.2:100) and over time (from 50:100 in February-April to 6.8:100 in May-June). Fever, cough and dyspnoea were the most common symptoms at presentation. Between 4%-38% required admission and 1-10.5% died within a month from their first positive test. Observed disparity in testing practices led to variable baseline characteristics and outcomes, both nationally (US) and internationally. Our findings highlight the importance of large scale characterization of COVID-19 international cohorts to inform planning and resource allocation including testing as countries face a second wave.

Application of Epidemiological Geographic Information System: An Open-Source Spatial Analysis Tool Based on the OMOP Common Data Model.

BACKGROUND: Spatial epidemiology is used to evaluate geographical variations and disparities in health outcomes; however, constructing geographic statistical models requires a labor-intensive process that limits the overall utility. We developed an open-source software for spatial epidemiological analysis and demonstrated its applicability and quality. METHODS: Based on standardized geocode and observational health data, the Application of Epidemiological Geographic Information System (AEGIS) provides two spatial analysis methods: disease mapping and detecting clustered medical conditions and outcomes. The AEGIS assesses the geographical distribution of incidences and health outcomes in Korea and the United States, specifically incidence of cancers and their mortality rates, endemic malarial areas, and heart diseases (only the United States). RESULTS: The AEGIS-generated spatial distribution of incident cancer in Korea was consistent with previous reports. The incidence of liver cancer in women with the highest Moran's I (0.44; p < 0.001) was 17.4 (10.3-26.9). The malarial endemic cluster was identified in Paju-si, Korea (p < 0.001). When the AEGIS was applied to the database of the United States, a heart disease cluster was appropriately identified (p < 0.001). CONCLUSIONS: As an open-source, cross-country, spatial analytics solution, AEGIS may globally assess the differences in geographical distribution of health outcomes through the use of standardized geocode and observational health databases.

Challenges with quality of race and ethnicity data in observational databases.

OBJECTIVE: We sought to assess the quality of race and ethnicity information in observational health databases, including electronic health records (EHRs), and to propose patient self-recording as an improvement strategy. MATERIALS AND METHODS: We assessed completeness of race and ethnicity information in large observational health databases in the United States (Healthcare Cost and Utilization Project and Optum Labs), and at a single healthcare system in New York City serving a racially and ethnically diverse population. We compared race and ethnicity data collected via administrative processes with data recorded directly by respondents via paper surveys (National Health and Nutrition Examination Survey and Hospital Consumer Assessment of Healthcare Providers and Systems). Respondent-recorded data were considered the gold standard for the collection of race and ethnicity information. RESULTS: Among the 160 million patients from the Healthcare Cost and Utilization Project and Optum Labs datasets, race or ethnicity was unknown for 25%. Among the 2.4 million patients in the single New York City healthcare system's EHR, race or ethnicity was unknown for 57%. However, when patients directly recorded their race and ethnicity, 86% provided clinically meaningful information, and 66% of patients reported information that was discrepant with the EHR. DISCUSSION: Race and ethnicity data are critical to support precision medicine initiatives and to determine healthcare disparities; however, the quality of this information in observational databases is concerning. Patient self-recording through the use of patient-facing tools can substantially increase the quality of the information while engaging patients in their health. CONCLUSIONS: Patient self-recording may improve the completeness of race and ethnicity information.

Technology Access, Technical Assistance, and Disparities in Inpatient Portal Use.

BACKGROUND: Disadvantaged populations, including minorities and the elderly, use patient portals less often than relatively more advantaged populations. Limited access to and experience with technology contribute to these disparities. Free access to devices, the Internet, and technical assistance may eliminate disparities in portal use. OBJECTIVE: To examine predictors of frequent versus infrequent portal use among hospitalized patients who received free access to an iPad, the Internet, and technical assistance. MATERIALS AND METHODS: This subgroup analysis includes 146 intervention-arm participants from a pragmatic randomized controlled trial of an inpatient portal. The participants received free access to an iPad and inpatient portal while hospitalized on medical and surgical cardiac units, together with hands-on help using them. We used logistic regression to identify characteristics predictive of frequent use. RESULTS: More technology experience (adjusted odds ratio [OR] = 5.39, p = 0.049), less severe illness (adjusted OR = 2.07, p = 0.077), and private insurance (adjusted OR = 2.25, p = 0.043) predicted frequent use, with a predictive performance (area under the curve) of 65.6%. No significant differences in age, gender, race, ethnicity, level of education, employment status, or patient activation existed between the frequent and infrequent users in bivariate analyses. Significantly more frequent users noticed medical errors during their hospital stay. DISCUSSION AND CONCLUSION: Portal use was not associated with several sociodemographic characteristics previously found to limit use in the inpatient setting. However, limited technology experience and high illness severity were still barriers to frequent use. Future work should explore additional strategies, such as enrolling health care proxies and improving usability, to reduce potential disparities in portal use.

Engaging hospital patients in the medication reconciliation process using tablet computers.

Objective: Unintentional medication discrepancies contribute to preventable adverse drug events in patients. Patient engagement in medication safety beyond verbal participation in medication reconciliation is limited. We conducted a pilot study to determine whether patients' use of an electronic home medication review tool could improve medication safety during hospitalization. Materials and Methods: Patients were randomized to use a tool before or after hospital admission medication reconciliation to review and modify their home medication list. We assessed the quantity, potential severity, and potential harm of patients' and clinicians' medication changes. We also surveyed clinicians to assess the tool's usefulness. Results: Of 76 patients approached, 65 (86%) participated. Forty-eight (74%) made changes to their home medication list [before: 29 (81%), after: 19 (66%), p = .170]. Before group participants identified 57 changes that clinicians subsequently missed on admission medication reconciliation. Thirty-nine (74%) had a significant or greater potential severity, and 19 (36%) had a greater than 50-50 chance of harm. After group patients identified 68 additional changes to their reconciled medication lists. Fifty-one (75%) had a significant or greater potential severity, and 33 (49%) had a greater than 50-50 chance of harm. Clinicians reported believing that the tool would save time, and patients would supply useful information. Discussion: The results demonstrate a high willingness of patients to engage in medication reconciliation, and show that patients were able to identify important medication discrepancies and often changes that clinicians missed. Conclusion: Engaging patients in admission medication reconciliation using an electronic home medication review tool may improve medication safety during hospitalization.

Beyond discrimination: A comparison of calibration methods and clinical usefulness of predictive models of readmission risk.

BACKGROUND: Prior to implementing predictive models in novel settings, analyses of calibration and clinical usefulness remain as important as discrimination, but they are not frequently discussed. Calibration is a model's reflection of actual outcome prevalence in its predictions. Clinical usefulness refers to the utilities, costs, and harms of using a predictive model in practice. A decision analytic approach to calibrating and selecting an optimal intervention threshold may help maximize the impact of readmission risk and other preventive interventions. OBJECTIVES: To select a pragmatic means of calibrating predictive models that requires a minimum amount of validation data and that performs well in practice. To evaluate the impact of miscalibration on utility and cost via clinical usefulness analyses. MATERIALS AND METHODS: Observational, retrospective cohort study with electronic health record data from 120,000 inpatient admissions at an urban, academic center in Manhattan. The primary outcome was thirty-day readmission for three causes: all-cause, congestive heart failure, and chronic coronary atherosclerotic disease. Predictive modeling was performed via L1-regularized logistic regression. Calibration methods were compared including Platt Scaling, Logistic Calibration, and Prevalence Adjustment. Performance of predictive modeling and calibration was assessed via discrimination (c-statistic), calibration (Spiegelhalter Z-statistic, Root Mean Square Error [RMSE] of binned predictions, Sanders and Murphy Resolutions of the Brier Score, Calibration Slope and Intercept), and clinical usefulness (utility terms represented as costs). The amount of validation data necessary to apply each calibration algorithm was also assessed. RESULTS: C-statistics by diagnosis ranged from 0.7 for all-cause readmission to 0.86 (0.78-0.93) for congestive heart failure. Logistic Calibration and Platt Scaling performed best and this difference required analyzing multiple metrics of calibration simultaneously, in particular Calibration Slopes and Intercepts. Clinical usefulness analyses provided optimal risk thresholds, which varied by reason for readmission, outcome prevalence, and calibration algorithm. Utility analyses also suggested maximum tolerable intervention costs, e.g., $1720 for all-cause readmissions based on a published cost of readmission of $11,862. CONCLUSIONS: Choice of calibration method depends on availability of validation data and on performance. Improperly calibrated models may contribute to higher costs of intervention as measured via clinical usefulness. Decision-makers must understand underlying utilities or costs inherent in the use-case at hand to assess usefulness and will obtain the optimal risk threshold to trigger intervention with intervention cost limits as a result.

A Data Quality Assessment Guideline for Electronic Health Record Data Reuse.

INTRODUCTION: We describe the formulation, development, and initial expert review of 3x3 Data Quality Assessment (DQA), a dynamic, evidence-based guideline to enable electronic health record (EHR) data quality assessment and reporting for clinical research. METHODS: 3x3 DQA was developed through the triangulation results from three studies: a review of the literature on EHR data quality assessment, a quantitative study of EHR data completeness, and a set of interviews with clinical researchers. Following initial development, the guideline was reviewed by a panel of EHR data quality experts. RESULTS: The guideline embraces the task-dependent nature of data quality and data quality assessment. The core framework includes three constructs of data quality: complete, correct, and current data. These constructs are operationalized according to the three primary dimensions of EHR data: patients, variables, and time. Each of the nine operationalized constructs maps to a methodological recommendation for EHR data quality assessment. The initial expert response to the framework was positive, but improvements are required. DISCUSSION: The initial version of 3x3 DQA promises to enable explicit guideline-based best practices for EHR data quality assessment and reporting. Future work will focus on increasing clarity on how and when 3x3 DQA should be used during the research process, improving the feasibility and ease-of-use of recommendation execution, and clarifying the process for users to determine which operationalized constructs and recommendations are relevant for a given dataset and study.

Preserving temporal relations in clinical data while maintaining privacy.

OBJECTIVE: Maintaining patient privacy is a challenge in large-scale observational research. To assist in reducing the risk of identifying study subjects through publicly available data, we introduce a method for obscuring date information for clinical events and patient characteristics. METHODS: The method, which we call Shift and Truncate (SANT), obscures date information to any desired granularity. Shift and Truncate first assigns each patient a random shift value, such that all dates in that patient's record are shifted by that amount. Data are then truncated from the beginning and end of the data set. RESULTS: The data set can be proven to not disclose temporal information finer than the chosen granularity. Unlike previous strategies such as a simple shift, it remains robust to frequent - even daily - updates and robust to inferring dates at the beginning and end of date-shifted data sets. Time-of-day may be retained or obscured, depending on the goal and anticipated knowledge of the data recipient. CONCLUSIONS: The method can be useful as a scientific approach for reducing re-identification risk under the Privacy Rule of the Health Insurance Portability and Accountability Act and may contribute to qualification for the Safe Harbor implementation.

Informatics to support the IOM social and behavioral domains and measures.

Consistent collection and use of social and behavioral determinants of health can improve clinical care, prevention and general health, patient satisfaction, research, and public health. A recent Institute of Medicine committee defined a panel of 11 domains and 12 measures to be included in electronic health records. Incorporating the panel into practice creates a number of informatics research opportunities as well as challenges. The informatics issues revolve around standardization, efficient collection and review, decision support, and support for research. The informatics community can aid the effort by simultaneously optimizing the collection of the selected measures while also partnering with social science researchers to develop and validate new sources of information about social and behavioral determinants of health.

Enabling claims-based decision support through non-interruptive capture of admission diagnoses and provider billing codes.

The patient problem list, like administrative claims data, has become an important source of data for decision support, patient cohort identification, and alerting systems. A two-fold intervention to increase capture of problems on the problem list automatically - with minimal disruption to admitting and provider billing workflows - is described. For new patients with no prior data in the electronic health record, the intervention resulted in a statistically significant increase in the number of problems recorded to the problem list (3.8 vs 2.9 problems post-and pre-intervention respectively, p value 2×10(-16)). The majority of problems were recorded in the first 24 hours of admission. The proportion of patients with at least one problem coded to the problem list within the first 24 hours increased from 94% to 98% before and after intervention (chi square 344, p value 2×10(-16)). ICD9 "V codes" connoting circumstances beyond disease were captured at a higher rate post intervention than before. Deyo/Charlson comorbidities derived from problem list data were more similar to those derived from claims data after the intervention than before (Jaccard similarity 0.3 post- vs 0.21 pre-intervention, p value 2×10(-16)). A workflow-sensitive, non-interruptive means of capturing provider-entered codes early in admission can improve both the quantity and content of problems on the patient problem list.

Correlating electronic health record concepts with healthcare process events.

OBJECTIVE: To study the relation between electronic health record (EHR) variables and healthcare process events. MATERIALS AND METHODS: Lagged linear correlation was calculated between five healthcare process events and 84 EHR variables (24 clinical laboratory values and 60 clinical concepts extracted from clinical notes) in a 24-year database. The EHR variables were clustered for each healthcare process event and interpreted. RESULTS: Laboratory tests tended to cluster together and note concepts tended to cluster together. Within each of those two classes, the variables clustered into clinically sensible groupings. The exact groupings varied from healthcare process event to event, with the largest differences occurring between inpatient events and outpatient events. DISCUSSION: Unlike previously reported pairwise associations between variables, which highlighted correlations across the laboratory-clinical note divide, incorporating healthcare process events appeared to be sensitive to the manner in which the variables were collected. CONCLUSION: We believe that it may be possible to exploit this sensitivity to help knowledge engineers select variables and correct for biases.

Caveats for the use of operational electronic health record data in comparative effectiveness research.

The growing amount of data in operational electronic health record systems provides unprecedented opportunity for its reuse for many tasks, including comparative effectiveness research. However, there are many caveats to the use of such data. Electronic health record data from clinical settings may be inaccurate, incomplete, transformed in ways that undermine their meaning, unrecoverable for research, of unknown provenance, of insufficient granularity, and incompatible with research protocols. However, the quantity and real-world nature of these data provide impetus for their use, and we develop a list of caveats to inform would-be users of such data as well as provide an informatics roadmap that aims to insure this opportunity to augment comparative effectiveness research can be best leveraged.

The future state of clinical data capture and documentation: a report from AMIA's 2011 Policy Meeting.

Much of what is currently documented in the electronic health record is in response toincreasingly complex and prescriptive medicolegal, reimbursement, and regulatory requirements. These requirements often result in redundant data capture and cumbersome documentation processes. AMIA's 2011 Health Policy Meeting examined key issues in this arena and envisioned changes to help move toward an ideal future state of clinical data capture and documentation. The consensus of the meeting was that, in the move to a technology-enabled healthcare environment, the main purpose of documentation should be to support patient care and improved outcomes for individuals and populations and that documentation for other purposes should be generated as a byproduct of care delivery. This paper summarizes meeting deliberations, and highlights policy recommendations and research priorities. The authors recommend development of a national strategy to review and amend public policies to better support technology-enabled data capture and documentation practices.

Using EHRs to integrate research with patient care: promises and challenges.

Clinical research is the foundation for advancing the practice of medicine. However, the lack of seamless integration between clinical research and patient care workflow impedes recruitment efficiency, escalates research costs, and hence threatens the entire clinical research enterprise. Increased use of electronic health records (EHRs) holds promise for facilitating this integration but must surmount regulatory obstacles. Among the unintended consequences of current research oversight are barriers to accessing patient information for prescreening and recruitment, coordinating scheduling of clinical and research visits, and reconciling information about clinical and research drugs. We conclude that the EHR alone cannot overcome barriers in conducting clinical trials and comparative effectiveness research. Patient privacy and human subject protection policies should be clarified at the local level to exploit optimally the full potential of EHRs, while continuing to ensure participant safety. Increased alignment of policies that regulate the clinical and research use of EHRs could help fulfill the vision of more efficiently obtaining clinical research evidence to improve human health.

Visualizing the operating range of a classification system.

The performance of a classification system depends on the context in which it will be used, including the prevalence of the classes and the relative costs of different types of errors. Metrics such as accuracy are limited to the context in which the experiment was originally carried out, and metrics such as sensitivity, specificity, and receiver operating characteristic area--while independent of prevalence--do not provide a clear picture of the performance characteristics of the system over different contexts. Graphing a prevalence-specific metric such as F-measure or the relative cost of errors over a wide range of prevalence allows a visualization of the performance of the system and a comparison of systems in different contexts.

A model for expanded public health reporting in the context of HIPAA.

The advent of electronic medical records and health information exchange raise the possibility of expanding public health reporting to detect a broad range of clinical conditions and of monitoring the health of the public on a broad scale. Expanding public health reporting may require patient anonymity, matching records, re-identifying cases, and recording patient characteristics for localization. The privacy regulations under the Health Insurance Portability and Accountability Act of 1996 (HIPAA) provide several mechanisms for public health surveillance, including using laws and regulations, public health activities, de-identification, research waivers, and limited data sets, and in addition, surveillance may be distributed with aggregate reporting. The appropriateness of these approaches varies with the definition of what data may be included, the requirements of the minimum necessary standard, the accounting of disclosures, and the feasibility of the approach.