Evaluating Algorithmic Bias in 30-Day Hospital Readmission Models: Retrospective Analysis.

BACKGROUND: The adoption of predictive algorithms in health care comes with the potential for algorithmic bias, which could exacerbate existing disparities. Fairness metrics have been proposed to measure algorithmic bias, but their application to real-world tasks is limited. OBJECTIVE: This study aims to evaluate the algorithmic bias associated with the application of common 30-day hospital readmission models and assess the usefulness and interpretability of selected fairness metrics. METHODS: We used 10.6 million adult inpatient discharges from Maryland and Florida from 2016 to 2019 in this retrospective study. Models predicting 30-day hospital readmissions were evaluated: LACE Index, modified HOSPITAL score, and modified Centers for Medicare & Medicaid Services (CMS) readmission measure, which were applied as-is (using existing coefficients) and retrained (recalibrated with 50% of the data). Predictive performances and bias measures were evaluated for all, between Black and White populations, and between low- and other-income groups. Bias measures included the parity of false negative rate (FNR), false positive rate (FPR), 0-1 loss, and generalized entropy index. Racial bias represented by FNR and FPR differences was stratified to explore shifts in algorithmic bias in different populations. RESULTS: The retrained CMS model demonstrated the best predictive performance (area under the curve: 0.74 in Maryland and 0.68-0.70 in Florida), and the modified HOSPITAL score demonstrated the best calibration (Brier score: 0.16-0.19 in Maryland and 0.19-0.21 in Florida). Calibration was better in White (compared to Black) populations and other-income (compared to low-income) groups, and the area under the curve was higher or similar in the Black (compared to White) populations. The retrained CMS and modified HOSPITAL score had the lowest racial and income bias in Maryland. In Florida, both of these models overall had the lowest income bias and the modified HOSPITAL score showed the lowest racial bias. In both states, the White and higher-income populations showed a higher FNR, while the Black and low-income populations resulted in a higher FPR and a higher 0-1 loss. When stratified by hospital and population composition, these models demonstrated heterogeneous algorithmic bias in different contexts and populations. CONCLUSIONS: Caution must be taken when interpreting fairness measures' face value. A higher FNR or FPR could potentially reflect missed opportunities or wasted resources, but these measures could also reflect health care use patterns and gaps in care. Simply relying on the statistical notions of bias could obscure or underplay the causes of health disparity. The imperfect health data, analytic frameworks, and the underlying health systems must be carefully considered. Fairness measures can serve as a useful routine assessment to detect disparate model performances but are insufficient to inform mechanisms or policy changes. However, such an assessment is an important first step toward data-driven improvement to address existing health disparities.

Ownership and Interoperability Challenges of Alzheimer Monoclonal Antibody Registries.

This Viewpoint addresses the challenges that the Centers for Medicare and Medicaid Services faces to collect real-world data on the effectiveness and safety of lecanemab from external registries to achieve its coverage with evidence development objectives.

Assessing Patient and Community-Level Social Factors; The Synergistic Effect of Social Needs and Social Determinants of Health on Healthcare Utilization at a Multilevel Academic Healthcare System.

We investigated the role of both individual-level social needs and community-level social determinants of health (SDOH) in explaining emergency department (ED) utilization rates. We also assessed the potential synergies between the two levels of analysis and their combined effect on patterns of ED visits. We extracted electronic health record (EHR) data between July 2016 and June 2020 for 1,308,598 unique Maryland residents who received care at Johns Hopkins Health System, of which 28,937 (2.2%) patients had at least one documented social need. There was a negative correlation between median household income in a neighborhood with having a social need such as financial resource strain, food insecurity, and residential instability (correlation coefficient: -0.05, -0.01, and - 0.06, p = 0, respectively). In a multilevel model with random effects after adjusting for other factors, living in a more disadvantaged neighborhood was found to be significantly associated with ED utilization statewide and within Baltimore City (OR: 1.005, 95% CI: 1.003-1.007 and 1.020, 95% CI: 1.017-1.022, respectively). However, individual-level social needs appeared to enhance the statewide effect of living in a more disadvantaged neighborhood with the OR for the interaction term between social needs and SDOH being larger, and more positive, than SDOH alone (OR: 1.012, 95% CI: 1.011-1.014). No such moderation was found in Baltimore City. To our knowledge, this study is one of the first attempts by a major academic healthcare system to assess the combined impact of patient-level social needs in association with community-level SDOH on healthcare utilization and can serve as a baseline for future studies using EHR data linked to population-level data to assess such synergistic association.

Health Disparities in Lapses in Diabetic Retinopathy Care.

Objective: To develop a novel methodology to identify lapses in diabetic retinopathy care in electronic health records (EHRs) and evaluate health disparities by race and ethnicity. Design: Retrospective cohort study. Subjects: Adult patients with diabetes mellitus who were evaluated at the Wilmer Eye Institute from January 1, 2013 to April 2, 2022. Methods: The methodology to identify lapses in care first identified diabetic retinopathy screening or treatment visits and then compared the providers' recommended follow-up timeframe with the patient's actual time to next encounter. The association of race and ethnicity with odds of lapses in care was evaluated using a mixed-effects logistic regression model controlling for age, sex, insurance, severity of diabetic retinopathy, presence of other retinal disorders, and glaucoma. Main Outcome Measures: Lapses in diabetic retinopathy care. Results: The methodology to identify diabetic retinopathy-related visits had a 95.0% (95% confidence interval, 93.0-96.6) sensitivity and 98.8% (98.1-99.3) specificity as compared with a gold standard grader. The methodology resulted in a 97.3% (96.2-98.4) sensitivity and 98.1% (97.3-98.9) specificity for detecting a follow-up recommendation, with an average error of -0.05 (-0.31 to 0.21) weeks in extracting the precise timeframe. A total of 39 561 patients with 91 104 office visits were included in the analysis. The average age was 61.4 years. More than 3 (77.6%) in 4 patients had a lapse in care. In multivariable analysis, non-Hispanic Black patients had 1.24 (1.19-1.30) odds and Hispanic patients had 1.26 (1.13-1.40) odds of ever having a lapse in care compared with non-Hispanic White patients (P < 0.001, respectively). Conclusions: We have developed a reliable methodology for identifying lapses in diabetic retinopathy care that is tailored to a provider's recommended follow-up. Using this approach, we find that 3 in 4 patients experience a lapse in diabetic retinopathy care and that these rates are higher among non-Hispanic Black and Hispanic patients. Deploying this methodology in the EHR is one potential means by which to identify and mitigate lapses in critical ophthalmic care in patients with diabetes. Financial Disclosures: Proprietary or commercial disclosure may be found after the references.

Trends and Patterns of Social History Data Collection Within an Electronic Health Record.

There is increased acceptance that social and behavioral determinants of health (SBDH) impact health outcomes, but electronic health records (EHRs) are not always set up to capture the full range of SBDH variables in a systematic manner. The purpose of this study was to explore rates and trends of social history (SH) data collection-1 element of SBDH-in a structured portion of an EHR within a large academic integrated delivery system. EHR data for individuals with at least 1 visit in 2017 were included in this study. Completeness rates were calculated for how often SBDH variable was assessed and documented. Logistic regressions identified factors associated with assessment rates for each variable. A total of 44,166 study patients had at least 1 SH variable present. Tobacco use and alcohol use were the most frequently captured SH variables. Black individuals were more likely to have their alcohol use assessed (odds ratio [OR] 1.21) compared with White individuals, whereas White individuals were more likely to have their "smokeless tobacco use" assessed (OR 0.92). There were also differences between insurance types. Drug use was more likely to be assessed in the Medicaid population for individuals who were single (OR 0.95) compared with the commercial population (OR 1.05). SH variable assessment is inconsistent, which makes use of EHR data difficult to gain better understanding of the impact of SBDH on health outcomes. Standards and guidelines on how and why to collect SBDH information within the EHR are needed.

Measuring the geographic disparity of comorbidity in commercially insured individuals compared to the distribution of physicians in South Africa.

BACKGROUND: Measuring and addressing the disparity between access to healthcare resources and underlying health needs of populations is a prominent focus in health policy development. More recently, the fair distribution of healthcare resources among population subgroups have become an important indication of health inequities. Single disease outcomes are commonly used for healthcare resource allocations; however, leveraging population-level comorbidity measures for health disparity research has been limited. This study compares the geographical distribution of comorbidity and associated healthcare utilization among commercially insured individuals in South Africa (SA) relative to the distribution of physicians. METHODS: A retrospective, cross-sectional analysis was performed comparing the geographical distribution of comorbidity and physicians for 2.6 million commercially insured individuals over 2016-2017, stratified by geographical districts and population groups in SA. We applied the Johns Hopkins ACG® System across the claims data of a large health plan administrator to measure a comorbidity risk score for each individual. By aggregating individual scores, we determined the average healthcare resource need of individuals per district, known as the comorbidity index (CMI), to describe the disease burden per district. Linear regression models were constructed to test the relationship between CMI, age, gender, population group, and population density against physician density. RESULTS: Our results showed a tendency for physicians to practice in geographic areas with more insurance enrollees and not necessarily where disease burden may be highest. This was confirmed by a negative relationship between physician density and CMI for the overall population and for three of the four major population groups. Among the population groups, the Black African population had, on average, access to fewer physicians per capita than other population groups, before and after adjusting for confounding factors. CONCLUSION: CMI is a novel measure for healthcare disparities research that considers both acute and chronic conditions contributing to current and future healthcare costs. Our study linked and compared the population-level geographical distribution of CMI to the distribution of physicians using routinely collected data. Our results could provide vital information towards the more equitable distribution of healthcare providers across population groups in SA, and to meet the healthcare needs of disadvantaged communities.

Assessing the Racial and Socioeconomic Disparities in Postpartum Depression Using Population-Level Hospital Discharge Data: Longitudinal Retrospective Study.

BACKGROUND: In the United States, >3.6 million deliveries occur annually. Among them, up to 20% (approximately 700,000) of women experience postpartum depression (PPD) according to the Centers for Disease Control and Prevention. Absence of accurate reporting and diagnosis has made phenotyping of patients with PPD difficult. Existing literature has shown that factors such as race, socioeconomic status, and history of substance abuse are associated with the differential risks of PPD. However, limited research has considered differential temporal associations with the outcome. OBJECTIVE: This study aimed to estimate the disparities in the risk of PPD and time to diagnosis for patients of different racial and socioeconomic backgrounds. METHODS: This is a longitudinal retrospective study using the statewide hospital discharge data from Maryland. We identified 160,066 individuals who had a hospital delivery from 2017 to 2019. We applied logistic regression and Cox regression to study the risk of PPD across racial and socioeconomic strata. Multinomial regression was used to estimate the risk of PPD at different postpartum stages. RESULTS: The cumulative incidence of PPD diagnosis was highest for White patients (8779/65,028, 13.5%) and lowest for Asian and Pacific Islander patients (248/10,760, 2.3%). Compared with White patients, PPD diagnosis was less likely to occur for Black patients (odds ratio [OR] 0.31, 95% CI 0.30-0.33), Asian or Pacific Islander patients (OR 0.17, 95% CI 0.15-0.19), and Hispanic patients (OR 0.21, 95% CI 0.19-0.22). Similar findings were observed from the Cox regression analysis. Multinomial regression showed that compared with White patients, Black patients (relative risk 2.12, 95% CI 1.73-2.60) and Asian and Pacific Islander patients (relative risk 2.48, 95% CI 1.46-4.21) were more likely to be diagnosed with PPD after 8 weeks of delivery. CONCLUSIONS: Compared with White patients, PPD diagnosis is less likely to occur in individuals of other races. We found disparate timing in PPD diagnosis across different racial groups and socioeconomic backgrounds. Our findings serve to enhance intervention strategies and policies for phenotyping patients at the highest risk of PPD and to highlight needs in data quality to support future work on racial disparities in PPD.

Assessing the Added Value of Vital Signs Extracted from Electronic Health Records in Healthcare Risk Adjustment Models.

Purpose: Patient vital signs are related to specific health risks and outcomes but are underutilized in the prediction of health-care utilization and cost. To measure the added value of electronic health record (EHR) extracted Body Mass Index (BMI) and blood pressure (BP) values in improving healthcare risk and utilization predictions. Patients and Methods: A sample of 12,820 adult outpatients from the Johns Hopkins Health System (JHHS) were identified between 2016 and 2017, having high data quality and recorded values for BMI and BP. We evaluated the added value of BMI and BP in predicting health-care utilization and cost through a retrospective cohort design. BMI, mean arterial pressure (MAP), systolic and diastolic BPs were summarized as annual aggregated values. Concurrent annual BMI and MAP changes were quantified as the difference between maximum and minimum recorded values. Model performance estimates consisted of repeated 10-fold cross validation, compared to base model point estimates for demographic and diagnostic, coded events: (1) patient age and sex, (2) age, sex, and the Charlson weighted index, (3) age, sex and the Johns Hopkins ACG system's DxPM risk score. Results: Both categorical BMI and BP were progressively indicative of disease comorbidity, but not uniformly related to health-care utilization or cost. Annual change in BMI and MAP improved predictions for most concurrent year outcomes when compared to base models. Conclusion: When a healthcare system lacks relevant diagnostic or risk assessment information for a patient, vital signs may be useful for a simple estimation of disease risk, cost and utilization.

Impact of Social Needs in Electronic Health Records and Claims on Health Care Utilization and Costs Risk-Adjustment Models Within Medicaid Population.

Patients enrolled in Medicaid have significantly higher social needs (SNs) than others. Using claims and electronic health records (EHRs) data, managed care organizations (MCOs) could systemically identify high-risk patients with SNs and develop population health management interventions. Impact of SNs on models predicting health care utilization and costs was assessed. This retrospective study included claims and EHRs data on 39,267 patients younger than age 65 years who were continuously enrolled during 2018-2019 in a Medicaid-managed care plan. SN marker was developed suggesting presence of International Classification of Diseases, 10th revision codes in any of the 5 SN domains. Impact of SN marker was compared across demographic and 2 diagnosis-based (ie, Charlson and Adjusted Clinical Groups risk score) prediction models of emergency department (ED) visit and hospitalizations, and total, medical, and pharmacy costs. After combining data sources, prevalence of documented SN marker increased from 11% and 13% to 18% of the study population across claims, EHRs, and both combined, respectively. SN marker improved predictions of demographic models for all utilization and total costs outcomes (area under the curve [AUC] of ED model increased from 0.57 to 0.61 and R2 of total cost model increased from 10.9 to 12.2). In both diagnosis-based models, adding SN marker marginally improved outcomes prediction (AUC of ED model increased from 0.65 to 0.66). This study demonstrated feasibility of using claims and EHRs data to systematically capture SNs and incorporate in prediction models that could enable MCOs and policy makers to adjust and develop effective population health interventions.

Impact of social determinants of health on improving the LACE index for 30-day unplanned readmission prediction.

Objective: Early and accurate prediction of patients at risk of readmission is key to reducing costs and improving outcomes. LACE is a widely used score to predict 30-day readmissions. We examine whether adding social determinants of health (SDOH) to LACE can improve its predictive performance. Methods: This is a retrospective study that included all inpatient encounters in the state of Maryland in 2019. We constructed predictive models by fitting Logistic Regression (LR) on LACE and different sets of SDOH predictors. We used the area under the curve (AUC) to evaluate discrimination and SHapley Additive exPlanations values to assess feature importance. Results: Our study population included 316 558 patients of whom 35 431 (11.19%) patients were readmitted after 30 days. Readmitted patients had more challenges with individual-level SDOH and were more likely to reside in communities with poor SDOH conditions. Adding a combination of individual and community-level SDOH improved LACE performance from AUC = 0.698 (95% CI [0.695-0.7]; ref) to AUC = 0.708 (95% CI [0.705-0.71]; P < .001). The increase in AUC was highest in black patients (+1.6), patients aged 65 years or older (+1.4), and male patients (+1.4). Discussion: We demonstrated the value of SDOH in improving the LACE index. Further, the additional predictive value of SDOH on readmission risk varies by subpopulations. Vulnerable populations like black patients and the elderly are likely to benefit more from the inclusion of SDOH in readmission prediction. Conclusion: These findings provide potential SDOH factors that health systems and policymakers can target to reduce overall readmissions.

Claims-based pharmacy markers for comprehensive medication management program case identification: Validation against concurrent and prospective healthcare costs and utilization.

BACKGROUND: Three claims-based pharmacy markers (complex, costly and risky medications) were developed to help automatically identify patients for comprehensive medication management. OBJECTIVE: To evaluate the association between newly-developed markers and healthcare outcomes. METHODS: This was a two-year retrospective cohort study using PharMetrics Plus patient-level administrative claims in 2014 and 2015. We included all claims from 1,541,873 individuals with: (1) 24-month medical and pharmacy enrollment in 2014 and 2015, (2) aged between 18 and 63 in 2014, and (3) known gender. Independent/control variables came from 2014 while outcomes came from 2014 (concurrent analysis) and 2015 (prospective analysis). Three pharmacy markers, separately or together, were added to four base models to predict concurrent and prospective healthcare costs (total, medical, and pharmacy) and utilization (having any hospitalization, having any emergency department visit, and having any readmission). We applied linear regression for costs while logistic regression for utilization. Measures of model performances and coefficients were derived from a 5-fold cross-validation repeated 20 times. RESULTS: Individuals with 1+ complex, risky or costly medication markers had higher comorbidity, healthcare costs and utilization than their counterparts. Nine binary risky category markers performed the best among the three types of risky medication markers; the Medication Complexity Score and three-level complex category both outperformed a simpler complex medication indicator. Adding three novel pharmacy markers separately or together into the base models provided the greatest improvement in explaining pharmacy costs, compared with medical (non-medication) costs. These pharmacy markers also added value in explaining healthcare utilization among the simple base models. CONCLUSIONS: Three claims-based pharmacy indicators had positive associations with healthcare outcomes and added value in predicting them. This initial study suggested that these novel markers can be used by pharmacy case management programs to help identify potential high-risk patients most likely to benefit from clinical pharmacist review and other interventions.

A bias evaluation checklist for predictive models and its pilot application for 30-day hospital readmission models.

OBJECTIVE: Health care providers increasingly rely upon predictive algorithms when making important treatment decisions, however, evidence indicates that these tools can lead to inequitable outcomes across racial and socio-economic groups. In this study, we introduce a bias evaluation checklist that allows model developers and health care providers a means to systematically appraise a model's potential to introduce bias. MATERIALS AND METHODS: Our methods include developing a bias evaluation checklist, a scoping literature review to identify 30-day hospital readmission prediction models, and assessing the selected models using the checklist. RESULTS: We selected 4 models for evaluation: LACE, HOSPITAL, Johns Hopkins ACG, and HATRIX. Our assessment identified critical ways in which these algorithms can perpetuate health care inequalities. We found that LACE and HOSPITAL have the greatest potential for introducing bias, Johns Hopkins ACG has the most areas of uncertainty, and HATRIX has the fewest causes for concern. DISCUSSION: Our approach gives model developers and health care providers a practical and systematic method for evaluating bias in predictive models. Traditional bias identification methods do not elucidate sources of bias and are thus insufficient for mitigation efforts. With our checklist, bias can be addressed and eliminated before a model is fully developed or deployed. CONCLUSION: The potential for algorithms to perpetuate biased outcomes is not isolated to readmission prediction models; rather, we believe our results have implications for predictive models across health care. We offer a systematic method for evaluating potential bias with sufficient flexibility to be utilized across models and applications.

Comparing and validating medication complexity from insurance claims against electronic health records.

BACKGROUND: Patient effort to comply with complex medication instructions is known to be related to nonadherence and subsequent medical complications or health care costs. A widely used Medication Regimen Complexity Index (MRCI) has been used with electronic health records (EHRs) to identify patients who could benefit from pharmacist intervention. A similar claims-derived measure may be better suited for clinical decision support, since claims offer a more complete view of patient care and health utilization. OBJECTIVE: To define and validate a novel insurance claims-based medication complexity score (MCS) patterned after the widely used MRCI, derived from EHRs. METHODS: Insurance claims and EHR data were provided by HealthPartners (N = 54,988) (Bloomington, Minnesota) and The Johns Hopkins Health System (N = 28,589) (Baltimore, Maryland) for years 2013 and 2017, respectively. Yearly measures of medication complexity were developed for each patient and evaluated with one another using rank correlation within different clinical subgroupings. Indicators for the presence of individually complex prescriptions were also developed and assessed using exact agreement. Complexity measures were then correlated with select covariates to further validate the concordance between MCS and MRCI with respect to clinical metrics. These included demographic, comorbidity, and health care utilization markers. Prescribed medications in each system's EHR were coded using the previously validated MRCI weighting rules. Insurance claims for retail pharmacy medications were coded using our novel MCS, which closely followed MRCI scoring rules. RESULTS: EHR-based MRCI and claims-based MCS were significantly correlated with one another for most clinical subgroupings. Likewise, both measures were correlated with several covariates, including count of active medications and chronic conditions. The MCS was, in most cases, more associated with key health covariates than was MRCI, although both were consistently significant. We found that the highest correlation between MCS and MRCI is obtained with patients who have similar counts of pharmacy records between EHRs and claims (HealthPartners: P = 0.796; Johns Hopkins Health System: P = 0.779). CONCLUSIONS: The findings suggest good correspondence between MCS and MRCI and that claims data represent a useful resource for assessing medication complexity. Claims data also have major practical advantages, such as interoperability across health care systems, although they lack the detailed clinical context of EHRs. DISCLOSURES: The Johns Hopkins University holds the copyright to the Adjusted Clinical Groups (ACG) system and receives royalties from the global distribution of the ACG system. This revenue supports a portion of the authors' salary. No additional or external funding supported this work. The authors have no conflict of interest to disclose.

Development and assessment of a natural language processing model to identify residential instability in electronic health records' unstructured data: a comparison of 3 integrated healthcare delivery systems.

OBJECTIVE: To evaluate whether a natural language processing (NLP) algorithm could be adapted to extract, with acceptable validity, markers of residential instability (ie, homelessness and housing insecurity) from electronic health records (EHRs) of 3 healthcare systems. MATERIALS AND METHODS: We included patients 18 years and older who received care at 1 of 3 healthcare systems from 2016 through 2020 and had at least 1 free-text note in the EHR during this period. We conducted the study independently; the NLP algorithm logic and method of validity assessment were identical across sites. The approach to the development of the gold standard for assessment of validity differed across sites. Using the EntityRuler module of spaCy 2.3 Python toolkit, we created a rule-based NLP system made up of expert-developed patterns indicating residential instability at the lead site and enriched the NLP system using insight gained from its application at the other 2 sites. We adapted the algorithm at each site then validated the algorithm using a split-sample approach. We assessed the performance of the algorithm by measures of positive predictive value (precision), sensitivity (recall), and specificity. RESULTS: The NLP algorithm performed with moderate precision (0.45, 0.73, and 1.0) at 3 sites. The sensitivity and specificity of the NLP algorithm varied across 3 sites (sensitivity: 0.68, 0.85, and 0.96; specificity: 0.69, 0.89, and 1.0). DISCUSSION: The performance of this NLP algorithm to identify residential instability in 3 different healthcare systems suggests the algorithm is generally valid and applicable in other healthcare systems with similar EHRs. CONCLUSION: The NLP approach developed in this project is adaptable and can be modified to extract types of social needs other than residential instability from EHRs across different healthcare systems.

Assessment of structured data elements for social risk factors.

OBJECTIVES: Computable social risk factor phenotypes derived from routinely collected structured electronic health record (EHR) or health information exchange (HIE) data may represent a feasible and robust approach to measuring social factors. This study convened an expert panel to identify and assess the quality of individual EHR and HIE structured data elements that could be used as components in future computable social risk factor phenotypes. STUDY DESIGN: Technical expert panel. METHODS: A 2-round Delphi technique included 17 experts with an in-depth knowledge of available EHR and/or HIE data. The first-round identification sessions followed a nominal group approach to generate candidate data elements that may relate to socioeconomics, cultural context, social relationships, and community context. In the second-round survey, panelists rated each data element according to overall data quality and likelihood of systematic differences in quality across populations (ie, bias). RESULTS: Panelists identified a total of 89 structured data elements. About half of the data elements (n = 45) were related to socioeconomic characteristics. The panelists identified a diverse set of data elements. Elements used in reimbursement-related processes were generally rated as higher quality. Panelists noted that several data elements may be subject to implicit bias or reflect biased systems of care, which may limit their utility in measuring social factors. CONCLUSIONS: Routinely collected structured data within EHR and HIE systems may reflect patient social risk factors. Identifying and assessing available data elements serves as a foundational step toward developing future computable social factor phenotypes.

Assessing the Added Value of Blood Pressure Information Derived from Electronic Health Records in Predicting Health Care Cost and Utilization.

Health care providers are increasingly using clinical measures derived from electronic health records (EHRs) for risk stratification and predictive modeling. EHR-specific data elements such as prescriptions, laboratory results, and vital signs have been shown to improve risk prediction models. In this study, the value of EHR-based blood pressure (BP) values was assessed in predicting health care costs (ie, total, medical, and pharmacy) and key utilization end points (ie, hospitalization, emergency department use, and being among the highest utilizers). The study population included 37,451 patients of a large integrated delivery system in the mid-western United States with complete EHR data files, who were 18-64 years old, had continuous insurance at an affiliated health plan, and had eligible BP records. Both EHRs and insurance claims of the study population were used to extract the predictors (ie, demographics, diagnosis, and BP values) and outcomes (ie, costs and utilizations). Predictors were extracted from 2012 data, whereas concurrent and prospective outcomes were extracted from 2012 to 2013 data. Three base models (BMs) were constructed to predict each of the outcomes. The first BM no. 1 used demographics. The second BM no. 2 added the Charlson comorbidity index to BM no. 1, whereas the third BM no. 3 added the Adjusted Clinical Group Dx-PM case-mix score to BM no. 1. BP was specified as means, ranges, and classes. Adding BP ranges to BM no. 1 and BM no. 2 showed the greatest improvements when predicting costs and utilization. More specifically, adjusted R2 and area under the curve of BM no. 2 improved by 32.9% and 14.1% when BP ranges were added to predict concurrent total cost and hospitalization, respectively. The effect of BP measures on improving the risk stratification models was diminished when predicting prospective outcomes after adding the measures to BM no. 3 (ie, the more comprehensive diagnostic model), specifically when represented as BP means. Given the increasing availability of BP information, this research suggests that these data should be integrated into provider-based population health analytic activities. Future research should focus on subpopulations that benefit the most from incorporating vital signs such as BP measures in risk stratification models.

Comparing the predictive value of screening to the use of electronic health record data for detecting future suicidal thoughts and behavior in an urban pediatric emergency department: A preliminary analysis.

AIM: Brief screening and predictive modeling have garnered attention for utility at identifying individuals at risk of suicide. Although previous research has investigated these methods, little is known about how these methods compare against each other or work in combination in the pediatric population. METHODS: Patients were aged 8-18 years old who presented from January 1, 2017, to June 30, 2019, to a Pediatric Emergency Department (PED). All patients were screened with the Ask Suicide Questionnaire (ASQ) as part of a universal screening approach. For all models, we used 5-fold cross-validation. We compared four models: Model 1 only included the ASQ; Model 2 included the ASQ and EHR data gathered at the time of ED visit (EHR data); Model 3 only included EHR data; and Model 4 included EHR data and a single item from the ASQ that asked about a lifetime history of suicide attempt. The main outcome was subsequent PED visit with suicide-related presenting problem within a 3-month follow-up period. RESULTS: Of the N = 13,420 individuals, n = 141 had a subsequent suicide-related PED visit. Approximately 63% identified as Black. Results showed that a model based only on EHR data (Model 3) had an area under the curve (AUC) of 0.775 compared to the ASQ alone (Model 1), which had an AUC of 0.754. Combining screening and EHR data (Model 4) resulted in a 17.4% (absolute difference = 3.6%) improvement in sensitivity and 13.4% increase in AUC (absolute difference = 6.6%) compared to screening alone (Model 1). CONCLUSION: Our findings show that predictive modeling based on EHR data is helpful either in the absence or as an addition to brief suicide screening. This is the first study to compare brief suicide screening to EHR-based predictive modeling and adds to our understanding of how best to identify youth at risk of suicidal thoughts and behaviors in clinical care settings.

Assessing the Value of Unsupervised Clustering in Predicting Persistent High Health Care Utilizers: Retrospective Analysis of Insurance Claims Data.

BACKGROUND: A high proportion of health care services are persistently utilized by a small subpopulation of patients. To improve clinical outcomes while reducing costs and utilization, population health management programs often provide targeted interventions to patients who may become persistent high users/utilizers (PHUs). Enhanced prediction and management of PHUs can improve health care system efficiencies and improve the overall quality of patient care. OBJECTIVE: The aim of this study was to detect key classes of diseases and medications among the study population and to assess the predictive value of these classes in identifying PHUs. METHODS: This study was a retrospective analysis of insurance claims data of patients from the Johns Hopkins Health Care system. We defined a PHU as a patient incurring health care costs in the top 20% of all patients' costs for 4 consecutive 6-month periods. We used 2013 claims data to predict PHU status in 2014-2015. We applied latent class analysis (LCA), an unsupervised clustering approach, to identify patient subgroups with similar diagnostic and medication patterns to differentiate variations in health care utilization across PHUs. Logistic regression models were then built to predict PHUs in the full population and in select subpopulations. Predictors included LCA membership probabilities, demographic covariates, and health utilization covariates. Predictive powers of the regression models were assessed and compared using standard metrics. RESULTS: We identified 164,221 patients with continuous enrollment between 2013 and 2015. The mean study population age was 19.7 years, 55.9% were women, 3.3% had ≥1 hospitalization, and 19.1% had 10+ outpatient visits in 2013. A total of 8359 (5.09%) patients were identified as PHUs in both 2014 and 2015. The LCA performed optimally when assigning patients to four probability disease/medication classes. Given the feedback provided by clinical experts, we further divided the population into four diagnostic groups for sensitivity analysis: acute upper respiratory infection (URI) (n=53,232; 4.6% PHUs), mental health (n=34,456; 12.8% PHUs), otitis media (n=24,992; 4.5% PHUs), and musculoskeletal (n=24,799; 15.5% PHUs). For the regression models predicting PHUs in the full population, the F1-score classification metric was lower using a parsimonious model that included LCA categories (F1=38.62%) compared to that of a complex risk stratification model with a full set of predictors (F1=48.20%). However, the LCA-enabled simple models were comparable to the complex model when predicting PHUs in the mental health and musculoskeletal subpopulations (F1-scores of 48.69% and 48.15%, respectively). F1-scores were lower than that of the complex model when the LCA-enabled models were limited to the otitis media and acute URI subpopulations (45.77% and 43.05%, respectively). CONCLUSIONS: Our study illustrates the value of LCA in identifying subgroups of patients with similar patterns of diagnoses and medications. Our results show that LCA-derived classes can simplify predictive models of PHUs without compromising predictive accuracy. Future studies should investigate the value of LCA-derived classes for predicting PHUs in other health care settings.

Internet Access, Social Risk Factors, and Web-Based Social Support Seeking Behavior: Assessing Correlates of the "Digital Divide" Across Neighborhoods in The State of Maryland.

We aimed to empirically measure the degree to which there is a "digital divide" in terms of access to the internet at the small-area community level within the State of Maryland and the City of Baltimore and to assess the relationship and association of this divide with community-level SDOH risk factors, community-based social service agency location, and web-mediated support service seeking behavior. To assess the socio-economic characteristics of the neighborhoods across the state, we calculated the Area Deprivation Index (ADI) using the U.S. Census, American Community Survey (5-year estimates) of 2017. To assess the digital divide, at the community level, we used the Federal Communications Commission (FCC) data on the number of residential fixed Internet access service connections. We assessed the availability of and web-based access to community-based social service agencies using data provided by the "Aunt Bertha" information platform. We performed community and regional level descriptive and special analyses for ADI social risk factors, connectivity, and both the availability of and web-based searches for community-based social services. To help assess potential neighborhood linked factors associated with the rates of web-based social services searches by individuals in need, we applied logistic regression using generalized estimating equation modeling. Baltimore City contained more disadvantaged neighborhoods compared to other areas in Maryland. In Baltimore City, 20.3% of neighborhoods (defined by census block groups) were disadvantaged with ADI at the 90th percentile while only 6.6% of block groups across Maryland were in this disadvantaged category. Across the State, more than half of all census tracts had 801-1000 households (per 1000 households) with internet subscription. In contrast, in Baltimore City about half of all census tracts had only 401-600 of the households (per 1000 households) with internet subscriptions. Most block groups in Maryland and Baltimore City lacked access to social services facilities (61% of block groups at the 90th percentile of disadvantage in Maryland and 61.3% of block groups at the 90th percentile of disadvantage in Baltimore City). After adjusting for other variables, a 1% increase in the ADI measure of social disadvantage, resulting in a 1.7% increase in the number of individuals seeking social services. While more work is needed, our findings support the premise that the digital divide is closely associated with other SDOH factors. The policymakers must propose policies to address the digital divide on a national level and also in disadvantaged communities experiencing the digital divide in addition to other SDOH challenges.

Measuring the Value of a Practical Text Mining Approach to Identify Patients With Housing Issues in the Free-Text Notes in Electronic Health Record: Findings of a Retrospective Cohort Study.

Introduction: Despite the growing efforts to standardize coding for social determinants of health (SDOH), they are infrequently captured in electronic health records (EHRs). Most SDOH variables are still captured in the unstructured fields (i.e., free-text) of EHRs. In this study we attempt to evaluate a practical text mining approach (i.e., advanced pattern matching techniques) in identifying phrases referring to housing issues, an important SDOH domain affecting value-based healthcare providers, using EHR of a large multispecialty medical group in the New England region, United States. To present how this approach would help the health systems to address the SDOH challenges of their patients we assess the demographic and clinical characteristics of patients with and without housing issues and briefly look into the patterns of healthcare utilization among the study population and for those with and without housing challenges. Methods: We identified five categories of housing issues [i.e., homelessness current (HC), homelessness history (HH), homelessness addressed (HA), housing instability (HI), and building quality (BQ)] and developed several phrases addressing each one through collaboration with SDOH experts, consulting the literature, and reviewing existing coding standards. We developed pattern-matching algorithms (i.e., advanced regular expressions), and then applied them in the selected EHR. We assessed the text mining approach for recall (sensitivity) and precision (positive predictive value) after comparing the identified phrases with manually annotated free-text for different housing issues. Results: The study dataset included EHR structured data for a total of 20,342 patients and 2,564,344 free-text clinical notes. The mean (SD) age in the study population was 75.96 (7.51). Additionally, 58.78% of the cohort were female. BQ and HI were the most frequent housing issues documented in EHR free-text notes and HH was the least frequent one. The regular expression methodology, when compared to manual annotation, had a high level of precision (positive predictive value) at phrase, note, and patient levels (96.36, 95.00, and 94.44%, respectively) across different categories of housing issues, but the recall (sensitivity) rate was relatively low (30.11, 32.20, and 41.46%, respectively). Conclusion: Results of this study can be used to advance the research in this domain, to assess the potential value of EHR's free-text in identifying patients with a high risk of housing issues, to improve patient care and outcomes, and to eventually mitigate socioeconomic disparities across individuals and communities.