Developing a Model to Predict High Health Care Utilization Among Patients in a New York City Safety Net System.

BACKGROUND: Health care facilities use predictive models to identify patients at risk of high future health care utilization who may benefit from tailored interventions. Previous predictive models that have focused solely on inpatient readmission risk, relied on commercial insurance claims data, or failed to incorporate social determinants of health may not be generalizable to safety net hospital populations. To address these limitations, we developed a payer-agnostic risk model for patients receiving care at the largest US safety net hospital system. METHODS: We transformed electronic health record and administrative data from 833,969 adult patients who received care during July 2016-July 2017 into demographic, utilization, diagnosis, medication, and social determinant variables (including homelessness and incarceration history) to predict health care utilization during the following year.We selected the final model by developing and validating multiple classification and regression models predicting 10+ acute days, 5+ acute days, or continuous acute days. We compared a portfolio of performance metrics while prioritizing positive predictive value for patients whose predicted utilization was among the top 1% to maximize clinical utility. RESULTS: The final model predicted continuous number of acute days and included 17 variables. For the top 1% of high acute care utilizers, the model had a positive predictive value of 47.6% and sensitivity of 17.3%. Previous health care utilization and psychosocial factors were the strongest predictors of future high acute care utilization. CONCLUSIONS: We demonstrated a feasible approach to predictive high acute care utilization in a safety net hospital using electronic health record data while incorporating social risk factors.

Social distancing and mask-wearing could avoid recurrent stay-at-home restrictions during COVID-19 respiratory pandemic in New York City.

Stay-at-home restrictions such as closure of non-essential businesses were effective at reducing SARS-CoV-2 transmission in New York City (NYC) in the spring of 2020. Relaxation of these restrictions was desirable for resuming economic and social activities, but could only occur in conjunction with measures to mitigate the expected resurgence of new infections, in particular social distancing and mask-wearing. We projected the impact of individuals' adherence to social distancing and mask-wearing on the duration, frequency, and recurrence of stay-at-home restrictions in NYC. We applied a stochastic discrete time-series model to simulate community transmission and household secondary transmission in NYC. The model was calibrated to hospitalizations, ICU admissions, and COVID-attributable deaths over March-July 2020 after accounting for the distribution of age and chronic health conditions in NYC. We projected daily new infections and hospitalizations up to May 31, 2021 under the different levels of adherence to social distancing and mask-wearing after relaxation of stay-at-home restrictions. We assumed that the relaxation of stay-at-home policies would occur in the context of adaptive reopening, where a new hospitalization rate of ≥ 2 per 100,000 residents would trigger reinstatement of stay-at-home restrictions while a new hospitalization rate of ≤ 0.8 per 100,000 residents would trigger relaxation of stay-at-home restrictions. Without social distancing and mask-wearing, simulated relaxation of stay-at-home restrictions led to epidemic resurgence and necessary reinstatement of stay-at-home restrictions within 42 days. NYC would have stayed fully open for 26% of the time until May 31, 2021, alternating reinstatement and relaxation of stay-at-home restrictions in four cycles. At a low (50%) level of adherence to mask-wearing, NYC would have needed to implement stay-at-home restrictions between 8% and 32% of the time depending on individual adherence to social distancing. At moderate to high levels of adherence to mask-wearing without social distancing, NYC would have needed to implement stay-at-home restrictions. In threshold analyses, avoiding reinstatement of stay-at-home restrictions required a minimum of 60% adherence to mask-wearing at 50% adherence to social distancing. With low adherence to mask-wearing and social distancing, reinstatement of stay-at-home restrictions in NYC was inevitable. High levels of adherence to social distancing and mask-wearing could have attributed to avoiding recurrent surges without reinstatement of stay-at-home restrictions.

Surge and Mortality in ICUs in New York City's Public Healthcare System.

OBJECTIVES: To evaluate the impact of ICU surge on mortality and to explore clinical and sociodemographic predictors of mortality. DESIGN: Retrospective cohort analysis. SETTING: NYC Health + Hospitals ICUs. PATIENTS: Adult ICU patients with coronavirus disease 2019 admitted between March 24, and May 12, 2020. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Hospitals reported surge levels daily. Uni- and multivariable analyses were conducted to assess factors impacting in-hospital mortality. Mortality in Hispanic patients was higher for high/very high surge compared with low/medium surge (69.6% vs 56.4%; p = 0.0011). Patients 65 years old and older had similar mortality across surge levels. Mortality decreased from high/very high surge to low/medium surge in, patients 18-44 years old and 45-64 (18-44 yr: 46.4% vs 27.3%; p = 0.0017 and 45-64 yr: 64.9% vs 53.2%; p = 0.002), and for medium, high, and very high poverty neighborhoods (medium: 69.5% vs 60.7%; p = 0.019 and high: 71.2% vs 59.7%; p = 0.0078 and very high: 66.6% vs 50.7%; p = 0.0003). In the multivariable model high surge (high/very high vs low/medium odds ratio, 1.4; 95% CI, 1.2-1.8), race/ethnicity (Black vs White odds ratio, 1.5; 95% CI, 1.1-2.0 and Asian vs White odds ratio 1.5; 95% CI, 1.0-2.3; other vs White odds ratio 1.5, 95% CI, 1.0-2.3), age (45-64 vs 18-44 odds ratio, 2.0; 95% CI, 1.6-2.5 and 65-74 vs 18-44 odds ratio, 5.1; 95% CI, 3.3-8.0 and 75+ vs 18-44 odds ratio, 6.8; 95% CI, 4.7-10.1), payer type (uninsured vs commercial/other odds ratio, 1.7; 95% CI, 1.2-2.3; medicaid vs commercial/other odds ratio, 1.3; 95% CI, 1.1-1.5), neighborhood poverty (medium vs low odds ratio 1.6, 95% CI, 1.0-2.4 and high vs low odds ratio, 1.8; 95% CI, 1.3-2.5), comorbidities (diabetes odds ratio, 1.6; 95% CI, 1.2-2.0 and asthma odds ratio, 1.4; 95% CI, 1.1-1.8 and heart disease odds ratio, 2.5; 95% CI, 2.0-3.3), and interventions (mechanical ventilation odds ratio, 8.8; 95% CI, 6.1-12.9 and dialysis odds ratio, 3.0; 95% CI, 1.9-4.7) were significant predictors for mortality. CONCLUSIONS: Patients admitted to ICUs with higher surge scores were at greater risk of death. Impact of surge levels on mortality varied across sociodemographic groups.

Generalizability of Indicators from the New York City Macroscope Electronic Health Record Surveillance System to Systems Based on Other EHR Platforms.

INTRODUCTION: The New York City (NYC) Macroscope is an electronic health record (EHR) surveillance system based on a distributed network of primary care records from the Hub Population Health System. In a previous 3-part series published in eGEMS, we reported the validity of health indicators from the NYC Macroscope; however, questions remained regarding their generalizability to other EHR surveillance systems. METHODS: We abstracted primary care chart data from more than 20 EHR software systems for 142 participants of the 2013-14 NYC Health and Nutrition Examination Survey who did not contribute data to the NYC Macroscope. We then computed the sensitivity and specificity for indicators, comparing data abstracted from EHRs with survey data. RESULTS: Obesity and diabetes indicators had moderate to high sensitivity (0.81-0.96) and high specificity (0.94-0.98). Smoking status and hypertension indicators had moderate sensitivity (0.78-0.90) and moderate to high specificity (0.88-0.98); sensitivity improved when the sample was restricted to records from providers who attested to Stage 1 Meaningful Use. Hyperlipidemia indicators had moderate sensitivity (≥0.72) and low specificity (≤0.59), with minimal changes when restricting to Stage 1 Meaningful Use. DISCUSSION: Indicators for obesity and diabetes used in the NYC Macroscope can be adapted to other EHR surveillance systems with minimal validation. However, additional validation of smoking status and hypertension indicators is recommended and further development of hyperlipidemia indicators is needed. CONCLUSION: Our findings suggest that many of the EHR-based surveillance indicators developed and validated for the NYC Macroscope are generalizable for use in other EHR surveillance systems.

Characterizing Adults Receiving Primary Medical Care in New York City: Implications for Using Electronic Health Records for Chronic Disease Surveillance.

INTRODUCTION: Electronic health records (EHRs) from primary care providers can be used for chronic disease surveillance; however, EHR-based prevalence estimates may be biased toward people who seek care. This study sought to describe the characteristics of an in-care population and compare them with those of a not-in-care population to inform interpretation of EHR data. METHODS: We used data from the 2013-2014 New York City Health and Nutrition Examination Survey (NYC HANES), considered the gold standard for estimating disease prevalence, and the 2013 Community Health Survey, and classified participants as in care or not in care, on the basis of their report of seeing a health care provider in the previous year. We used χ(2) tests to compare the distribution of demographic characteristics, health care coverage and access, and chronic conditions between the 2 populations. RESULTS: According to the Community Health Survey, approximately 4.1 million (71.7%) adults aged 20 or older had seen a health care provider in the previous year; according to NYC HANES, approximately 4.7 million (75.1%) had. In both surveys, the in-care population was more likely to be older, female, non-Hispanic, and insured compared with the not-in-care population. The in-care population from the NYC HANES also had a higher prevalence of diabetes (16.7% vs 6.9%; P < .001), hypercholesterolemia (35.7% vs 22.3%; P < .001), and hypertension (35.5% vs 26.4%; P < .001) than the not-in-care population. CONCLUSION: Systematic differences between in-care and not-in-care populations warrant caution in using primary care data to generalize to the population at large. Future efforts to use primary care data for chronic disease surveillance need to consider the intended purpose of data collected in these systems as well as the characteristics of the population using primary care.

Design of the New York City Macroscope: Innovations in Population Health Surveillance Using Electronic Health Records.

INTRODUCTION: Electronic health records (EHRs) have the potential to offer real-time, inexpensive standardized health data about chronic health conditions. Despite rapid expansion, EHR data evaluations for chronic disease surveillance have been limited. We present design and methods for the New York City (NYC) Macroscope, an EHR-based chronic disease surveillance system. This methods report is the first in a three part series describing the development and validation of the NYC Macroscope. This report describes in detail the infrastructure underlying the NYC Macroscope; indicator definitions; design decisions that were made to maximize data quality; characteristics of the population sampled; completeness of data collected; and lessons learned from doing this work. The second report describes the methods used to evaluate the validity and robustness of NYC Macroscope prevalence estimates; presents validation results for estimates of obesity, smoking, depression and influenza vaccination; and discusses the implications of our findings for NYC and for other jurisdictions embarking on similar work. The third report applies the same validation methods to metabolic outcomes, including the prevalence, treatment and control of diabetes, hypertension and hyperlipidemia. METHODS: We designed the NYC Macroscope for comparison to a local "gold standard," the 2013-14 NYC Health and Nutrition Examination Survey, and the telephonic 2013 Community Health Survey. NYC Macroscope indicators covered prevalence, treatment, and control of diabetes, hypertension, and hyperlipidemia; and prevalence of influenza vaccination, obesity, depression and smoking. Indicators were stratified by age, sex, and neighborhood poverty, and weighted to the in-care NYC population and limited to primary care patients. Indicator queries were distributed to a virtual network of primary care practices; 392 practices and 716,076 adult patients were retained in the final sample. FINDINGS: The NYC Macroscope covered 10% of primary care providers and 15% of all adult patients in NYC in 2013 (8-47% of patients by neighborhood). Data completeness varied by domain from 98% for blood pressure among patients with hypertension to 33% for depression screening. DISCUSSION: Design and validation efforts undertaken by NYC are described here to provide one potential blueprint for leveraging EHRs for population health monitoring. To replicate a model like NYC Macroscope, jurisdictions should establish buy-in; build informatics capacity; use standard, simple case defnitions; establish documentation quality thresholds; restrict to primary care providers; and weight the sample to a target population.

Monitoring Prevalence, Treatment, and Control of Metabolic Conditions in New York City Adults Using 2013 Primary Care Electronic Health Records: A Surveillance Validation Study.

INTRODUCTION: Electronic health records (EHRs) can potentially extend chronic disease surveillance, but few EHR-based initiatives tracking population-based metrics have been validated for accuracy. We designed a new EHR-based population health surveillance system for New York City (NYC) known as NYC Macroscope. This report is the third in a 3-part series describing the development and validation of that system. The first report describes governance and technical infrastructure underlying the NYC Macroscope. The second report describes validation methods and presents validation results for estimates of obesity, smoking, depression and influenza vaccination. In this third paper we present validation findings for metabolic indicators (hypertension, hyperlipidemia, diabetes). METHODS: We compared EHR-based estimates to those from a gold standard surveillance source - the 2013-2014 NYC Health and Nutrition Examination Survey (NYC HANES) - overall and stratified by sex and age group, using the two one-sided test of equivalence and other validation criteria. RESULTS: EHR-based hypertension prevalence estimates were highly concordant with NYC HANES estimates. Diabetes prevalence estimates were highly concordant when measuring diagnosed diabetes but less so when incorporating laboratory results. Hypercholesterolemia prevalence estimates were less concordant overall. Measures to assess treatment and control of the 3 metabolic conditions performed poorly. DISCUSSION: While indicator performance was variable, findings here confirm that a carefully constructed EHR-based surveillance system can generate prevalence estimates comparable to those from gold-standard examination surveys for certain metabolic conditions such as hypertension and diabetes. CONCLUSIONS: Standardized EHR metrics have potential utility for surveillance at lower annual costs than surveys, especially as representativeness of contributing clinical practices to EHR-based surveillance systems increases.

Can Electronic Health Records Be Used for Population Health Surveillance? Validating Population Health Metrics Against Established Survey Data.

INTRODUCTION: Electronic health records (EHRs) offer potential for population health surveillance but EHR-based surveillance measures require validation prior to use. We assessed the validity of obesity, smoking, depression, and influenza vaccination indicators from a new EHR surveillance system, the New York City (NYC) Macroscope. This report is the second in a 3-part series describing the development and validation of the NYC Macroscope. The first report describes in detail the infrastructure underlying the NYC Macroscope; design decisions that were made to maximize data quality; characteristics of the population sampled; completeness of data collected; and lessons learned from doing this work. This second report, which addresses concerns related to sampling bias and data quality, describes the methods used to evaluate the validity and robustness of NYC Macroscope prevalence estimates; presents validation results for estimates of obesity, smoking, depression and influenza vaccination; and discusses the implications of our findings for NYC and for other jurisdictions embarking on similar work. The third report applies the same validation methods described in this report to metabolic outcomes, including the prevalence, treatment and control of diabetes, hypertension and hyperlipidemia. METHODS: NYC Macroscope prevalence estimates, overall and stratified by sex and age group, were compared to reference survey estimates for adult New Yorkers who reported visiting a doctor in the past year. Agreement was evaluated against 5 a priori criteria. Sensitivity and specificity were assessed by examining individual EHR records in a subsample of 48 survey participants. RESULTS: Among adult New Yorkers in care, the NYC Macroscope prevalence estimate for smoking (15.2%) fell between estimates from NYC HANES (17.7 %) and CHS (14.9%) and met all 5 a priori criteria. The NYC Macroscope obesity prevalence estimate (27.8%) also fell between the NYC HANES (31.3%) and CHS (24.7%) estimates, but met only 3 a priori criteria. Sensitivity and specificity exceeded 0.90 for both the smoking and obesity indicators. The NYC Macroscope estimates of depression and influenza vaccination prevalence were more than 10 percentage points lower than the estimates from either reference survey. While specificity was > 0.90 for both of these indicators, sensitivity was < 0.70. DISCUSSION: Through this work we have demonstrated that EHR data from a convenience sample of providers can produce acceptable estimates of smoking and obesity prevalence among adult New Yorkers in care; gained a better understanding of the challenges involved in estimating depression prevalence from EHRs; and identified areas for additional research regarding estimation of influenza vaccination prevalence. We have also shared lessons learned about how EHR indicators should be constructed and offer methodologic suggestions for validating them. CONCLUSIONS: This work adds to a rapidly emerging body of literature about how to define, collect and interpret EHR-based surveillance measures and may help guide other jurisdictions.

The state of population health surveillance using electronic health records: a narrative review.

Electronic health records (EHRs) are transforming the practice of clinical medicine, but the extent to which they are being harnessed to advance public health goals remains uncertain. Data extracted from integrated EHR networks offer the potential for almost real-time determination of the health status of populations in care, for targeting interventions to vulnerable populations, and for monitoring the impact of such initiatives over time. This is especially true in ambulatory care settings, which are uniquely suited for monitoring population health indicators including risk factors and disease management indicators associated with chronic diseases. As efforts gather steam to integrate health data across delivery systems, large networks of electronic patient information are increasingly emerging. Few of the national population health surveillance systems that rely on EHR data have progressed beyond laying groundwork to launch and maintain EHR-based surveillance, but a limited number of more focused or local efforts have demonstrated innovation in population health surveillance. Common challenges include incompleteness of population coverage, lack of interoperability across data systems, and variable data quality. This review defines progress, opportunities, and challenges in using EHR data for population health surveillance.

Evaluating the 2010 New York State HIV testing law in NYC ambulatory practices using electronic health records.

INTRODUCTION: In 2010, the New York State Legislature made it mandatory to offer an HIV test to people aged 13-64 years receiving hospital or primary care services, with limited exceptions. In this study, we used data from New York City practices to evaluate the impact of the law on HIV testing rates in ambulatory care. METHODS: We collected quarterly testing data from the electronic health records of 218 practices. We calculated overall and stratified crude testing rates. Using univariate and multivariate generalized estimating equation models, we assessed the odds of testing in the year before the law (baseline) versus the first and second year after the law's implementation (year 1 and year 2). RESULTS: During baseline, the odds of testing did not increase significantly. During year 1, the odds of testing significantly increased by 50% in the univariate model and 200% after adjusting for confounders. During year 2, the odds of testing increased 10%. This was only significant in the univariate model. The crude quarterly testing rate increased from 2.8% to 5.7% from baseline to year 2. CONCLUSIONS: Our evaluation showed that after the implementation of the HIV testing law, there was an increase in HIV testing among NYC ambulatory practices. Testing rates remained modest, but considerable improvement was seen in community health centers, in age ranges targeted by the law and in practices that were screening for HIV at baseline. This study suggests that legislation may be effective when used in a comprehensive prevention strategy.

Assessing capacity and disease burden in a virtual network of New York City primary care providers following Hurricane Sandy.

Urban contexts introduce unique challenges that must be addressed to ensure that areas of high population density can function when disasters occur. The ability to generate useful data to guide decision-making is critical in this context. Widespread adoption of electronic health record (EHR) systems in recent years has created electronic data sources and networks that may play an important role in public health surveillance efforts, including in post-disaster situations. The Primary Care Information Project (PCIP) at the New York City Department of Health and Mental Hygiene has partnered with local clinicians to establish an electronic data system, and this network provides infrastructure to support primary care surveillance activities in New York City. After Hurricane Sandy, PCIP generated several sets of data to contribute to the city's efforts to assess the impact of the storm, including daily connectivity data to establish practice operations, data to examine patterns of primary care utilization in severely affected and less affected areas, and data on the frequency of respiratory infection diagnosis in the primary care setting. Daily patient visit data from three heavily affected neighborhoods showed the health department where primary care capacity was most affected in the weeks following Sandy. Overall transmission data showed that practices in less affected areas were quicker to return to normal reporting patterns, while those in more affected areas did not resume normal data transmissions for a few months. Rates of bronchitis increased after Sandy compared to the two prior years; while this was most likely attributable to a more severe flu season, it demonstrates the capacity of primary care networks to pick up on these types of post-emergency trends. Hurricane Sandy was the first disaster situation where PCIP was asked to assess public health impact, generating information that could contribute to aid and recovery efforts. This experience allowed us to explore the strengths and weaknesses of ambulatory EHR data in post-disaster settings. Data from ambulatory EHR networks can augment existing surveillance streams by providing sentinel population snapshots on clinically available indicators in near real time.