Behind the Curtain: Comparing Predictive Models Performance in 2 Publicly Insured Populations.

INTRODUCTION: Predictive models have proliferated in the health system in recent years and have been used to predict both health services utilization and medical outcomes. Less is known, however, on how these models function and how they might adapt to different contexts. The purpose of the current study is to shed light on the inner workings of a large-scale predictive model deployed in 2 distinct populations, with a particular emphasis on adaptability issues. METHODS: We compared the performance and functioning of a predictive model of avoidable hospitalization in 2 very different populations: Medicaid and Medicare enrollees in Maryland. Specifically, we assessed characteristics of the risk scores from March 2022 for the 2 populations, the predictive ability of the scores, and the driving risk factors behind the scores. In addition, we created and assessed the performance of an "unadapted" model by applying coefficients from the Medicare model to the Medicaid population. RESULTS: The model adapted to, and performed well in, both populations, despite demographic differences in these 2 groups. However, the most salient risk factors and their relative weightings differed, sometimes dramatically, across the 2 populations. The unadapted Medicaid model displayed poor performance relative to the adapted model. CONCLUSIONS: Our findings speak to the need to "peek behind the curtain" of predictive models that may be applied to different populations, and we caution that risk prediction is not "one size fits all": for optimal performance, models should be adapted to, and trained on, the target population.

Facts about hospital-insurer contracting.

OBJECTIVES: To use publicly available price transparency data files to establish empirical regularities about hospital-insurer contracting. STUDY DESIGN: Retrospective analysis of 10 price transparency data files from HCA Healthcare. METHODS: Cross-sectional qualitative analysis of 524 hospital-insurer contracts across 10 hospitals. RESULTS: We ascertain 4 empirical regularities in these files. First, hospitals contract with many payers, ranging from 35 to 82 across the hospitals in the sample. Second, contract structure varies significantly within and across hospitals: Of the 524 contracts in our sample, the median contract contained 9 contract elements, whereas the mean contract contained 1285 contract elements. Third, most of the contracts in our sample contained multiple contracting methodologies (eg, both fixed fee and percentage of charges). Fourth, these contracts indicated substantial variation for the same service within and across hospitals, validating findings from analyses based on claims data and hospital price transparency files. CONCLUSIONS: Hospital-insurer contracts dictate the flow and structure of a significant portion of total health care expenditure in the US. Increased attention by both researchers and policy makers would lead to a greater understanding of this vital-yet understudied-element of the market for hospital services.

Comparison of Hospital Online Price and Telephone Price for Shoppable Services.

Importance: US hospitals are required to publicly post their prices for specified shoppable services online. However, the extent to which a hospital's prices posted online correlate with the prices they give to a telephone caller is unknown. Objective: To compare hospitals' online cash prices for vaginal childbirth and brain magnetic resonance imaging (MRI) with prices offered to secret shopper callers requesting price estimates by telephone. Design, Setting, and Participants: This cross-sectional study included cash online prices from each hospital's website for vaginal childbirth and brain MRI collected from representative US hospitals between August and October 2022. Thereafter, again between August and October 2022, simulated secret shopper patients called each hospital requesting their lowest cash price for these procedures. Main Outcomes and Measures: We calculated the difference between each hospital's online and phone prices for vaginal childbirth and brain MRI, and the Pearson correlation coefficient (r) between the online and phone prices for each procedure, among hospitals able to provide both prices. Results: A total of 60 representative US hospitals (20 top-ranked, 20 safety-net, and 20 non-top-ranked, non-safety-net hospitals) were included in the analysis. For vaginal childbirth, 63% (12 of 19) of top-ranked hospitals, 30% (6 of 20) of safety-net hospitals, and 21% (4 of 19) of non-top-ranked, non-safety-net hospitals provided both online and telephone prices. For brain MRI, 85% (17 of 20) of top-ranked hospitals, 50% (10 of 20) of safety-net hospitals, and 100% (20 of 20) of non-top-ranked, non-safety-net hospitals provided prices both online and via telephone. Online prices and telephone prices for both procedures varied widely. For example, online prices for vaginal childbirth posted by top-ranked hospitals ranged from $0 to $55 221 (mean, $23 040), from $4361 to $14 377 (mean $10 925) for safety-net hospitals, and from $1183 to $30 299 (mean $15 861) for non-top-ranked, non-safety-net hospitals. Among the 22 hospitals providing prices both online and by telephone for vaginal childbirth, prices were within 25% of each other for 45% (10) of hospitals, while 41% (9) of hospitals had differences of 50% or more (Pearson r = 0.118). Among the 47 hospitals providing both online and phone prices for brain MRI, prices were within 25% of each other for 66% (31) of hospitals), while 26% (n = 12) had differences of 50% or more (Pearson r = -0.169). Among hospitals that provided prices both online and via telephone, there was a complete match between the online and telephone prices for vaginal childbirth in 14% (3 of 22) of hospitals and for brain MRI in 19% (9 of 47) of hospitals. Conclusions and Relevance: Findings of this cross-sectional study suggest that there was poor correlation between hospitals' self-posted online prices and prices they offered by telephone to secret shoppers. These results demonstrate hospitals' continued problems in knowing and communicating their prices for specific services. The findings also highlight the continued challenges for uninsured patients and others who attempt to comparison shop for health care.

Outpatient apheresis billing: A photopheresis model shows that hospital price transparency data remain difficult to interpret.

OBJECTIVES: The US health care payment system is complex and difficult to interpret. Although federal regulations require that more data, in the form of charges and negotiated rates, be made available, compliance remains variable. We review chargemaster and negotiated rate values for extracorporeal photopheresis (ECP) to assess this variability. We sought to determine the availability of chargemaster and negotiated rates for health care consumers and to assess compliance and pricing among institutions using ECP as a model for apheresis billing. METHODS: We obtained ECP chargemaster data and negotiated rates from 20 institutions. We analyzed the availability of ECP chargemaster data and compared values with a previously published historic cohort. We evaluated the availability of negotiated rates and determined relative reimbursement using charge to reimbursement ratios. We determined calculated fines for hospitals based on bed size. RESULTS: Chargemaster availability increased from 2019 to 2022, though only 65% (13/20) of hospitals had both chargemaster and negotiated rate data. Chargemaster prices increased significantly from 2019 to 2022 (range, $3,586.83-$34,043.00). We reviewed 1,191 negotiated rates, with institutions averaging 93.6 different rates (SD, 189.5). Negotiated rates were variable, ranging from $3,586.83 to $34,043.00 per procedure. Reimbursement was higher among private insurers compared with reported Centers for Medicare & Medicaid Services negotiated rates. Of the 35% (7/20) that lacked chargemaster and negotiated rates, institutions faced an average annual fine of $1,430,800. CONCLUSIONS: Despite recent financial penalties, ECP pricing data are often unavailable or inadequate. Current available resources are unlikely to benefit the average health care consumer who requires ECP.

Assessing performance of ZCTA-level and Census Tract-level social and environmental risk factors in a model predicting hospital events.

Predictive analytics are used in primary care to efficiently direct health care resources to high-risk patients to prevent unnecessary health care utilization and improve health. Social determinants of health (SDOH) are important features in these models, but they are poorly measured in administrative claims data. Area-level SDOH can be proxies for unavailable individual-level indicators, but the extent to which the granularity of risk factors impacts predictive models is unclear. We examined whether increasing the granularity of area-based SDOH features from ZIP code tabulation area (ZCTA) to Census Tract strengthened an existing clinical prediction model for avoidable hospitalizations (AH events) in Maryland Medicare fee-for-service beneficiaries. We created a person-month dataset for 465,749 beneficiaries (59.4% female; 69.8% White; 22.7% Black) with 144 features indexing medical history and demographics using Medicare claims (September 2018 through July 2021). Claims data were linked with 37 SDOH features associated with AH events from 11 publicly-available sources (e.g., American Community Survey) based on the beneficiaries' ZCTA and Census Tract of residence. Individual AH risk was estimated using six discrete time survival models with different combinations of demographic, condition/utilization, and SDOH features. Each model used stepwise variable selection to retain only meaningful predictors. We compared model fit, predictive performance, and interpretation across models. Results showed that increasing the granularity of area-based risk factors did not dramatically improve model fit or predictive performance. However, it did affect model interpretation by altering which SDOH features were retained during variable selection. Further, the inclusion of SDOH at either granularity level meaningfully reduced the risk that was attributed to demographic predictors (e.g., race, dual-eligibility for Medicaid). Differences in interpretation are critical given that this model is used by primary care staff to inform the allocation of care management resources, including those available to address drivers of health beyond the bounds of traditional health care.

Predicting Hospital Readmissions in a Commercially Insured Population over Varying Time Horizons.

BACKGROUND: Reducing hospital readmissions is a federal policy priority, and predictive models of hospital readmissions have proliferated in recent years; however, most such models tend to focus on the 30-day readmission time horizon and do not consider readmission over shorter (or longer) windows. OBJECTIVES: To evaluate the performance of a predictive model of hospital readmissions over three different readmission timeframes in a commercially insured population. DESIGN: Retrospective multivariate logistic regression with an 80/20 train/test split. PARTICIPANTS: A total of 2,213,832 commercially insured inpatient admissions from 2016 to 2017 comprising 782,768 unique patients from the Health Care Cost Institute. MAIN MEASURES: Outcomes are readmission within 14 days, 15-30 days, and 31-60 days from discharge. Predictor variables span six different domains: index admission, condition history, demographic, utilization history, pharmacy, and environmental controls. KEY RESULTS: Our model generates C-statistics for holdout samples ranging from 0.618 to 0.915. The model's discriminative power declines with readmission time horizon: discrimination for readmission predictions within 14 days following discharge is higher than for readmissions 15-30 days following discharge, which in turn is higher than predictions 31-60 days following discharge. Additionally, the model's predictive power increases nonlinearly with the inclusion of successive risk factor domains: patient-level measures of utilization and condition history add substantially to the discriminative power of the model, while demographic information, pharmacy utilization, and environmental risk factors add relatively little. CONCLUSION: It is more difficult to predict distant readmissions than proximal readmissions, and the more information the model uses, the better the predictions. Inclusion of utilization-based risk factors add substantially to the discriminative ability of the model, much more than any other included risk factor domain. Our best-performing models perform well relative to other published readmission prediction models. It is possible that these predictions could have operational utility in targeting readmission prevention interventions among high-risk individuals.

Designing optimal recommended budgeting thresholds for a Medicaid program.

OBJECTIVES: To develop and test a methodology for optimally setting automatic auditing thresholds to minimize administrative costs without encouraging overall budget growth in a state Medicaid program. STUDY DESIGN: Two-stage optimization using administrative Maryland Medicaid plan-of-service data from fiscal year (FY) 2019. METHODS: In the first stage, we use an unsupervised machine learning method to regroup acuity levels so that plans of service with similar spending profiles are grouped together. Then, using these regroupings, we employ numerical optimization to estimate the recommended budget levels that could minimize the number of audits across those groupings. We simulate the effects of this proposed methodology on FY 2019 plans of service and compare the resulting number of simulated audits with actual experience. RESULTS: Using optimal regrouping and numerical optimization, this method could reduce the number of audits by 10.4% to 36.7% relative to the status quo, depending on the search space parameters. This reduction is a result of resetting recommended budget levels across acuity groupings, with no anticipated increase in the total recommended budget amount across plans of service. These reductions are driven, in general, by an increase in recommended budget level for acuity groupings with low variance in plan-of-service spending and a reduction in recommended budget level for acuity groupings with high variance in plan-of-service spending. CONCLUSIONS: Using machine learning and optimization methods, it is possible to design recommended budget thresholds that could lead to significant reductions in administrative burden without encouraging overall cost growth.

Hospital And Regional Characteristics Associated With Emergency Department Facility Fee Cash Pricing.

Self-pay patients are an understudied yet important and financially vulnerable population of emergency department (ED) users. As ED facility fees may be a key cost driver in patient ED bills, we leveraged newly available hospital pricing data to describe ED facility fees for self-pay patients (cash prices) and how they vary according to hospital and regional characteristics in a sample of 1,621 hospitals across the United States. The median cash price for ED facility fees ranged from $160.78 for a level 1 visit to $1,097.43 for a level 5 visit. Hospital for-profit status and a bed count of 251 or more beds were associated with higher cash prices for ED facility fees across all visit levels. Meanwhile, location in a county with a poverty rate of 16 percent or more was correlated with lower facility fee cash prices for ED visit levels 2 and up. We hope that these findings can inform targeted policy efforts to better ensure affordable ED care for vulnerable patients.

Predicting avoidable hospital events in Maryland.

OBJECTIVE: To develop and validate a prediction model of avoidable hospital events among Medicare fee-for-service (FFS) beneficiaries in Maryland. DATA SOURCES: Medicare FFS claims from Maryland from 2017 to 2020 and other publicly available ZIP code-level data sets. STUDY DESIGN: Multivariable logistic regression models were used to estimate the relationship between a variety of risk factors and future avoidable hospital events. The predictive power of the resulting risk scores was gauged using a concentration curve. DATA COLLECTION/EXTRACTION METHODS: One hundred and ninety-eight individual- and ZIP code-level risk factors were used to create an analytic person-month data set of over 11.6 million person-month observations. PRINCIPAL FINDINGS: We included 198 risk factors for the model based on the results of a targeted literature review, both at the individual and neighborhood levels. These risk factors span six domains as follows: diagnoses, pharmacy utilization, procedure history, prior utilization, social determinants of health, and demographic information. Feature selection retained 73 highly statistically significant risk factors (p < 0.0012) in the primary model. Risk scores were estimated for each individual in the cohort, and, for scores released in April 2020, the top 10% riskiest individuals in the cohort account for 48.7% of avoidable hospital events in the following month. These scores significantly outperform the Centers for Medicare & Medicaid Services hierarchical condition category risk scores in terms of predictive power. CONCLUSIONS: A risk prediction model based on standard administrative claims data can identify individuals at risk of incurring a future avoidable hospital event with good accuracy.

Identifying Symptom Information in Clinical Notes Using Natural Language Processing.

BACKGROUND: Symptoms are a core concept of nursing interest. Large-scale secondary data reuse of notes in electronic health records (EHRs) has the potential to increase the quantity and quality of symptom research. However, the symptom language used in clinical notes is complex. A need exists for methods designed specifically to identify and study symptom information from EHR notes. OBJECTIVES: We aim to describe a method that combines standardized vocabularies, clinical expertise, and natural language processing to generate comprehensive symptom vocabularies and identify symptom information in EHR notes. We piloted this method with five diverse symptom concepts: constipation, depressed mood, disturbed sleep, fatigue, and palpitations. METHODS: First, we obtained synonym lists for each pilot symptom concept from the Unified Medical Language System. Then, we used two large bodies of text (clinical notes from Columbia University Irving Medical Center and PubMed abstracts containing Medical Subject Headings or key words related to the pilot symptoms) to further expand our initial vocabulary of synonyms for each pilot symptom concept. We used NimbleMiner, an open-source natural language processing tool, to accomplish these tasks and evaluated NimbleMiner symptom identification performance by comparison to a manually annotated set of nurse- and physician-authored common EHR note types. RESULTS: Compared to the baseline Unified Medical Language System synonym lists, we identified up to 11 times more additional synonym words or expressions, including abbreviations, misspellings, and unique multiword combinations, for each symptom concept. Natural language processing system symptom identification performance was excellent. DISCUSSION: Using our comprehensive symptom vocabularies and NimbleMiner to label symptoms in clinical notes produced excellent performance metrics. The ability to extract symptom information from EHR notes in an accurate and scalable manner has the potential to greatly facilitate symptom science research.

How Well Do Automated Linking Methods Perform? Lessons from U.S. Historical Data.

This paper reviews the literature in historical record linkage in the U.S. and examines the performance of widely-used record linking algorithms and common variations in their assumptions. We use two high-quality, hand-linked datasets and one synthetic ground truth to examine the direct effects of linking algorithms on data quality. We find that (1) no algorithm (including hand-linking) consistently produces representative samples; (2) 15 to 37 percent of links chosen by widely-used algorithms are classified as errors by trained human reviewers; and (3) false links are systematically related to baseline sample characteristics, showing that some algorithms may induce systematic measurement error into analyses. A case study shows that the combined effects of (1)-(3) attenuate estimates of the intergenerational income elasticity by up to 20 percent, and common variations in algorithm assumptions result in greater attenuation. As current practice moves to automate linking and increase link rates, these results highlight the important potential consequences of linking errors on inferences with linked data. We conclude with constructive suggestions for reducing linking errors and directions for future research.

Functions that Protect Escherichia coli from Tightly Bound DNA-Protein Complexes Created by Mutant EcoRII Methyltransferase.

Expression of mutant EcoRII methyltransferase protein (M.EcoRII-C186A) in Escherichia coli leads to tightly bound DNA-protein complexes (TBCs), located sporadically on the chromosome rather than in tandem arrays. The mechanisms behind the lethality induced by such sporadic TBCs are not well studied, nor is it clear whether very tight binding but non-covalent complexes are processed in the same way as covalent DNA-protein crosslinks (DPCs). Using 2D gel electrophoresis, we found that TBCs induced by M.EcoRII-C186A block replication forks in vivo. Specific bubble molecules were detected as spots on the 2D gel, only when M.EcoRII-C186A was induced, and a mutation that eliminates a specific EcoRII methylation site led to disappearance of the corresponding spot. We also performed a candidate gene screen for mutants that are hypersensitive to TBCs induced by M.EcoRII-C186A. We found several gene products necessary for protection against these TBCs that are known to also protect against DPCs induced with wild-type M.EcoRII (after 5-azacytidine incorporation): RecA, RecBC, RecG, RuvABC, UvrD, FtsK, XerCD and SsrA (tmRNA). In contrast, the RecFOR pathway and Rep helicase are needed for protection against TBCs but not DPCs induced by M.EcoRII. We propose that stalled fork processing by RecFOR and RecA promotes release of tightly bound (but non-covalent) blocking proteins, perhaps by licensing Rep helicase-driven dissociation of the blocking M.EcoRII-C186A. Our studies also argued against the involvement of several proteins that might be expected to protect against TBCs. We took the opportunity to directly compare the sensitivity of all tested mutants to two quinolone antibiotics, which target bacterial type II topoisomerases and induce a unique form of DPC. We uncovered rep, ftsK and xerCD as novel quinolone hypersensitive mutants, and also obtained evidence against the involvement of a number of functions that might be expected to protect against quinolones.