Development and Validation of a Hospital Indicator of Activity-Based Costs for Injury Admissions.

OBJECTIVES: To develop a hospital indicator of resource use for injury admissions. METHODS: We focused on resource use for acute injury care and therefore adopted a hospital perspective. We included patients ≥16 years old with an Injury Severity Score >9 admitted to any of the 57 trauma centers of an inclusive Canadian trauma system from 2014 to 2018. We extracted data from the trauma registry and hospital financial reports and estimated resource use with activity-based costing. We developed risk-adjustment models by trauma center designation level (I/II and III/IV) for the whole sample, traumatic brain injuries, thoraco-abdominal injuries, orthopedic injuries, and patients ≥65 years old. Candidate variables were selected using bootstrap resampling. We performed benchmarking by comparing the adjusted mean cost in each center, obtained using shrinkage estimates, to the provincial mean. RESULTS: We included 38 713 patients. The models explained between 12% and 36% (optimism-corrected r2) of the variation in resource use. In the whole sample and in all subgroups, we identified centers with higher- or lower-than-expected resource use across level I/II and III/IV centers. CONCLUSIONS: We propose an algorithm to produce the indicator using data routinely collected in trauma registries to prompt targeted exploration of potential areas for improvement in resource use for injury admissions. The r2 of our models suggest that between 64% and 88% of the variation in resource use for injury care is dictated by factors other than patient baseline risk.

Inadequate Risk Adjustment Impacts Geriatricians' Performance on Medicare Cost and Quality Measures.

OBJECTIVES: Medicare value-based payment programs evaluate physicians' performance on their patients' annual Medicare costs and clinical outcomes. However, little is known about how geriatricians, who disproportionately provide care for medically complex older adults, perform on these measures. DESIGN: A retrospective study using multivariable regression methods to estimate the association of geriatric risk factors with annualized Medicare costs and preventable hospitalization rates and to compare geriatricians' performance on these outcomes to other primary care physicians (PCPs) under standard Medicare risk adjustment and after adding additional adjustment for geriatric risk factors. SETTING: Eight years (2006-2013) of cohort data from the Medicare Current Beneficiary Survey. PARTICIPANTS: Medicare beneficiaries, aged 65 years and older, with primary care services contributing 27 027 person-years of data. MEASUREMENTS: Outcomes were costs and preventable hospitalization rates; geriatric risk factors were patient frailty, long-term institutionalization, dementia, and depression. RESULTS: Geriatricians were more likely to care for patients with frailty (22.8% vs 14.1%), long-term institutionalization (12.0% vs 4.7%), dementia (21.6% vs 10.2%), and depression (23.6% vs 17.4%) than other PCPs (P < .001 for each). Under standard Medicare risk adjustment, geriatricians performed more poorly on costs compared to other PCPs (observed-expected [O-E] ratio = 1.24 vs 0.99) and preventable hospitalizations (O-E ratio = 1.16 vs 0.98). Adding frailty, institutionalization, dementia, and depression to risk adjustment improved geriatricians' performance on costs by 25% and on preventable hospitalization rates by 35%, relative to other PCPs. Concurrent-year risk prediction that removed the influence of unpredictable acute events further improved geriatricians' performance vs other PCPs (O-E ratio = 0.99 vs 1.00). CONCLUSION: Medicare should consider risk adjusting for frailty, long-term institutionalization, dementia, and depression to avoid inappropriately penalizing geriatricians who care for vulnerable older adults. J Am Geriatr Soc 68:297-304, 2020.

Problematic Risk Adjustment in National Healthcare Safety Network Measures.

This article reviews the risk-adjustment models underpinning the National Healthcare Safety Network (NHSN) standardized infection ratios. After first describing the models, the authors focus on hospital intensive care unit (ICU) designation as a variable employed across the various risk models. The risk-adjusted frequency with which ICU services are reported in Medicare fee-for-service claims data was compared as a proxy for determining whether reporting of ICU days is similar across hospitals. Extreme variation was found in the reporting of ICU utilization among admissions for congestive heart failure, ranging from 25% in the lowest admission hospital quartile to 95% in the highest. The across-hospital variation in reported ICU utilization was found to be unrelated to patient severity. Given that such extreme variation appears in a designation of ICU versus non-ICU utilization, the NHSN risk-adjustment models' dependence on nursing unit designation should be a cause for concern.

Imperfect risk adjustment, risk preferences, and sorting in competitive health insurance markets.

I develop a model of insurer price-setting and consumer welfare under risk-adjustment, a policy commonly used to combat inefficient sorting due to adverse selection in health insurance markets. I use the model to illustrate graphically that risk-adjustment causes health plan prices to be based on costs not predicted by the risk-adjustment model ("residual costs") rather than total costs, either weakening or exacerbating selection problems depending on the correlation between demand and costs predicted by the risk-adjustment model. I then use a structural model to estimate the welfare consequences of risk-adjustment, finding a welfare gain of over $600 per person-year.

Development of a Risk-adjustment Model for the Inpatient Rehabilitation Facility Discharge Self-care Functional Status Quality Measure.

BACKGROUND: Functional status measures are important patient-centered indicators of inpatient rehabilitation facility (IRF) quality of care. We developed a risk-adjusted self-care functional status measure for the IRF Quality Reporting Program. This paper describes the development and performance of the measure's risk-adjustment model. METHODS: Our sample included IRF Medicare fee-for-service patients from the Centers for Medicare & Medicaid Services' 2008-2010 Post-Acute Care Payment Reform Demonstration. Data sources included the Continuity Assessment Record and Evaluation Item Set, IRF-Patient Assessment Instrument, and Medicare claims. Self-care scores were based on 7 Continuity Assessment Record and Evaluation items. The model was developed using discharge self-care score as the dependent variable, and generalized linear modeling with generalized estimation equation to account for patient characteristics and clustering within IRFs. Patient demographics, clinical characteristics at IRF admission, and clinical characteristics related to the recent hospitalization were tested as risk adjusters. RESULTS: A total of 4769 patient stays from 38 IRFs were included. Approximately 57% of the sample was female; 38.4%, 75-84 years; and 31.0%, 65-74 years. The final model, containing 77 risk adjusters, explained 53.7% of variance in discharge self-care scores (P<0.0001). Admission self-care function was the strongest predictor, followed by admission cognitive function and IRF primary diagnosis group. The range of expected and observed scores overlapped very well, with little bias across the range of predicted self-care functioning. CONCLUSIONS: Our risk-adjustment model demonstrated strong validity for predicting discharge self-care scores. Although the model needs validation with national data, it represents an important first step in evaluation of IRF functional outcomes.

The Impact of Hospital Size on CMS Hospital Profiling.

BACKGROUND: The Centers for Medicare & Medicaid Services (CMS) profile hospitals using a set of 30-day risk-standardized mortality and readmission rates as a basis for public reporting. These measures are affected by hospital patient volume, raising concerns about uniformity of standards applied to providers with different volumes. OBJECTIVES: To quantitatively determine whether CMS uniformly profile hospitals that have equal performance levels but different volumes. RESEARCH DESIGN: Retrospective analysis of patient-level and hospital-level data using hierarchical logistic regression models with hospital random effects. Simulation of samples including a subset of hospitals with different volumes but equal poor performance (hospital effects=+3 SD in random-effect logistic model). SUBJECTS: A total of 1,085,568 Medicare fee-for-service patients undergoing 1,494,993 heart failure admissions in 4930 hospitals between July 1, 2005 and June 30, 2008. MEASURES: CMS methodology was used to determine the rank and proportion (by volume) of hospitals reported to perform "Worse than US National Rate." RESULTS: Percent of hospitals performing "Worse than US National Rate" was ∼40 times higher in the largest (fifth quintile by volume) compared with the smallest hospitals (first quintile). A similar gradient was seen in a cohort of 100 hospitals with simulated equal poor performance (0%, 0%, 5%, 20%, and 85% in quintiles 1 to 5) effectively leaving 78% of poor performers undetected. CONCLUSIONS: Our results illustrate the disparity of impact that the current CMS method of hospital profiling has on hospitals with higher volumes, translating into lower thresholds for detection and reporting of poor performance.

A population health approach to reducing observational intensity bias in health risk adjustment: cross sectional analysis of insurance claims.

OBJECTIVE: To compare the performance of two new approaches to risk adjustment that are free of the influence of observational intensity with methods that depend on diagnoses listed in administrative databases. SETTING: Administrative data from the US Medicare program for services provided in 2007 among 306 US hospital referral regions. DESIGN: Cross sectional analysis. PARTICIPANTS: 20% sample of fee for service Medicare beneficiaries residing in one of 306 hospital referral regions in the United States in 2007 (n = 5,153,877). MAIN OUTCOME MEASURES: The effect of health risk adjustment on age, sex, and race adjusted mortality and spending rates among hospital referral regions using four indices: the standard Centers for Medicare and Medicaid Services--Hierarchical Condition Categories (HCC) index used by the US Medicare program (calculated from diagnoses listed in Medicare's administrative database); a visit corrected HCC index (to reduce the effects of observational intensity on frequency of diagnoses); a poverty index (based on US census); and a population health index (calculated using data on incidence of hip fractures and strokes, and responses from a population based annual survey of health from the Centers for Disease Control and Prevention). RESULTS: Estimated variation in age, sex, and race adjusted mortality rates across hospital referral regions was reduced using the indices based on population health, poverty, and visit corrected HCC, but increased using the standard HCC index. Most of the residual variation in age, sex, and race adjusted mortality was explained (in terms of weighted R2) by the population health index: R2=0.65. The other indices explained less: R2=0.20 for the visit corrected HCC index; 0.19 for the poverty index, and 0.02 for the standard HCC index. The residual variation in age, sex, race, and price adjusted spending per capita across the 306 hospital referral regions explained by the indices (in terms of weighted R2) were 0.50 for the standard HCC index, 0.21 for the population health index, 0.12 for the poverty index, and 0.07 for the visit corrected HCC index, implying that only a modest amount of the variation in spending can be explained by factors most closely related to mortality. Further, once the HCC index is visit corrected it accounts for almost none of the residual variation in age, sex, and race adjusted spending. CONCLUSION: Health risk adjustment using either the poverty index or the population health index performed substantially better in terms of explaining actual mortality than the indices that relied on diagnoses from administrative databases; the population health index explained the majority of residual variation in age, sex, and race adjusted mortality. Owing to the influence of observational intensity on diagnoses from administrative databases, the standard HCC index over-adjusts for regional differences in spending. Research to improve health risk adjustment methods should focus on developing measures of risk that do not depend on observation influenced diagnoses recorded in administrative databases.

Issues in quality measurement: target population, risk adjustment, and ratings.

This review investigates three fundamental issues in health care performance measurement: selection of a homogeneous target population, risk adjustment, and assignment of quality rating categories. Many but not all organizations involved in quality measurement have adopted similar approaches to these important methodological issues. To illustrate the practical implications of different profiling strategies, we use The Society of Thoracic Surgeons' data to compare profiling results derived using prevailing analytical methodologies with those obtained from alternative approaches, exemplified by those of a well-known health care performance rating organization. We demonstrate the differences in provider classification that may result from these methodologic decisions.

Improvements in Medicare Part D risk adjustment: beneficiary access and payment accuracy.

INTRODUCTION: The continued success of the Medicare Part D program is contingent on appropriate Medicare payment adjustments for the projected drug costs of Part D plan enrollees. This article describes a major revision of these "risk adjustments," intended to more accurately match payments to costs, especially for high-cost, disadvantaged populations. METHODS: For the first time actual Part D data are used to calibrate risk adjustment. The sample is Medicare beneficiaries with fee-for-service enrollment in 2007 and Part D standalone prescription drug plan enrollment in 2008 (N = 14,224,301). Part D plan liability expenditures are predicted using demographic and diagnostic factors in a weighted least squares regression. Models for Medicare subpopulations are analyzed. The predictive accuracy of risk adjustment models is evaluated using R and predictive ratio statistics. RESULTS: Based on differences in both mean expenditures and incremental expenditures by diagnosis, separate Part D risk adjustment models are calibrated for 5 Medicare subpopulations: aged not low income; aged low income; nonaged not low income; nonaged low income; and institutionalized. The variation in plan liability drug expenditures (R) explained by these models ranges from 13% to 29%. The 5 separate models accurately predict mean plan liability expenditures ranging from $967 to $1762 across subpopulations and account for differences in incremental disease coefficients by subpopulation. CONCLUSIONS: The refined Part D risk adjustment model represents a significant improvement in the accuracy and fairness of payment to Part D plans. The new model provides greater incentives for drug plans to compete for low-income and institutionalized enrollees.

Impact of outcomes monitoring on innovation and risk in liver transplantation.

KEY POINTS: 1. The reporting of liver transplant center outcomes is required by the final rule of the Department of Health and Human Services. The reported patient and graft survival outcomes are risk-adjusted for specific donor and recipient factors, and the observed survival is compared to the expected survival. Both the Centers for Medicare and Medicaid Services and the Organ Procurement and Transplantation Network flag programs for corrective action when the observed survival is significantly less than the expected survival. Both agencies can take action up to the closure of a center. In the last 5 years, the Organ Procurement and Transplantation Network has not taken an adverse action that required the closure of a liver transplant center because of outcomes. 2. Center survey data suggest that centers may try to select donors and recipients to minimize poor outcomes. This strategy may not be effective if centers stop accepting donors or recipients according to factors that are included in the risk adjustment model. For example, limiting recipients to those less than 65 years old may improve the observed outcomes, but the expected outcomes will also improve because a recipient 65 years or older is included in the model's risk adjustment. 3. For factors such as cardiovascular risk that are not included in the model, it may be reasonable to exclude patients in an attempt to improve the observed outcomes without affecting the expected outcomes. Other examples of these types of factors are smoking, nutritional status, and donor liver biopsy findings. 4. Currently, there is no exemption for patients undergoing experimental protocols. Down-staging for hepatocellular carcinoma, transplantation for human immunodeficiency virus-positive recipients, and the use of left lobe grafts with inflow modification are relatively recent areas of innovation in liver transplantation. Because innovation is frequently associated with a learning curve and, therefore, poor outcomes, the inclusion of patients in innovative protocols potentially could lead to centers being subjected to an adverse action by the Organ Procurement and Transplantation Network or the Centers for Medicare and Medicaid Services. Active consideration is being given to the exclusion of patients in innovative protocols from center-specific outcomes.

Comparison of alternative risk adjustment measures for predictive modeling: high risk patient case finding using Taiwan's National Health Insurance claims.

BACKGROUND: Predictive modeling presents an opportunity to contain the expansion of medical expenditures by focusing on very few people. Evaluation of how risk adjustment models perform in predictive modeling in Taiwan or Asia has been rare. The aims of this study were to evaluate the performance of different risk adjustment models (the ACG risk adjustment system and prior expenditures) in predictive modeling, using Taiwan's National Health Insurance (NHI) claims data, and to compare characteristics of potentially high-expenditure subjects identified through different models. METHODS: A random sample of NHI enrollees continuously enrolled in 2002 and 2003 (n = 164,562) was selected. Health status measures and total expenditures derived from 2002 NHI claims data were used to predict the possibility of becoming 2003 top users. Statistics-based indicators (C-statistics, sensitivity, & Predictive Positive Value) and characteristics of identified top groups by different models (expenditures and prevalence of manageable diseases) were presented. RESULTS: Both diagnosis-based and prior expenditures models performed much better than the demographic model. Diagnosis-based models were better in identifying top users with manageable diseases; prior expenditures models were better in statistics-based indicators and identifying people with higher average expenditures. Prior expenditures status could correctly identify more actual top users than diagnosis-based or demographic models. The proportions of actual top users that could be identified by diagnosis-based models alone were much lower than that identified by prior expenditures status. CONCLUSIONS: Predicted top users identified by different models have different characteristics and there is little agreement between modes regarding which groups would be potentially top users; therefore, which model to use should depend on the purpose of predictive modeling. Prior expenditures are a more powerful tool than diagnosis-based risk adjusters in terms of correctly identifying more actual high expenditures users. There is still much room left for improvement of diagnosis-based models in predictive modeling.

Broadly applicable risk stratification system for predicting duration of hospitalization and mortality.

BACKGROUND: Hospitals are increasingly required to publicly report outcomes, yet performance is best interpreted in the context of population and procedural risk. We sought to develop a risk-adjustment method using administrative claims data to assess both national-level and hospital-specific performance. METHODS: A total of 35,179,507 patient stay records from 2001-2006 Medicare Provider Analysis and Review (MEDPAR) files were randomly divided into development and validation sets. Risk stratification indices (RSIs) for length of stay and mortality endpoints were derived from aggregate risk associated with individual diagnostic and procedure codes. Performance of RSIs were tested prospectively on the validation database, as well as a single institution registry of 103,324 adult surgical patients, and compared with the Charlson comorbidity index, which was designed to predict 1-yr mortality. The primary outcome was the C statistic indicating the discriminatory power of alternative risk-adjustment methods for prediction of outcome measures. RESULTS: A single risk-stratification model predicted 30-day and 1-yr postdischarge mortality; separate risk-stratification models predicted length of stay and in-hospital mortality. The RSIs performed well on the national dataset (C statistics for median length of stay and 30-day mortality were 0.86 and 0.84). They performed significantly better than the Charlson comorbidity index on the Cleveland Clinic registry for all outcomes. The C statistics for the RSIs and Charlson comorbidity index were 0.89 versus 0.60 for median length of stay, 0.98 versus 0.65 for in-hospital mortality, 0.85 versus 0.76 for 30-day mortality, and 0.83 versus 0.77 for 1-yr mortality. Addition of demographic information only slightly improved performance of the RSI. CONCLUSION: RSI is a broadly applicable and robust system for assessing hospital length of stay and mortality for groups of surgical patients based solely on administrative data.

Improving American College of Surgeons National Surgical Quality Improvement Program risk adjustment: incorporation of a novel procedure risk score.

BACKGROUND: Risk-adjusted evaluation is a key component of the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). The purpose of this study was to improve standard ACS NSQIP risk adjustment using a novel procedure risk score. STUDY DESIGN: Current Procedural Terminology codes (CPTs) represented in ACS NSQIP data were assigned to 136 procedure groups. Log odds predicted risk from preliminary logistic regression modeling generated a continuous risk score for each procedure group, used in subsequent modeling. Appropriate subsets of 271,368 patients in the 2008 ACS NSQIP were evaluated using logistic models for overall 30-day morbidity, 30-day mortality, and surgical site infection (SSI). Models were compared when including either work Relative Value Unit (RVU), RVU and the standard ACS NSQIP CPT range variable (CPT range), or RVU and the newly constructed CPT risk score (CPT risk), plus routine ACS NSQIP predictors. RESULTS: When comparing the CPT risk models with the CPT range models for morbidity in the overall general and vascular surgery dataset, CPT risk models provided better discrimination through higher c statistics at earlier steps (0.81 by step 3 vs 0.81 by step 46), more information through lower Akaike's information criterion (127,139 vs 130,019), and improved calibration through a smaller Hosmer-Lemeshow chi-square statistic (48.76 vs 116.79). Improved model characteristics of CPT risk over CPT range were most apparent for broader patient populations and outcomes. The CPT risk and standard CPT range models were moderately consistent in identification of outliers as well as assignment of hospitals to quality deciles (weighted kappa ≥ 0.870). CONCLUSIONS: Information from focused, clinically meaningful CPT procedure groups improves the risk estimation of ACS NSQIP models.

Performance of Pediatric Risk of Mortality, Pediatric Index of Mortality and PIM2 in term Chinese neonates.

Pediatric Risk of Mortality (PRISM), Pediatric Index of Mortality (PIM) and PIM2 could be applicable to the subset of term neonates has not been well investigated. The purpose of this study is to access and compare the performance of these scoring systems in predicting mortality probability in term Chinese neonates with critical illness. PRISM, PIM and PIM2 scores were calculated prospectively during a 1-year period on 243 neonates admitted to the neonatal intensive care unit (NICU) in the Children's Hospital of Zhejiang University in China. Of these, 36 neonates (14.81%) died in the NICU, while the mortality rates estimated by PRISM, PIM and PIM2 were 16.19, 14.58 and 11.12%, respectively. The area under the receiver-operating characteristic (ROC) curve [95% confidence intervals (CIs)] were 0.834 (0.767-0.902), 0.851 (0.786-0.916) and 0.854 (0.790-0.918) for PRISM, PIM and PIM2, respectively. The Hosmer-Lemeshow test gave a chi-square of 1.35 (p = 0.930) for PRISM, 1.03 (p = 0.960) for PIM and 4.58 (p = 0.469) for PIM2. The standardized mortality rates (SMRs) (95% CI) using PRISM, PIM and PIM2 were 0.92 (0.79-1.08), 1.02 (0.88-1.20) and 1.33 (1.13-1.62), respectively. Although PRISM, PIM and PIM2 have displayed good discrimination and calibration in the present setting, PIM is considered as the most accurate and appropriate tool for predicting mortality in the studied NICU.

Relationship between discharge practices and intensive care unit in-hospital mortality performance: evidence of a discharge bias.

CONTEXT: Current intensive care unit performance measures include in-hospital mortality after intensive care unit admission. This measure does not account for deaths occurring after transfer to another hospital or soon after discharge and therefore, may be biased. OBJECTIVE: Determine how transfer rates to other acute care hospitals and early post-discharge mortality rates impact hospital performance assessments using an in-hospital mortality model. DESIGN, SETTING, AND PARTICIPANTS: Data were retrospectively collected on 10,502 eligible intensive care unit patients across 35 California hospitals between 2001 and 2004. MEASURES: We calculated the rates of acute care hospital transfers and early post-discharge mortality (30-day overall mortality-30-day in-hospital mortality) for each hospital. We assessed hospital performance with standardized mortality ratios (SMRs) using the Mortality Probability Model III. Using regression models, we explored the relationship between in-hospital SMRs and the rates of hospital transfers or early post-discharge mortality. We explored the same relationship using a 30-day SMR. RESULTS: In multivariable models, for each 1% increase in patients transferred to another acute care hospital, there was an in-hospital SMR reduction of -0.021 (-0.040-0.001). Additionally, a 1% increase in early post-discharge mortality was associated with an in-hospital SMR reduction of -0.049 (-0.142-0.045). Assessing hospital performance based upon 30-day mortality end point resulted in SMRs closer to 1.0 for hospitals at high and low ends of in-hospital mortality performance. CONCLUSIONS: Variations in transfer rates and potentially discharge timing appear to bias in-hospital SMR calculations. A 30-day mortality model is a potential alternative that may limit this bias.

Performance of Pediatric Risk of Mortality (PRISM), Pediatric Index of Mortality (PIM), and PIM2 in a pediatric intensive care unit in a developing country.

OBJECTIVE: To determine the discriminative ability and calibration of existing scoring systems in predicting the outcome (mortality) in children admitted to an Indian pediatric intensive care unit (PICU). DESIGN: Prospective cohort study. SETTING: Pediatric Intensive Care Unit, Department of Pediatrics, All India Institute of Medical Sciences, New Delhi, from July 1, 2002, to July 31, 2003. PATIENTS: A total of 246 patients were admitted. After exclusion of 29 neonates and two patients who stayed in the PICU for 0.8. However, all the models underpredicted mortality. The likely reasons for this could be differences in the patient profile and greater load of severity of illness being managed with lesser resources--both physical and human--and differences in the quality of care.