Extended-age Out-of-sample Validation of Risk Stratification Index 3.0 Models Using Commercial All-payer Claims.

BACKGROUND: The authors previously reported a broad suite of individualized Risk Stratification Index 3.0 (Health Data Analytics Institute, Inc., USA) models for various meaningful outcomes in patients admitted to a hospital for medical or surgical reasons. The models used International Classification of Diseases, Tenth Revision, trajectories and were restricted to information available at hospital admission, including coding history in the previous year. The models were developed and validated in Medicare patients, mostly age 65 yr or older. The authors sought to determine how well their models predict utilization outcomes and adverse events in younger and healthier populations. METHODS: The authors' analysis was based on All Payer Claims for surgical and medical hospital admissions from Utah and Oregon. Endpoints included unplanned hospital admissions, in-hospital mortality, acute kidney injury, sepsis, pneumonia, respiratory failure, and a composite of major cardiac complications. They prospectively applied previously developed Risk Stratification Index 3.0 models to the younger and healthier 2017 Utah and Oregon state populations and compared the results to their previous out-of-sample Medicare validation analysis. RESULTS: In the Utah dataset, there were 55,109 All Payer Claims admissions across 40,710 patients. In the Oregon dataset, there were 21,213 admissions from 16,951 patients. Model performance on the two state datasets was similar or better than in Medicare patients, with an average area under the curve of 0.83 (0.71 to 0.91). Model calibration was reasonable with an R2 of 0.93 (0.84 to 0.97) for Utah and 0.85 (0.71 to 0.91) for Oregon. The mean sensitivity for the highest 5% risk population was 28% (17 to 44) for Utah and 37% (20 to 56) for Oregon. CONCLUSIONS: Predictive analytical modeling based on administrative claims history provides individualized risk profiles at hospital admission that may help guide patient management. Similar predictive performance in Medicare and in younger and healthier populations indicates that Risk Stratification Index 3.0 models are valid across a broad range of adult hospital admissions.

Risk Stratification Index 3.0, a Broad Set of Models for Predicting Adverse Events during and after Hospital Admission.

BACKGROUND: Risk stratification helps guide appropriate clinical care. Our goal was to develop and validate a broad suite of predictive tools based on International Classification of Diseases, Tenth Revision, diagnostic and procedural codes for predicting adverse events and care utilization outcomes for hospitalized patients. METHODS: Endpoints included unplanned hospital admissions, discharge status, excess length of stay, in-hospital and 90-day mortality, acute kidney injury, sepsis, pneumonia, respiratory failure, and a composite of major cardiac complications. Patient demographic and coding history in the year before admission provided features used to predict utilization and adverse events through 90 days after admission. Models were trained and refined on 2017 to 2018 Medicare admissions data using an 80 to 20 learn to test split sample. Models were then prospectively tested on 2019 out-of-sample Medicare admissions. Predictions based on logistic regression were compared with those from five commonly used machine learning methods using a limited dataset. RESULTS: The 2017 to 2018 development set included 9,085,968 patients who had 18,899,224 inpatient admissions, and there were 5,336,265 patients who had 9,205,835 inpatient admissions in the 2019 validation dataset. Model performance on the validation set had an average area under the curve of 0.76 (range, 0.70 to 0.82). Model calibration was strong with an average R 2 for the 99% of patients at lowest risk of 1.00. Excess length of stay had a root-mean-square error of 0.19 and R 2 of 0.99. The mean sensitivity for the highest 5% risk population was 19.2% (range, 11.6 to 30.1); for positive predictive value, it was 37.2% (14.6 to 87.7); and for lift (enrichment ratio), it was 3.8 (2.3 to 6.1). Predictive accuracies from regression and machine learning techniques were generally similar. CONCLUSIONS: Predictive analytical modeling based on administrative claims history can provide individualized risk profiles at hospital admission that may help guide patient management. Similar results from six different modeling approaches suggest that we have identified both the value and ceiling for predictive information derived from medical claims history.

Comparison of an Updated Risk Stratification Index to Hierarchical Condition Categories.

BACKGROUND: The Risk Stratification Index and the Hierarchical Condition Categories model baseline risk using comorbidities and procedures. The Hierarchical Condition categories are rederived yearly, whereas the Risk Stratification Index has not been rederived since 2010. The two models have yet to be directly compared. The authors thus rederived the Risk Stratification Index using recent data and compared their results to contemporaneous Hierarchical Condition Categories. METHODS: The authors reimplemented procedures used to derive the original Risk Stratification Index derivation using the 2007 to 2011 Medicare Analysis and Provider review file. The Hierarchical Condition Categories were constructed on the entire data set using software provided by the Center for Medicare and Medicaid Services. C-Statistics were used to compare discrimination between the models. After calibration, accuracy for each model was evaluated by plotting observed against predicted event rates. RESULTS: Discrimination of the Risk Stratification Index improved after rederivation. The Risk Stratification Index discriminated considerably better than the Hierarchical Condition Categories for in-hospital, 30-day, and 1-yr mortality and for hospital length-of-stay. Calibration plots for both models demonstrated linear predictive accuracy, but the Risk Stratification Index predictions had less variance. CONCLUSIONS: Risk Stratification discrimination and minimum-variance predictions make it superior to Hierarchical Condition Categories. The Risk Stratification Index provides a solid basis for care-quality metrics and for provider comparisons.

Validation and Calibration of the Risk Stratification Index.

BACKGROUND: The Risk Stratification Index was developed from 35 million Medicare hospitalizations from 2001 to 2006 but has yet to be externally validated on an independent large national data set, nor has it been calibrated. Finally, the Medicare Analysis and Provider Review file now allows 25 rather than 9 diagnostic codes and 25 rather than 6 procedure codes and includes present-on-admission flags. The authors sought to validate the index on new data, test the impact of present-on-admission codes, test the impact of the expansion to 25 diagnostic and procedure codes, and calibrate the model. METHODS: The authors applied the original index coefficients to 39,753,036 records from the 2007-2012 Medicare Analysis data set and calibrated the model. The authors compared their results with 25 diagnostic and 25 procedure codes, with results after restricting the model to the first 9 diagnostic and 6 procedure codes and to codes present on admission. RESULTS: The original coefficients applied to the 2007-2012 data set yielded C statistics of 0.83 for 1-yr mortality, 0.84 for 30-day mortality, 0.94 for in-hospital mortality, and 0.86 for median length of stay-values nearly identical to those originally reported. Calibration equations performed well against observed outcomes. The 2007-2012 model discriminated similarly when codes were restricted to nine diagnostic and six procedure codes. Present-on-admission models were about 10% less predictive for in-hospital mortality and hospital length of stay but were comparably predictive for 30-day and 1-yr mortality. CONCLUSIONS: Risk stratification performance was largely unchanged by additional diagnostic and procedure codes and only slightly worsened by restricting analysis to codes present on admission. The Risk Stratification Index, after calibration, thus provides excellent discrimination and calibration for important health services outcomes and thus appears to be a good basis for making hospital comparisons.

Surgical Treatment of Single-Level Lumbar Stenosis Is Associated with Lower 2-Year Mortality and Total Cost Compared with Nonsurgical Treatment: A Risk-Adjusted, Paired Analysis.

BACKGROUND: Spine surgery has demonstrated cost-effectiveness in reducing pain and restoring function, but the impact of spine surgery relative to nonsurgical care on longer-term outcomes has been less well described. Our objective was to compare single-level surgical treatment for lumbar stenosis, with or without spondylolisthesis, and nonsurgical treatment with respect to patient mortality, resource utilization, and health-care payments over the first 2 years following initial treatment. METHODS: A retrospective review of the Medicare National Database Fee for Service Files from 2011 to 2017 was performed. A 2-year prediction of mortality risk (risk stratification index, RSI) was used as a measure of patient baseline health. Patients (88%) were matched by RSI and demographics. Mortality, spine-related health-care utilization, and 2-year total Medicare payments for patients undergoing surgical treatment were compared with matched patients undergoing nonsurgical treatment. RESULTS: We identified 61,534 patients with stenosis alone and 83,813 with stenosis and spondylolisthesis. Surgical treatment was associated with 28% lower 2-year mortality compared with matched patients undergoing nonsurgical treatment. Total Medicare payments were significantly lower for patients with stenosis alone undergoing laminectomy alone and for patients with stenosis and spondylolisthesis undergoing laminectomy with or without fusion compared with patients undergoing nonsurgical treatment. There was no significant difference in mortality when fusion or laminectomy was compared with combined fusion and laminectomy. However, laminectomy alone was associated with significantly lower 2-year payments when treating stenosis with or without spondylolisthesis. CONCLUSIONS: Surgical treatment for stenosis with or without spondylolisthesis within the Medicare population was associated with significantly lower mortality and total medical payments at 2 years compared with nonsurgical treatment, although residual confounding could have contributed to these findings. LEVEL OF EVIDENCE: Therapeutic Level III. See Instructions for Authors for a complete description of levels of evidence.

Hospital stay and mortality are increased in patients having a "triple low" of low blood pressure, low bispectral index, and low minimum alveolar concentration of volatile anesthesia.

BACKGROUND: Low mean arterial pressure (MAP) and deep hypnosis have been associated with complications and mortality. The normal response to high minimum alveolar concentration (MAC) fraction of anesthetics is hypotension and low Bispectral Index (BIS) scores. Low MAP and/or BIS at lower MAC fractions may represent anesthetic sensitivity. The authors sought to characterize the effect of the triple low state (low MAP and low BIS during a low MAC fraction) on duration of hospitalization and 30-day all-cause mortality. METHODS: Mean intraoperative MAP, BIS, and MAC were determined for 24,120 noncardiac surgery patients at the Cleveland Clinic, Cleveland, Ohio. The hazard ratios associated with combinations of MAP, BIS, and MAC values greater or less than a reference value were determined. The authors also evaluated the association between cumulative triple low minutes, and excess length-of-stay and 30-day mortality. RESULTS: Means (±SD) defining the reference, low, and high states were 87 ± 5 mmHg (MAP), 46 ± 4 (BIS), and 0.56 ± 0.11 (MAC). Triple lows were associated with prolonged length of stay (hazard ratio 1.5, 95% CI 1.3-1.7). Thirty-day mortality was doubled in double low combinations and quadrupled in the triple low group. Triple low duration ≥60 min quadrupled 30-day mortality compared with ≤15 min. Excess length of stay increased progressively from ≤15 min to ≥60 min of triple low. CONCLUSIONS: The occurrence of low MAP during low MAC fraction was a strong and highly significant predictor for mortality. When these occurrences were combined with low BIS, mortality risk was even greater. The values defining the triple low state were well within the range that many anesthesiologists tolerate routinely.

Covid-19 and excess mortality in medicare beneficiaries.

We estimated excess mortality in Medicare recipients in the United States with probable and confirmed Covid-19 infections in the general community and amongst residents of long-term care (LTC) facilities. We considered 28,389,098 Medicare and dual-eligible recipients from one year before February 29, 2020 through September 30, 2020, with mortality followed through November 30th, 2020. Probable and confirmed Covid-19 diagnoses, presumably mostly symptomatic, were determined from ICD-10 codes. We developed a Risk Stratification Index (RSI) mortality model which was applied prospectively to establish baseline mortality risk. Excess deaths attributable to Covid-19 were estimated by comparing actual-to-expected deaths based on historical (2017-2019) comparisons and in closely matched concurrent (2020) cohorts with and without Covid-19. Overall, 677,100 (2.4%) beneficiaries had confirmed Covid-19 and 2,917,604 (10.3%) had probable Covid-19. A total of 472,329 confirmed cases were community living and 204,771 were in LTC. Mortality following a probable or confirmed diagnosis in the community increased from an expected incidence of about 4.0% to actual incidence of 7.5%. In long-term care facilities, the corresponding increase was from 20.3% to 24.6%. The absolute increase was therefore similar at 3-4% in the community and in LTC residents. The percentage increase was far greater in the community (89.5%) than among patients in chronic care facilities (21.1%) who had higher baseline risk of mortality. The LTC population without probable or confirmed Covid-19 diagnoses experienced 38,932 excess deaths (34.8%) compared to historical estimates. Limitations in access to Covid-19 testing and disease under-reporting in LTC patients probably were important factors, although social isolation and disruption in usual care presumably also contributed. Remarkably, there were 31,360 (5.4%) fewer deaths than expected in community dwellers without probable or confirmed Covid-19 diagnoses. Disruptions to the healthcare system and avoided medical care were thus apparently offset by other factors, representing overall benefit. The Covid-19 pandemic had marked effects on mortality, but the effects were highly context-dependent.

Elucidating the association between regional variation in diagnostic frequency with risk-adjusted mortality through analysis of claims data of medicare inpatients: a cross-sectional study.

OBJECTIVE: The validity of risk-adjustment methods based on administrative data has been questioned because hospital referral regions with higher diagnosis frequencies report lower case-fatality rates, implying that diagnoses do not track the underlying health risk. The objective of this study is to test the hypothesis that regional variation of diagnostic frequency in inpatient records is not associated with different coding practices but a reflection of the underlying health risks. DESIGN: We applied two stratification methods to Medicare Analysis and Provider Review data from 2009 through 2014: (1) the number of chronic conditions; and, (2) quartiles of Risk Stratification Index (RSI)-defined risk. After sorting hospital referral regions into quintiles of diagnostic frequency, we examined all-cause mortality. SETTING: Medicare Analysis and Provider Review administrative database. PARTICIPANTS: 18 126 301 hospitalised Medicare fee-for-service beneficiaries aged 65 or older who had at least one hospital-based procedure between 2009 and 2014. EXPOSURE: Coding frequency and baseline regional population risk factors by hospital referral region. PRIMARY AND SECONDARY OUTCOMES AND MEASURES: One year all-cause mortality in patients having the same number of chronic conditions or within the same RSI score quartile across US health referral regions, grouped by diagnostic frequency. RESULTS: No consistent relationship between diagnostic frequency and mortality in the risk stratum defined by number of chronic conditions was detected. In the strata defined by RSI quartile, there was no decrease in mortality as a function of diagnostic frequency. Instead, adjusted mortality was positively correlated with socioeconomic risk factors. CONCLUSIONS: Using present-on-admission codes only, diagnostic frequency among inpatients with at least one hospital-based procedure appears to be consequent to differences in baseline population health status, rather than diagnostic coding practices. In this population, claims-based risk-adjustment using RSI appears to be useful for assessing hospital outcomes and performance.

Association Between First-Line Immune Checkpoint Inhibition and Survival for Medicare-Insured Patients With Advanced Non-Small Cell Lung Cancer.

Importance: Immunotherapy is now a cornerstone of treatment for advanced non-small cell lung cancer (NSCLC), but its uptake and effectiveness among older patients outside clinical trials remain poorly understood. Objective: To understand treatment patterns and evaluate the overall survival associated with checkpoint inhibitor immunotherapy, cytotoxic chemotherapy, and combined chemoimmunotherapy for older patients who have advanced NSCLC and Medicare coverage. Design, Setting, and Participants: This retrospective cohort study included Medicare-insured patients in the US aged 66 to 89 years who initiated first palliative-intent systemic therapy for lung cancer between January 1, 2016, and December 31, 2018. Survival follow-up continued through March 31, 2020. A total of 19 529 patients who had advanced lung cancer and were insured by a Medicare fee-for-service plan were included in the analysis. Exposures: Regimens included pembrolizumab monotherapy (n = 3079), combined platinum-based drug (ie, cisplatin or carboplatin [hereinafter, platinum]) and pemetrexed disodium (n = 5159), combined platinum and a taxane (ie, paclitaxel, nab-paclitaxel, or docetaxel) (n = 9866), and combined platinum, pemetrexed, and pembrolizumab (n = 1425), as ascertained using Medicare claims from the Centers for Medicare & Medicaid Services. Main Outcomes and Measures: The primary outcome was overall survival, which was measured using the restricted mean survival time (RMST) with propensity score adjustment for clinical and sociodemographic characteristics. Median survival was also reported for comparison with outcomes from registrational trials. Results: A total of 19 529 patients (54% male, 46% female; median age, 73.8 [interquartile range, 69.9-78.4] years) were identified for analysis. The uptake of pembrolizumab-containing regimens in the Medicare population was rapid, increasing from 0.7% of first-line treatments in the second quarter of 2016 to 42.4% in the third quarter of 2018. Patients who were older (≥70 years, 2484 [81%]), were female (1577 [51%]), and/or had higher Risk Stratification Index scores (highest quintile, 922 [30%]) were more likely to receive single-agent pembrolizumab than chemotherapy. After propensity score adjustment, pembrolizumab was associated with survival similar to platinum/pemetrexed (RMST difference, -0.2 [95% CI, -0.5 to 0.2] months) or platinum/taxane (RMST difference, -0.7 [95% CI, -1.0 to -0.4] months). Patients receiving platinum/pemetrexed/pembrolizumab chemoimmunotherapy also had adjusted survival similar to those receiving platinum/pemetrexed chemotherapy (RMST difference, 0.5 [95% CI, 0.1-0.9] months). The unadjusted median survival was 11.4 (95% CI, 10.5-12.3) months among patients receiving single-agent pembrolizumab, approximately 15 months shorter than observed among pembrolizumab-treated participants in the KEYNOTE-024 trial. The unadjusted median survival was 12.9 (95% CI, 11.8-14.0) months among patients receiving platinum/pemetrexed/pembrolizumab chemoimmunotherapy, approximately 10 months shorter than observed among platinum/pemetrexed/pembrolizumab-treated participants in the KEYNOTE-189 trial. Conclusions and Relevance: Immunotherapy has been incorporated rapidly into treatment for patients with advanced NSCLC. However, survival estimates in the Medicare population are much shorter than those reported in registrational trials. These results provide contemporary estimates of survival for older patients with advanced NSCLC treated in routine practice, facilitating patient-centered decision-making.

Broadly applicable risk stratification system for predicting duration of hospitalization and mortality.

BACKGROUND: Hospitals are increasingly required to publicly report outcomes, yet performance is best interpreted in the context of population and procedural risk. We sought to develop a risk-adjustment method using administrative claims data to assess both national-level and hospital-specific performance. METHODS: A total of 35,179,507 patient stay records from 2001-2006 Medicare Provider Analysis and Review (MEDPAR) files were randomly divided into development and validation sets. Risk stratification indices (RSIs) for length of stay and mortality endpoints were derived from aggregate risk associated with individual diagnostic and procedure codes. Performance of RSIs were tested prospectively on the validation database, as well as a single institution registry of 103,324 adult surgical patients, and compared with the Charlson comorbidity index, which was designed to predict 1-yr mortality. The primary outcome was the C statistic indicating the discriminatory power of alternative risk-adjustment methods for prediction of outcome measures. RESULTS: A single risk-stratification model predicted 30-day and 1-yr postdischarge mortality; separate risk-stratification models predicted length of stay and in-hospital mortality. The RSIs performed well on the national dataset (C statistics for median length of stay and 30-day mortality were 0.86 and 0.84). They performed significantly better than the Charlson comorbidity index on the Cleveland Clinic registry for all outcomes. The C statistics for the RSIs and Charlson comorbidity index were 0.89 versus 0.60 for median length of stay, 0.98 versus 0.65 for in-hospital mortality, 0.85 versus 0.76 for 30-day mortality, and 0.83 versus 0.77 for 1-yr mortality. Addition of demographic information only slightly improved performance of the RSI. CONCLUSION: RSI is a broadly applicable and robust system for assessing hospital length of stay and mortality for groups of surgical patients based solely on administrative data.