Reformulating provider profiling by grouping providers treating similar patients prior to evaluating performance.

Standard approaches to comparing health providers' performance rely on hierarchical logistic regression models that adjust for patient characteristics at admission. Estimates from these models may be misleading when providers treat different patient populations and the models are misspecified. To address this limitation, we propose a novel profiling approach that identifies groups of providers treating similar populations of patients and then evaluates providers' performance within each group. The groups of providers are identified using a Bayesian multilevel finite mixture of general location models. To compare the performance of our proposed profiling approach to standard methods, we use patient-level data from 119 skilled nursing facilities in Massachusetts. We use simulated and observed outcome data to explore the performance of these profiling methods in different settings. In simulations, our proposed method classifies providers to groups with similar patients' admission characteristics. In addition, in the presence of limited overlap in patient characteristics across providers and misspecifications of the outcome model, the provider-level estimates obtained using our approach identified providers that under- and overperformed compared to the standard regression-based approaches more accurately.

Variation in Risk-Adjusted Ventilator-Associated Pneumonia Days Within a Quality Collaborative.

INTRODUCTION: Ventilator-associated pneumonia (VAP) is associated with increased mortality, prolonged mechanical ventilation, and longer intensive care unit stays. The rate of VAP (VAPs per 1000 ventilator days) within a hospital is an important quality metric. Despite adoption of preventative strategies, rates of VAP in injured patients remain high in trauma centers. Here, we report variation in risk-adjusted VAP rates within a statewide quality collaborative. METHODS: Using Michigan Trauma Quality Improvement Program data from 35 American College of Surgeons-verified Level I and Level II trauma centers between November 1, 2020 and January 31, 2023, a patient-level Poisson model was created to evaluate the risk-adjusted rate of VAP across institutions given the number of ventilator days, adjusting for injury severity, physiologic parameters, and comorbid conditions. Patient-level model results were summed to create center-level estimates. We performed observed-to-expected adjustments to calculate each center's risk-adjusted VAP days and flagged outliers as hospitals whose confidence intervals lay above or below the overall mean. RESULTS: We identified 538 VAP occurrences among a total of 33,038 ventilator days within the collaborative, with an overall mean of 16.3 VAPs per 1000 ventilator days. We found wide variation in risk-adjusted rates of VAP, ranging from 0 (0-8.9) to 33.0 (14.4-65.1) VAPs per 1000 d. Several hospitals were identified as high or low outliers. CONCLUSIONS: There exists significant variation in the rate of VAP among trauma centers. Investigation of practices and factors influencing the differences between low and high outlier institutions may yield information to reduce variation and improve outcomes.

Patient-Reported Outcome-Based Performance Measures in Alternative Payment Models: Current Use, Implementation Barriers, and Principles to Succeed.

OBJECTIVES: Patient-reported outcome (PRO)-based performance measures (PRO-PMs) offer opportunities to aggregate survey data into a reliable and valid assessment of performance at the entity-level (eg, clinician, hospital, and accountable care organization). Our objective was to address the existing literature gap regarding the implementation barriers, current use, and principles for PRO-PMs to succeed. METHODS: As quality measurement experts, we first highlighted key principles of PRO-PMs and how alternative payment models (APMs) may be integral in promoting more widespread use. In May 2023, we reviewed the Centers for Medicare and Medicaid Services (CMS) Measures Inventory Tool for active PRO-PM usage within CMS programs. We finally present principles to prioritize as part PRO-PMs succeeding within APMs. RESULTS: We identified 5 implementation barriers to PRO-PM use: original development of instrument, response rate sufficiency, provider burden, hesitancy regarding fairness, and attribution of desired outcomes. There existed 54 instances of active PRO-PM usage across CMS programs, including 46 unique PRO-PMs within 14 CMS programs. Five principles to prioritize as part of greater PRO-PM development and incorporation within APMs include the following: (1) clinical salience, (2) adequate sample size, (3) meaningful range of performance among measured entities and the ability to detect performance change in a reasonable time frame, (4) equity focus, and (5) appropriate risk adjustment. CONCLUSIONS: Identified barriers and principles to prioritize should be considered during PRO-PM development and implementation phases to link available and novel measures to payment programs while ensuring provider and stakeholder engagement.

Impact of operative time on textbook outcome after minimally invasive esophagectomy, a risk-adjusted analysis from a high-volume center.

BACKGROUND: We aimed to study the impact of operative time on textbook outcome (TO), especially postoperative complications and length of postoperative stay in minimally invasive esophagectomy. METHODS: Patients undergoing esophagectomy for curative intent within a prospectively maintained database from 2016 to 2022 were retrieved. Relationships between operative time and outcomes were quantified using multivariable mixed-effects models with medical teams random effects. A restricted cubic spline (RCS) plotting was used to characterize correlation between operative time and the odds for achieving TO. RESULTS: Data of 2210 patients were examined. Median operative time was 270 mins (interquartile range, 233-313) for all cases. Overall, 902 patients (40.8%) achieved TO. Among non-TO patients, 226 patients (10.2%) had a major complication (grade ≥ III), 433 patients (19.6%) stayed postoperatively longer than 14 days. Multivariable analysis revealed operative time was associated with higher odds of major complications (odds ratio 1.005, P < 0.001) and prolonged postoperative stay (≥ 14 days) (odds ratio 1.003, P = 0.006). The relationship between operative time and TO exhibited an inverse-U shape, with 298 mins identified as the tipping point for the highest odds of achieving TO. CONCLUSIONS: Longer operative time displayed an adverse influence on postoperative morbidity and increased lengths of postoperative stay. In the present study, the TO displayed an inverse U-shaped correlation with operative time, with a significant peak at 298 mins. Potential factors contributing to prolonged operative time may potentiate targets for quality metrics and risk-adjustment process.

Comparing risk-adjusted inpatient fall rates internationally: validation of a risk-adjustment model using multicentre cross-sectional data from hospitals in Switzerland and Austria.

BACKGROUND: Inpatient falls in hospitals are an acknowledged indicator of quality of care. International comparisons could highlight quality improvement potential and enable cross-national learning. Key to fair cross-national comparison is the availability of a risk adjustment model validated in an international context. This study aimed to 1) ascertain that the variables of the inpatient fall risk adjustment model do not interact with country and thus can be used for risk adjustment, 2) compare the risk of falling in hospitals between Switzerland and Austria after risk adjustment. METHODS: The data on inpatient falls from Swiss and Austrian acute care hospitals were collected on a single measurement day in 2017, 2018 and 2019 as part of an international multicentre cross-sectional study. Multilevel logistic regression models were used to screen for interaction effects between the patient-related fall risk factors and the countries. The risks of falling in hospital in Switzerland and in Austria were compared after applying the risk-adjustment model. RESULTS: Data from 176 hospitals and 43,984 patients revealed an inpatient fall rate of 3.4% in Switzerland and 3.9% in Austria. Two of 15 patient-related fall risk variables showed an interaction effect with country: Patients who had fallen in the last 12 months (OR 1.49, 95% CI 1.10-2.01, p = 0.009) or had taken sedatives/psychotropic medication (OR 1.40, 95% CI 1.05-1.87, p = 0.022) had higher odds of falling in Austrian hospitals. Significantly higher odds of falling were observed in Austrian (OR 1.38, 95% CI 1.13-1.68, p = 0.002) compared to Swiss hospitals after applying the risk-adjustment model. CONCLUSIONS: Almost all patient-related fall risk factors in the model are suitable for a risk-adjusted cross-country comparison, as they do not interact with the countries. Further model validation with additional countries is warranted, particularly to assess the interaction of risk factors "fall in the last 12 months" and "sedatives/psychotropic medication intake" with country variable. The study underscores the crucial role of an appropriate risk-adjustment model in ensuring fair international comparisons of inpatient falls, as the risk-adjusted, as opposed to the non-risk-adjusted country comparison, indicated significantly higher odds of falling in Austrian compared to Swiss hospitals.

Impact of Pulmonary Complications on Outcomes and Resource Use After Elective Cardiac Surgery.

OBJECTIVE: To characterize the impact of pulmonary complications (PCs) on mortality, costs, and readmissions after elective cardiac operations in a national cohort and to test for hospital-level variation in PC. BACKGROUND: PC after cardiac surgery are targets for quality improvement efforts. Contemporary studies evaluating the impact of PC on outcomes are lacking, as is data regarding hospital-level variation in the incidence of PC. METHODS: Adults undergoing elective coronary artery bypass grafting and/or valve operations were identified in the 2016-2019 Nationwide Readmissions Database. PC was defined as a composite of reintubation, prolonged (>24 hours) ventilation, tracheostomy, or pneumonia. Generalized linear models were fit to evaluate associations between PC and outcomes. Institutional variation in PC was studied using observed-to-expected ratios. RESULTS: Of 588,480 patients meeting study criteria, 6.7% developed PC. After risk adjustment, PC was associated with increased odds of mortality (14.6, 95% CI, 12.6-14.8), as well as a 7.9-day (95% CI, 7.6-8.2) increase in length of stay and $41,300 (95% CI, 39,600-42,900) in attributable costs. PC was associated with 1.3-fold greater hazard of readmission and greater incident mortality at readmission (6.7% vs 1.9%, P <0.001). Significant hospital-level variation in PC was present, with observed-to-expected ratios ranging from 0.1 to 7.7. CONCLUSIONS: Pulmonary complications remain common after cardiac surgery and are associated with substantially increased mortality and expenditures. Significant hospital-level variation in PC exists in the United States, suggesting the need for systematic quality improvement efforts to reduce PC and their impact on outcomes.

Risk Adjustment of ICD-10-CM Coded Potential Inpatient Complications Using Administrative Data.

OBJECTIVE: To risk-adjust the Potential Inpatient Complication (PIC) measure set and propose a method to identify large deviations between observed and expected PIC counts. DATA SOURCES: Acute inpatient stays from the Premier Healthcare Database from January 1, 2019 to December 31, 2021. STUDY DESIGN: In 2014, the PIC list was developed to identify a broader set of potential complications that can occur as a result of care decisions. Risk adjustment for 111 PIC measures is performed across 3 age-based strata. Using patient-level risk factors and PIC occurrences, PIC-specific probabilities of occurrence are estimated through multivariate logistic regression models. Poisson Binomial cumulative mass function estimates identify deviations between observed and expected PIC counts across levels of patient-visit aggregation. Area under the curve (AUC) estimates are used to demonstrate PIC predictive performance in an 80:20 derivation-validation split framework. DATA COLLECTION/EXTRACTION METHODS: We used N=3,363,149 administrative hospitalizations between 2019 and 2021 from the Premier Healthcare Database. PRINCIPAL FINDINGS: PIC-specific model predictive performance was strong across PICs and age strata. Average area under the curve estimates across PICs were 0.95 (95% CI: 0.93-0.96), 0.91 (95% CI: 0.90-0.93), and 0.90 (95% CI: 0.89-0.91) for the neonate and infant, pediatric, and adult strata, respectively. CONCLUSIONS: The proposed method provides a consistent quality metric that adjusts for the population's case mix. Age-specific risk stratification further addresses currently ignored heterogeneity in PIC prevalence across age groups. Finally, the proposed aggregation method identifies large PIC-specific deviations between observed and expected counts, flagging areas with a potential need for quality improvements.

An Update to the Kaiser Permanente Inpatient Risk Adjustment Methodology Accurately Predicts In-Hospital Mortality: a Retrospective Cohort Study.

BACKGROUND: Methods to accurately predict the risk of in-hospital mortality are important for applications including quality assessment of healthcare institutions and research. OBJECTIVE: To update and validate the Kaiser Permanente inpatient risk adjustment methodology (KP method) to predict in-hospital mortality, using open-source tools to measure comorbidity and diagnosis groups, and removing troponin which is difficult to standardize across modern clinical assays. DESIGN: Retrospective cohort study using electronic health record data from GEMINI. GEMINI is a research collaborative that collects administrative and clinical data from hospital information systems. PARTICIPANTS: Adult general medicine inpatients at 28 hospitals in Ontario, Canada, between April 2010 and December 2022. MAIN MEASURES: The outcome was in-hospital mortality, modeled by diagnosis group using 56 logistic regressions. We compared models with and without troponin as an input to the laboratory-based acute physiology score. We fit and validated the updated method using internal-external cross-validation at 28 hospitals from April 2015 to December 2022. KEY RESULTS: In 938,103 hospitalizations with 7.2% in-hospital mortality, the updated KP method accurately predicted the risk of mortality. The c-statistic at the median hospital was 0.866 (see Fig. 3) (25th-75th 0.848-0.876, range 0.816-0.927) and calibration was strong for nearly all patients at all hospitals. The 95th percentile absolute difference between predicted and observed probabilities was 0.038 at the median hospital (25th-75th 0.024-0.057, range 0.006-0.118). Model performance was very similar with and without troponin in a subset of 7 hospitals, and performance was similar with and without troponin for patients hospitalized for heart failure and acute myocardial infarction. CONCLUSIONS: An update to the KP method accurately predicted in-hospital mortality for general medicine inpatients in 28 hospitals in Ontario, Canada. This updated method can be implemented in a wider range of settings using common open-source tools.

Impact of case-mix adjustment on observed variation in the healing of diabetic foot ulcers at 12 weeks using data from the National Diabetes Foot Care Audit of England and Wales: A cohort study.

AIM: This cohort study investigates the extent to which variation in ulcer healing between services can be explained by demographic and clinical characteristics. METHODS: The National Diabetes Foot Care Audit collated data on people with diabetic foot ulcers presenting to specialist services in England and Wales between July 2014 and March 2018. Logistic regression models were created to describe associations between risk factors and a person being alive and ulcer-free 12 weeks from presentation, and to investigate whether variation between 120 participating services persisted after risk factor adjustment. RESULTS: Of 27,030 people with valid outcome data, 12,925 (47.8%) were alive and ulcer-free at 12 weeks, 13,745 (50.9%) had an unhealed ulcer and 360 had died (1.3%). Factors associated with worse outcome were male sex, more severe ulcers, history of cardiac or renal disease and a longer time between first presentation to a non-specialist healthcare professional and first expert assessment. After adjustment for these factors, four services (3.3%) were more than 3SD above and seven services (5.8%) were more than 3SD below the national mean for proportions that were alive and ulcer-free at follow-up. CONCLUSIONS/INTERPRETATIONS: Variation in the healing of diabetic foot ulcers between specialist services in England and Wales persisted after adjusting for demographic characteristics, ulcer severity, smoking, body mass index and co-morbidities. We conclude that other factors contribute to variation in healing of diabetic foot ulcers and include the time to specialist assessment.

Risk of pancreatic cancer after acute pancreatitis: A population-based matched cohort study.

BACKGROUND: We investigated the short- and long-term risks of pancreatic cancer after the diagnosis of acute pancreatitis. METHODS: This population-based matched-cohort study used data from the Korean National Health Insurance Service database. Patients with acute pancreatitis (n = 25,488) were matched with the control group (n = 127,440) based on age, sex, body mass index, smoking status, and diabetes. We estimated the hazard ratios for developing pancreatic cancer in both groups using Cox regression analysis. RESULTS: During a median follow-up of 5.4 years, pancreatic cancer developed in 479 patients (1.9%) in the acute pancreatitis group and 317 patients (0.2%) in the control group. Compared with the control group, the risk of pancreatic cancer in the acute pancreatitis group was very high within the first 2 years, which gradually decreased over time. The hazard ratio for the risk of developing pancreatitis was 8.46 (95% confidence interval, 5.57-12.84) at 1-2 years, and then decreased to 3.62 (95% confidence interval, 2.26-4.91) at 2-4 years. However, even after 8-10 years, the hazard ratio was still statistically significantly increased to 2.80 (95% confidence interval, 1.42-5.53). After 10 years, there was no significant difference in the risk of pancreatic cancer between the two groups. CONCLUSIONS: The risk of pancreatic cancer increases rapidly after acute pancreatitis diagnosis, gradually declines after 2 years, and remains elevated for up to 10 years. Further studies are needed to determine the long-term effects of acute pancreatitis on the risk of pancreatic cancer.

Case-mix adjustment of patient-reported experience measures in National Regional Center for Pediatric.

BACKGROUND: The aim of the study was to identify case-mix adjusters for the Chinese version of the Child Hospital Consumer Assessment of Healthcare Providers and Systems (Child-HCAHPS) and assess the impact of case-mix adjustment on patient experience measures in China. METHODS: This study analyzed data collected from six National Regional Center for Pediatric across China retrospectively. Participants were children aged ≤17 years and their guardians who completed the survey. The Chinese Child-HCAHPS was used to measure pediatric inpatient care experience. Candidate case-mix adjusters were assessed using a summary measure of explanatory power. Changes in scores and rankings of the six centers were quantified to assess the impact of adjustment. RESULTS: A total of 2708 respondents completed the survey from January to March 2021, with a response rate of 7-15%. The child's global health status and the respondent being the child's mother were identified as case-mix adjusters, and case-mix adjustment models for 18 patient experience items were constructed. Kendall's τ correlation of hospital rankings before and after adjustment ranged from 0.73 to 1.00. CONCLUSIONS: Although the impact of case-mix adjustment may appear modest in our sample, it demonstrated the feasibility, necessity, and methodology for further development of case-mix adjustment models in pediatric healthcare facilities in China. IMPACT: Case-mix adjustment models adjust for factors that are unamendable by healthcare providers that may affect patient experience ratings, thereby improving the comparability of institutional-level ratings. Standardized case-mix adjustment protocols for quality measures need to be modified in different settings. This is the first study to identify adjustment variables and the possible impact of case-mix adjustment on pediatric inpatients' experience measures in a Chinese population. This study provided evidence on the feasibility and necessity for further development of case-mix adjustment models for pediatric healthcare facilities in China.

Does Work Relative Value Unit Measure Surgical Complexity for Risk Adjustment of Surgical Outcomes?

INTRODUCTION: The purpose of this study was to determine whether the work relative value unit (workRVU) of a patient's operation can be useful as a measure of surgical complexity for the risk adjustment of surgical outcomes. METHODS: We retrospectively analyzed the American College of Surgeon's National Surgical Quality Improvement Program database (2005-2018). We examined the associations of workRVU of the patient's primary operation with preoperative patient characteristics and associations with postoperative complications. We performed forward selection multiple logistic regression analysis to determine the predictive importance of workRVU. We then generated prediction models using patient characteristics with and without workRVU and compared c-indexes to assess workRVU's additive predictive value. RESULTS: 7,507,991 operations were included. Patients who were underweight, functionally dependent, transferred from an acute care hospital, had higher American Society of Anesthesiologists class or who had medical comorbidities had operations with higher workRVU (all P < 0.0001). The subspecialties with the highest workRVU were neurosurgery (mean = 22.2), thoracic surgery (mean = 21.1), and vascular surgery (mean = 18.8) (P < 0.0001). For all postoperative complications, mean workRVU was higher for patients with the complication than those without (all P < 0.0001). For eight of 12 postoperative complications, workRVU entered the logistic regression models as a predictor variable in the 1st to 4th steps. Addition of workRVU as a preoperative predictive variable improved the c-index of the prediction models. CONCLUSIONS: WorkRVU was associated with sicker patients and patients experiencing postoperative complications and was an important predictor of postoperative complications. When added to a prediction model including patient characteristics, it only marginally improved prediction. This is possibly because workRVU is associated with patient characteristics.

Development and validation of a prognostic model for death 30 days after adult emergency laparotomy.

The probability of death after emergency laparotomy varies greatly between patients. Accurate pre-operative risk prediction is fundamental to planning care and improving outcomes. We aimed to develop a model limited to a few pre-operative factors that performed well irrespective of surgical indication: obstruction; sepsis; ischaemia; bleeding; and other. We derived a model with data from the National Emergency Laparotomy Audit for patients who had emergency laparotomy between December 2016 and November 2018. We tested the model on patients who underwent emergency laparotomy between December 2018 and November 2019. There were 4077/40,816 (10%) deaths 30 days after surgery in the derivation cohort. The final model had 13 pre-operative variables: surgical indication; age; blood pressure; heart rate; respiratory history; urgency; biochemical markers; anticipated malignancy; anticipated peritoneal soiling; and ASA physical status. The predicted mortality probability deciles ranged from 0.1% to 47%. There were 1888/11,187 deaths in the test cohort. The scaled Brier score, integrated calibration index and concordance for the model were 20%, 0.006 and 0.86, respectively. Model metrics were similar for the five surgical indications. In conclusion, we think that this prognostic model is suitable to support decision-making before emergency laparotomy as well as for risk adjustment for comparing organisations.

Risk adjustment model for tuberculosis compared to non-tuberculosis mycobacterium or latent tuberculosis infection: Center for Personalized Precision Medicine of Tuberculosis (cPMTb) cohort database.

BACKGROUND: The Center for Personalized Precision Medicine of Tuberculosis (cPMTb) was constructed to develop personalized pharmacotherapeutic systems for tuberculosis (TB). This study aimed to introduce the cPMTb cohort and compare the distinct characteristics of patients with TB, non-tuberculosis mycobacterium (NTM) infection, or latent TB infection (LTBI). We also determined the prevalence and specific traits of polymorphisms in N-acetyltransferase-2 (NAT2) and solute carrier organic anion transporter family member 1B1 (SLCO1B1) phenotypes using this prospective multinational cohort. METHODS: Until August 2021, 964, 167, and 95 patients with TB, NTM infection, and LTBI, respectively, were included. Clinical, laboratory, and radiographic data were collected. NAT2 and SLCO1B1 phenotypes were classified by genomic DNA analysis. RESULTS: Patients with TB were older, had lower body mass index (BMI), higher diabetes rate, and higher male proportion than patients with LTBI. Patients with NTM infection were older, had lower BMI, lower diabetes rate, higher previous TB history, and higher female proportion than patients with TB. Patients with TB had the lowest albumin levels, and the prevalence of the rapid, intermediate, and slow/ultra-slow acetylator phenotypes were 39.2%, 48.1%, and 12.7%, respectively. The prevalence of rapid, intermediate, and slow/ultra-slow acetylator phenotypes were 42.0%, 44.6%, and 13.3% for NTM infection, and 42.5%, 48.3%, and 9.1% for LTBI, respectively, which did not differ significantly from TB. The prevalence of the normal, intermediate, and lower transporter SLCO1B1 phenotypes in TB, NTM, and LTBI did not differ significantly; 74.9%, 22.7%, and 2.4% in TB; 72.0%, 26.1%, and 1.9% in NTM; and 80.7%, 19.3%, and 0% in LTBI, respectively. CONCLUSIONS: Understanding disease characteristics and identifying pharmacokinetic traits are fundamental steps in optimizing treatment. Further longitudinal data are required for personalized precision medicine. TRIAL REGISTRATION: This study registered ClinicalTrials.gov NO. NCT05280886.

Comparison of diagnosis-based risk adjustment methods for episode-based costs to apply in efficiency measurement.

BACKGROUND: The recent rising health spending intrigued efficiency and cost-based performance measures. However, mortality risk adjustment methods are still under consideration in cost estimation, though methods specific to cost estimate have been developed. Therefore, we aimed to compare the performance of diagnosis-based risk adjustment methods based on the episode-based cost to utilize in efficiency measurement. METHODS: We used the Health Insurance Review and Assessment Service-National Patient Sample as the data source. A separate linear regression model was constructed within each Major Diagnostic Category (MDC). Individual models included explanatory (demographics, insurance type, institutional type, Adjacent Diagnosis Related Group [ADRG], diagnosis-based risk adjustment methods) and response variables (episode-based costs). The following risk adjustment methods were used: Refined Diagnosis Related Group (RDRG), Charlson Comorbidity Index (CCI), National Health Insurance Service Hierarchical Condition Categories (NHIS-HCC), and Department of Health and Human Service-HCC (HHS-HCC). The model accuracy was compared using R-squared (R2), mean absolute error, and predictive ratio. For external validity, we used the 2017 dataset. RESULTS: The model including RDRG improved the mean adjusted R2 from 40.8% to 45.8% compared to the adjacent DRG. RDRG was inferior to both HCCs (RDRG adjusted R2 45.8%, NHIS-HCC adjusted R2 46.3%, HHS-HCC adjusted R2 45.9%) but superior to CCI (adjusted R2 42.7%). Model performance varied depending on the MDC groups. While both HCCs had the highest explanatory power in 12 MDCs, including MDC P (Newborns), RDRG showed the highest adjusted R2 in 6 MDCs, such as MDC O (pregnancy, childbirth, and puerperium). The overall mean absolute errors were the lowest in the model with RDRG ($1,099). The predictive ratios showed similar patterns among the models regardless of the  subgroups according to age, sex, insurance type, institutional type, and the upper and lower 10th percentiles of actual costs. External validity also showed a similar pattern in the model performance. CONCLUSIONS: Our research showed that either NHIS-HCC or HHS-HCC can be useful in adjusting comorbidities for episode-based costs in the process of efficiency measurement.

Outlier identification and monitoring of institutional or clinician performance: an overview of statistical methods and application to national audit data.

BACKGROUND: Institutions or clinicians (units) are often compared according to a performance indicator such as in-hospital mortality. Several approaches have been proposed for the detection of outlying units, whose performance deviates from the overall performance. METHODS: We provide an overview of three approaches commonly used to monitor institutional performances for outlier detection. These are the common-mean model, the 'Normal-Poisson' random effects model and the 'Logistic' random effects model. For the latter we also propose a visualisation technique. The common-mean model assumes that the underlying true performance of all units is equal and that any observed variation between units is due to chance. Even after applying case-mix adjustment, this assumption is often violated due to overdispersion and a post-hoc correction may need to be applied. The random effects models relax this assumption and explicitly allow the true performance to differ between units, thus offering a more flexible approach. We discuss the strengths and weaknesses of each approach and illustrate their application using audit data from England and Wales on Adult Cardiac Surgery (ACS) and Percutaneous Coronary Intervention (PCI). RESULTS: In general, the overdispersion-corrected common-mean model and the random effects approaches produced similar p-values for the detection of outliers. For the ACS dataset (41 hospitals) three outliers were identified in total but only one was identified by all methods above. For the PCI dataset (88 hospitals), seven outliers were identified in total but only two were identified by all methods. The common-mean model uncorrected for overdispersion produced several more outliers. The reason for observing similar p-values for all three approaches could be attributed to the fact that the between-hospital variance was relatively small in both datasets, resulting only in a mild violation of the common-mean assumption; in this situation, the overdispersion correction worked well. CONCLUSION: If the common-mean assumption is likely to hold, all three methods are appropriate to use for outlier detection and their results should be similar. Random effect methods may be the preferred approach when the common-mean assumption is likely to be violated.

Improving hospital quality risk-adjustment models using interactions identified by hierarchical group lasso regularisation.

BACKGROUND: Risk-adjustment (RA) models are used to account for severity of illness in comparing patient outcomes across hospitals. Researchers specify covariates as main effects, but they often ignore interactions or use stratification to account for effect modification, despite limitations due to rare events and sparse data. Three Agency for Healthcare Research and Quality (AHRQ) hospital-level Quality Indicators currently use stratified models, but their variable performance and limited interpretability motivated the design of better models. METHODS: We analysed patient discharge de-identified data from 14 State Inpatient Databases, AHRQ Healthcare Cost and Utilization Project, California Department of Health Care Access and Information, and New York State Department of Health. We used hierarchical group lasso regularisation (HGLR) to identify first-order interactions in several AHRQ inpatient quality indicators (IQI) - IQI 09 (Pancreatic Resection Mortality Rate), IQI 11 (Abdominal Aortic Aneurysm Repair Mortality Rate), and Patient Safety Indicator 14 (Postoperative Wound Dehiscence Rate). These models were compared with stratum-specific and composite main effects models with covariates selected by least absolute shrinkage and selection operator (LASSO). RESULTS: HGLR identified clinically meaningful interactions for all models. Synergistic IQI 11 interactions, such as between hypertension and respiratory failure, suggest patients who merit special attention in perioperative care. Antagonistic IQI 11 interactions, such as between shock and chronic comorbidities, illustrate that naïve main effects models overestimate risk in key subpopulations. Interactions for PSI 14 suggest key subpopulations for whom the risk of wound dehiscence is similar between open and laparoscopic approaches, whereas laparoscopic approach is safer for other groups. Model performance was similar or superior for composite models with HGLR-selected features, compared to those with LASSO-selected features. CONCLUSIONS: In this application to high-profile, high-stakes risk-adjustment models, HGLR selected interactions that maintained or improved model performance in populations with heterogeneous risk, while identifying clinically important interactions. The HGLR package is scalable to handle a large number of covariates and their interactions and is customisable to use multiple CPU cores to reduce analysis time. The HGLR method will allow scholars to avoid creating stratified models on sparse data, improve model calibration, and reduce bias. Future work involves testing using other combinations of risk factors, such as vital signs and laboratory values. Our study focuses on a real-world problem of considerable importance to hospitals and policy-makers who must use RA models for statutorily mandated public reporting and payment programmes.

Benchmarking short-term postoperative mortality across neurosurgery units: is hospital administrative data good enough for risk-adjustment?

BACKGROUND: Surgical mortality indicators should be risk-adjusted when evaluating the performance of organisations. This study evaluated the performance of risk-adjustment models that used English hospital administrative data for 30-day mortality after neurosurgery. METHODS: This retrospective cohort study used Hospital Episode Statistics (HES) data from 1 April 2013 to 31 March 2018. Organisational-level 30-day mortality was calculated for selected subspecialties (neuro-oncology, neurovascular and trauma neurosurgery) and the overall cohort. Risk adjustment models were developed using multivariable logistic regression and incorporated various patient variables: age, sex, admission method, social deprivation, comorbidity and frailty indices. Performance was assessed in terms of discrimination and calibration. RESULTS: The cohort included 49,044 patients. Overall, 30-day mortality rate was 4.9%, with unadjusted organisational rates ranging from 3.2 to 9.3%. The variables in the best performing models varied for the subspecialties; for trauma neurosurgery, a model that included deprivation and frailty had the best calibration, while for neuro-oncology a model with these variables plus comorbidity performed best. For neurovascular surgery, a simple model of age, sex and admission method performed best. Levels of discrimination varied for the subspecialties (range: 0.583 for trauma and 0.740 for neurovascular). The models were generally well calibrated. Application of the models to the organisation figures produced an average (median) absolute change in mortality of 0.33% (interquartile range (IQR) 0.15-0.72) for the overall cohort model. Median changes for the subspecialty models were 0.29% (neuro-oncology, IQR 0.15-0.42), 0.40% (neurovascular, IQR 0.24-0.78) and 0.49% (trauma neurosurgery, IQR 0.23-1.68). CONCLUSIONS: Reasonable risk-adjustment models for 30-day mortality after neurosurgery procedures were possible using variables from HES, although the models for trauma neurosurgery performed less well. Including a measure of frailty often improved model performance.

Validation of a neurovascular comorbidity index for risk adjustment of comorbid conditions among ischemic stroke patients receiving reperfusion treatment.

OBJECTIVE: To validate a comorbidity index specific to neurovascular patients and determine its performance relative to the Charlson Comorbidity Index (CCI) and Elixhauser Comorbidity Index (ECI) among ischemic stroke patients receiving reperfusion treatments. METHODS: Patients with ischemic stroke were identified in the National Inpatient Sample from Quarter 4 2015 to Quarter 4 2017. Ischemic stroke patients receiving reperfusion treatment, either with intravenous thrombolysis (IVT) only or mechanical thrombectomy (MT), were studied. The accuracy of the neurovascular comorbidities index (NCI) was compared to both the CCI and ECI in predicting in-hospital death and poor outcome (defined as death prior to discharge or discharge to a short-term hospital, a skilled nursing facility, an intermediate care facility, another long-term facility, or home health care). RESULTS: There were 25,147 ischemic stroke patients who received reperfusion therapy with either IVT only or MT (with or without IVT). Approximately 6.9% of patients died during their hospitalization, and 65.4% of patients were classified as having a poor outcome based on their discharge disposition. The NCI outperformed both the CCI and ECI in predicting in-hospital death (IVT only, p<0.0001; MT, p<0.0001) and poor outcomes (IVT only, p<0.0001; MT, p<0.0001). CONCLUSION: The NCI is a more powerful predictor of in-hospital death and poor outcomes when compared to the CCI or ECI among ischemic stroke patients receiving reperfusion therapies. Further validation studies are needed to confirm the accuracy of the NCI among other neurovascular patient populations.

[Risk-Adjustment of Return to Work After Medical Rehabilitation: Methodical Advancements in Quality Assurance of the German Pension Insurance]. / Adjustierung des sozialmedizinischen Verlaufs nach medizinischer Rehabilitation: methodische Weiterentwicklung der Reha-Qualitätssicherung der Deutschen Rentenversicherung.

PURPOSE: Besides the quality of life, patients' return to work is one of the most important treatment results of medical rehabilitation paid by the German Pension Insurance. In order to be able to use the return to work as a quality indicator for medical rehabilitation, a risk adjustment strategy for pre-existing characteristics of patients, rehabilitation departments and labour markets had to be developed. METHODS: Multiple regression analyses and cross validation were used to develop a risk adjustment strategy, which mathematically compensates the influence of confounders and thus allows for appropriate comparisons between rehabilitation departments regarding patients' return to work after medical rehabilitation. Under the inclusion of experts, the number of employment days in the first and second year after medical rehabilitation were chosen as an appropriate operationalization of return to work. Methodological challenges in the development of the risk adjustment strategy were the identification of a suitable regression method for the distribution of the dependent variable, modelling the multilevel structure of the data appropriately and selecting relevant confounders for return to work. A user-friendly way of communicating the results was developed. RESULTS: The fractional logit regression was chosen as an appropriate regression method to model the U-shaped distribution of the employment days. Low intraclass correlations indicate that the multilevel structure of the data (cross-classified labour market regions and rehabilitation departments) is statistically negligible. Potential confounding factors were theoretically preselected (medical experts were involved for medical parameters) and tested for their prognostic relevance in each indication area using backwards selection. Cross validations proved the risk adjustment strategy to be stable. Adjustment results were displayed in a user-friendly report, including the users' perspective (focus groups and interviews). CONCLUSIONS: The developed risk adjustment strategy allows for adequate comparisons between rehabilitation departments and thus enables a quality assessment of treatment results. Methodological challenges, decisions and limitations are discussed in details throughout this paper.