RESUMO
OBJECTIVE: Despite increased availability of methodologies to identify algorithmic bias, the operationalization of bias evaluation for healthcare predictive models is still limited. Therefore, this study proposes a process for bias evaluation through an empirical assessment of common hospital readmission models. The process includes selecting bias measures, interpretation, determining disparity impact and potential mitigations. METHODS: This retrospective analysis evaluated racial bias of four common models predicting 30-day unplanned readmission (i.e., LACE Index, HOSPITAL Score, and the CMS readmission measure applied as is and retrained). The models were assessed using 2.4 million adult inpatient discharges in Maryland from 2016 to 2019. Fairness metrics that are model-agnostic, easy to compute, and interpretable were implemented and apprised to select the most appropriate bias measures. The impact of changing model's risk thresholds on these measures was further assessed to guide the selection of optimal thresholds to control and mitigate bias. RESULTS: Four bias measures were selected for the predictive task: zero-one-loss difference, false negative rate (FNR) parity, false positive rate (FPR) parity, and generalized entropy index. Based on these measures, the HOSPITAL score and the retrained CMS measure demonstrated the lowest racial bias. White patients showed a higher FNR while Black patients resulted in a higher FPR and zero-one-loss. As the models' risk threshold changed, trade-offs between models' fairness and overall performance were observed, and the assessment showed all models' default thresholds were reasonable for balancing accuracy and bias. CONCLUSIONS: This study proposes an Applied Framework to Assess Fairness of Predictive Models (AFAFPM) and demonstrates the process using 30-day hospital readmission model as the example. It suggests the feasibility of applying algorithmic bias assessment to determine optimized risk thresholds so that predictive models can be used more equitably and accurately. It is evident that a combination of qualitative and quantitative methods and a multidisciplinary team are necessary to identify, understand and respond to algorithm bias in real-world healthcare settings. Users should also apply multiple bias measures to ensure a more comprehensive, tailored, and balanced view. The results of bias measures, however, must be interpreted with caution and consider the larger operational, clinical, and policy context.
RESUMO
BACKGROUND: This study provides an updated and expanded analysis of the impact of the COVID-19 pandemic on routine vaccinations across the life-course in the United States. RESEARCH DESIGN AND METHODS: Routine wellness visits and vaccination rates were calculated using structured claims data for each month during the impact period (January 2020 to August 2022) and compared to the respective baseline period (January 2018 to December 2019). Monthly rates were aggregated as annual accumulated and cumulative percent changes. RESULTS: The complete monthly rate interactive dataset can be viewed at https://vaccinationtrends.com. The greatest decrease in annual accumulated administration rates in the 0-2 and 4-6 years age groups was for the measles, mumps, and rubella vaccine; for adolescents and older adults, it was for human papillomavirus and pneumococcal vaccines, respectively. Routine in-person wellness visit rates recovered faster and more completely than vaccination rates in all age groups, indicating potential missed opportunities to administer vaccines during visits. CONCLUSIONS: This updated analysis reveals that the negative impact of the COVID-19 pandemic on routine vaccination continued through 2021 and into 2022. Proactive efforts to reverse this decline are needed to increase individual- and population-level vaccination coverage and avoid the associated preventable morbidity, mortality, and health care costs.