Evaluating equity in performance of an electronic health record-based 6-month mortality risk model to trigger palliative care consultation: a retrospective model validation analysis.

OBJECTIVE: Evaluate predictive performance of an electronic health record (EHR)-based, inpatient 6-month mortality risk model developed to trigger palliative care consultation among patient groups stratified by age, race, ethnicity, insurance and socioeconomic status (SES), which may vary due to social forces (eg, racism) that shape health, healthcare and health data. DESIGN: Retrospective evaluation of prediction model. SETTING: Three urban hospitals within a single health system. PARTICIPANTS: All patients ≥18 years admitted between 1 January and 31 December 2017, excluding observation, obstetric, rehabilitation and hospice (n=58 464 encounters, 41 327 patients). MAIN OUTCOME MEASURES: General performance metrics (c-statistic, integrated calibration index (ICI), Brier Score) and additional measures relevant to health equity (accuracy, false positive rate (FPR), false negative rate (FNR)). RESULTS: For black versus non-Hispanic white patients, the model's accuracy was higher (0.051, 95% CI 0.044 to 0.059), FPR lower (-0.060, 95% CI -0.067 to -0.052) and FNR higher (0.049, 95% CI 0.023 to 0.078). A similar pattern was observed among patients who were Hispanic, younger, with Medicaid/missing insurance, or living in low SES zip codes. No consistent differences emerged in c-statistic, ICI or Brier Score. Younger age had the second-largest effect size in the mortality prediction model, and there were large standardised group differences in age (eg, 0.32 for non-Hispanic white versus black patients), suggesting age may contribute to systematic differences in the predicted probabilities between groups. CONCLUSIONS: An EHR-based mortality risk model was less likely to identify some marginalised patients as potentially benefiting from palliative care, with younger age pinpointed as a possible mechanism. Evaluating predictive performance is a critical preliminary step in addressing algorithmic inequities in healthcare, which must also include evaluating clinical impact, and governance and regulatory structures for oversight, monitoring and accountability.

Exploring the impact of missingness on racial disparities in predictive performance of a machine learning model for emergency department triage.

Objective: To investigate how missing data in the patient problem list may impact racial disparities in the predictive performance of a machine learning (ML) model for emergency department (ED) triage. Materials and Methods: Racial disparities may exist in the missingness of EHR data (eg, systematic differences in access, testing, and/or treatment) that can impact model predictions across racialized patient groups. We use an ML model that predicts patients' risk for adverse events to produce triage-level recommendations, patterned after a clinical decision support tool deployed at multiple EDs. We compared the model's predictive performance on sets of observed (problem list data at the point of triage) versus manipulated (updated to the more complete problem list at the end of the encounter) test data. These differences were compared between Black and non-Hispanic White patient groups using multiple performance measures relevant to health equity. Results: There were modest, but significant, changes in predictive performance comparing the observed to manipulated models across both Black and non-Hispanic White patient groups; c-statistic improvement ranged between 0.027 and 0.058. The manipulation produced no between-group differences in c-statistic by race. However, there were small between-group differences in other performance measures, with greater change for non-Hispanic White patients. Discussion: Problem list missingness impacted model performance for both patient groups, with marginal differences detected by race. Conclusion: Further exploration is needed to examine how missingness may contribute to racial disparities in clinical model predictions across settings. The novel manipulation method demonstrated may aid future research.

Effects of Neighborhood-level Data on Performance and Algorithmic Equity of a Model That Predicts 30-day Heart Failure Readmissions at an Urban Academic Medical Center.

BACKGROUND: Socioeconomic data may improve predictions of clinical events. However, owing to structural racism, algorithms may not perform equitably across racial subgroups. Therefore, we sought to compare the predictive performance overall, and by racial subgroup, of commonly used predictor variables for heart failure readmission with and without the area deprivation index (ADI), a neighborhood-level socioeconomic measure. METHODS AND RESULTS: We conducted a retrospective cohort study of 1316 Philadelphia residents discharged with a primary diagnosis of congestive heart failure from the University of Pennsylvania Health System between April 1, 2015, and March 31, 2017. We trained a regression model to predict the probability of a 30-day readmission using clinical and demographic variables. A second model also included the ADI as a predictor variable. We measured predictive performance with the Brier Score (BS) in a held-out test set. The baseline model had moderate performance overall (BS 0.13, 95% CI 0.13-0.14), and among White (BS 0.12, 95% CI 0.12-0.13) and non-White (BS 0.13, 95% CI 0.13-0.14) patients. Neither performance nor algorithmic equity were significantly changed with the addition of the ADI. CONCLUSIONS: The inclusion of neighborhood-level data may not reliably improve performance or algorithmic equity.

Factors Associated with Loss to Follow-up among Cervical Cancer Patients in Rwanda.

Background: Cervical cancer is among the most common cancers affecting women globally. Where treatment is available in low- and middle-income countries, many women become lost to follow-up (LTFU) at various points of care. Objective: This study assessed predictors of LTFU among cervical cancer patients in rural Rwanda. Methods: We conducted a retrospective study of cervical cancer patients enrolled at Butaro Cancer Center of Excellence (BCCOE) between 2012 and 2017 who were either alive and in care or LTFU at 12 months after enrollment. Patients are considered early LTFU if they did not return to clinic after the first visit and late LTFU if they did not return to clinic after the second visit. We conducted two multivariable logistic regressions to determine predictors of early and late LTFU. Findings: Of 652 patients in the program, 312 women met inclusion criteria, of whom 47 (15.1%) were early LTFU, 78 (25.0%) were late LTFU and 187 (59.9%) were alive and in care. In adjusted analyses, patients with no documented disease stage at presentation were more likely to be early LTFU vs. patients with stage 1 and 2 when controlling for other factors (aOR: 14.93, 95% CI 6.12-36.43). Patients who travel long distances (aOR: 2.25, 95% CI 1.11, 4.53), with palliative care as type of treatment received (aOR: 6.65, CI 2.28, 19.40) and patients with missing treatment (aOR: 7.99, CI 3.56, 17.97) were more likely to be late LTFU when controlling for other factors. Patients with ECOG status of 2 and higher were less likely to be late LTFU (aOR: 0.26, 95% CI 0.08, 0.85). Conclusion: Different factors were associated with early and later LTFU. Enhanced patient education, mechanisms to facilitate diagnosis at early stages of disease, and strategies that improve patient tracking and follow-up may reduce LTFU and improve patient retention.

Recognizing Racism in Medicine: A Student-Organized and Community-Engaged Health Professional Conference.

Purpose: This piece details the evaluation and implementation of a student-led educational intervention designed to train health professionals on the impact of racism in health care and provide tools to mitigate it. In addition, this conference, cosponsored by medical, nursing, and social work training programs, facilitates development of networks of providers with the knowledge and skills to recognize and address racism in health care. Methods: The conference included 2 keynote speakers, an interprofessional panel, and 15 workshops. Participants (n=220) were asked to complete a survey assessing perceptions of conference content and impact. We compared responses pre- and postconference using Wilcoxon signed-rank tests. Results: Of the survey respondents (n=44), 45.5% were medical students, 13.6% nursing students, and 9% social work students; 65.9% self-identified as a race/ethnicity other than non-Hispanic white; and 63.6% self-identified as female. We found that 47.7% respondents reported they were more comfortable discussing how racism affects health (p<0.001), 36.4% had better understanding of the impact of racism on an individual's health (p<0.001), and 54.5% felt more connected to other health professionals working to recognize and address racism in medicine (p<0.001). Conclusion: These findings suggest that a student-organized conference could potentially be an effective strategy in addressing a critical gap in racism training for health care professionals.

Small area estimation of under-5 mortality in Bangladesh, Cameroon, Chad, Mozambique, Uganda, and Zambia using spatially misaligned data.

BACKGROUND: The under-5 mortality rate (U5MR) is an important metric of child health and survival. Country-level estimates of U5MR are readily available, but efforts to estimate U5MR subnationally have been limited, in part, due to spatial misalignment of available data sources (e.g., use of different administrative levels, or as a result of historical boundary changes). METHODS: We analyzed all available complete and summary birth history data in surveys and censuses in six countries (Bangladesh, Cameroon, Chad, Mozambique, Uganda, and Zambia) at the finest geographic level available in each data source. We then developed small area estimation models capable of incorporating spatially misaligned data. These small area estimation models were applied to the birth history data in order to estimate trends in U5MR from 1980 to 2015 at the second administrative level in Cameroon, Chad, Mozambique, Uganda, and Zambia and at the third administrative level in Bangladesh. RESULTS: We found substantial variation in U5MR in all six countries: there was more than a two-fold difference in U5MR between the area with the highest rate and the area with the lowest rate in every country. All areas in all countries experienced declines in U5MR between 1980 and 2015, but the degree varied both within and between countries. In Cameroon, Chad, Mozambique, and Zambia we found areas with U5MRs in 2015 that were higher than in other parts of the same country in 1980. Comparing subnational U5MR to country-level targets for the Millennium Development Goals (MDG), we find that 12.8% of areas in Bangladesh did not meet the country-level target, although the country as whole did. A minority of areas in Chad, Mozambique, Uganda, and Zambia met the country-level MDG targets while these countries as a whole did not. CONCLUSIONS: Subnational estimates of U5MR reveal significant within-country variation. These estimates could be used for identifying high-need areas and positive deviants, tracking trends in geographic inequalities, and evaluating progress towards international development targets such as the Sustainable Development Goals.