Fairness in Predicting Cancer Mortality Across Racial Subgroups.

Importance: Machine learning has potential to transform cancer care by helping clinicians prioritize patients for serious illness conversations. However, models need to be evaluated for unequal performance across racial groups (ie, racial bias) so that existing racial disparities are not exacerbated. Objective: To evaluate whether racial bias exists in a predictive machine learning model that identifies 180-day cancer mortality risk among patients with solid malignant tumors. Design, Setting, and Participants: In this cohort study, a machine learning model to predict cancer mortality for patients aged 21 years or older diagnosed with cancer between January 2016 and December 2021 was developed with a random forest algorithm using retrospective data from the Mount Sinai Health System cancer registry, Social Security Death Index, and electronic health records up to the date when databases were accessed for cohort extraction (February 2022). Exposure: Race category. Main Outcomes and Measures: The primary outcomes were model discriminatory performance (area under the receiver operating characteristic curve [AUROC], F1 score) among each race category (Asian, Black, Native American, White, and other or unknown) and fairness metrics (equal opportunity, equalized odds, and disparate impact) among each pairwise comparison of race categories. True-positive rate ratios represented equal opportunity; both true-positive and false-positive rate ratios, equalized odds; and the percentage of predictive positive rate ratios, disparate impact. All metrics were estimated as a proportion or ratio, with variability captured through 95% CIs. The prespecified criterion for the model's clinical use was a threshold of at least 80% for fairness metrics across different racial groups to ensure the model's prediction would not be biased against any specific race. Results: The test validation dataset included 43 274 patients with balanced demographics. Mean (SD) age was 64.09 (14.26) years, with 49.6% older than 65 years. A total of 53.3% were female; 9.5%, Asian; 18.9%, Black; 0.1%, Native American; 52.2%, White; and 19.2%, other or unknown race; 0.1% had missing race data. A total of 88.9% of patients were alive, and 11.1% were dead. The AUROCs, F1 scores, and fairness metrics maintained reasonable concordance among the racial subgroups: the AUROCs ranged from 0.75 (95% CI, 0.72-0.78) for Asian patients and 0.75 (95% CI, 0.73-0.77) for Black patients to 0.77 (95% CI, 0.75-0.79) for patients with other or unknown race; F1 scores, from 0.32 (95% CI, 0.32-0.33) for White patients to 0.40 (95% CI, 0.39-0.42) for Black patients; equal opportunity ratios, from 0.96 (95% CI, 0.95-0.98) for Black patients compared with White patients to 1.02 (95% CI, 1.00-1.04) for Black patients compared with patients with other or unknown race; equalized odds ratios, from 0.87 (95% CI, 0.85-0.92) for Black patients compared with White patients to 1.16 (1.10-1.21) for Black patients compared with patients with other or unknown race; and disparate impact ratios, from 0.86 (95% CI, 0.82-0.89) for Black patients compared with White patients to 1.17 (95% CI, 1.12-1.22) for Black patients compared with patients with other or unknown race. Conclusions and Relevance: In this cohort study, the lack of significant variation in performance or fairness metrics indicated an absence of racial bias, suggesting that the model fairly identified cancer mortality risk across racial groups. It remains essential to consistently review the model's application in clinical settings to ensure equitable patient care.

Is the Epworth Sleepiness scale a valid outcome measure to evaluate the effectiveness of positive airway pressure treatments on daytime sleepiness? Psychometric insights from measurement invariance and response shifts.

PURPOSE: Although the validity of the Epworth Sleepiness Scale (ESS) as an effectiveness measure for sleep apnea treatments such as continuous positive airway pressure (CPAP) has been supported by multiple studies, some researchers continue to challenge it. They suggest that in addition to its impact on relieving patients' daytime sleepiness, CPAP also alters the internal standards patients use to evaluate their sleepiness (i.e., response shift; RS), confounding the meaning of the difference in the ESS scores. We believe an issue yet to be addressed in this debate is that all existing evidence of RS has been obtained through the then-test approach, a retrospective method sensitive to various cognitive mechanisms. Thus, in the current study, we re-examined this issue using the structural equation modeling (SEM) approach, a method that can be directly applied to randomized clinical trial (RCT) data without retrospective measures. METHODS: With the ESS data from two independent RCTs, we conducted cross-sectional and longitudinal measure invariance tests in SEM to examine whether CPAP would lead to RS. RESULTS: The ESS demonstrated cross-sectional and longitudinal scalar invariance against CPAP treatments. Its factorial pattern, loadings, and thresholds were invariant between the treatment and control groups and pre- and post-treatment, supporting the comparability of the observed mean ESS scores across time and groups. CONCLUSION: Our results support the validity of the average difference scores of the ESS for quantifying the effectiveness of CPAP on group-level daytime sleepiness in RCTs with relatively large sample sizes.

Dealing with missing data in multi-informant studies: A comparison of approaches.

Multi-informant studies are popular in social and behavioral science. However, their data analyses are challenging because data from different informants carry both shared and unique information and are often incomplete. Using Monte Carlo Simulation, the current study compares three approaches that can be used to analyze incomplete multi-informant data when there is a distinction between reference and nonreference informants. These approaches include a two-method measurement model for planned missing data (2MM-PMD), treating nonreference informants' reports as auxiliary variables with the full-information maximum likelihood method or multiple imputation, and listwise deletion. The result suggests that 2MM-PMD, when correctly specified and data are missing at random, has the best overall performance among the examined approaches regarding point estimates, type I error rates, and statistical power. In addition, it is also more robust to data that are not missing at random.