Uncertainty aware training to improve deep learning model calibration for classification of cardiac MR images.

Quantifying uncertainty of predictions has been identified as one way to develop more trustworthy artificial intelligence (AI) models beyond conventional reporting of performance metrics. When considering their role in a clinical decision support setting, AI classification models should ideally avoid confident wrong predictions and maximise the confidence of correct predictions. Models that do this are said to be well calibrated with regard to confidence. However, relatively little attention has been paid to how to improve calibration when training these models, i.e. to make the training strategy uncertainty-aware. In this work we: (i) evaluate three novel uncertainty-aware training strategies with regard to a range of accuracy and calibration performance measures, comparing against two state-of-the-art approaches, (ii) quantify the data (aleatoric) and model (epistemic) uncertainty of all models and (iii) evaluate the impact of using a model calibration measure for model selection in uncertainty-aware training, in contrast to the normal accuracy-based measures. We perform our analysis using two different clinical applications: cardiac resynchronisation therapy (CRT) response prediction and coronary artery disease (CAD) diagnosis from cardiac magnetic resonance (CMR) images. The best-performing model in terms of both classification accuracy and the most common calibration measure, expected calibration error (ECE) was the Confidence Weight method, a novel approach that weights the loss of samples to explicitly penalise confident incorrect predictions. The method reduced the ECE by 17% for CRT response prediction and by 22% for CAD diagnosis when compared to a baseline classifier in which no uncertainty-aware strategy was included. In both applications, as well as reducing the ECE there was a slight increase in accuracy from 69% to 70% and 70% to 72% for CRT response prediction and CAD diagnosis respectively. However, our analysis showed a lack of consistency in terms of optimal models when using different calibration measures. This indicates the need for careful consideration of performance metrics when training and selecting models for complex high risk applications in healthcare.

The International Prognostic Score and HIV status predict red cell concentrate transfusion needs in Hodgkin lymphoma.

Despite the burden of anemia among Hodgkin lymphoma (HL) patients, data evaluating red cell concentrate transfusion are limited. We retrospectively studied 285 newly diagnosed HL patients who received first-line adriamycin, bleomycin sulfate, vinblastine sulfate, and dacarbazine (ABVD) treatment at Groote Schuur Hospital, Cape Town. HIV prevalence in the cohort was 39.5% and 74.2% of patients had advanced stage HL. Patient prognosis was scored using the HL International Prognostic Score (IPS-7) and HL IPS-3. Seventy (24.6%) patients were transfused with a median of 2 (IQR 1-5) units per patient. Compared to HIV-negative patients, more HIV-positive patients were transfused (14.1% vs. 40.4%, p < .001) and received more units, median 2 (IQR 1-3) vs. 3 (IQR 2-5), p = .035. HL IPS-7 (OR 2.1, p < .001) and HL IPS-3 (OR 2.6, p < .001) were independently associated with transfusion. HL IPS-7, HL IPS-3, and HIV positivity remained associated with transfusion after adjusting for covariates. For patients with newly diagnosed HL, HL IPS-7, HL IPS-3, and HIV status predicted transfusion.