Are better AI algorithms for breast cancer detection also better at predicting risk? A paired case-control study.

BACKGROUND: There is increasing evidence that artificial intelligence (AI) breast cancer risk evaluation tools using digital mammograms are highly informative for 1-6 years following a negative screening examination. We hypothesized that algorithms that have previously been shown to work well for cancer detection will also work well for risk assessment and that performance of algorithms for detection and risk assessment is correlated. METHODS: To evaluate our hypothesis, we designed a case-control study using paired mammograms at diagnosis and at the previous screening visit. The study included n = 3386 women from the OPTIMAM registry, that includes mammograms from women diagnosed with breast cancer in the English breast screening program 2010-2019. Cases were diagnosed with invasive breast cancer or ductal carcinoma in situ at screening and were selected if they had a mammogram available at the screening examination that led to detection, and a paired mammogram at their previous screening visit 3y prior to detection when no cancer was detected. Controls without cancer were matched 1:1 to cases based on age (year), screening site, and mammography machine type. Risk assessment was conducted using a deep-learning model designed for breast cancer risk assessment (Mirai), and three open-source deep-learning algorithms designed for breast cancer detection. Discrimination was assessed using a matched area under the curve (AUC) statistic. RESULTS: Overall performance using the paired mammograms followed the same order by algorithm for risk assessment (AUC range 0.59-0.67) and detection (AUC 0.81-0.89), with Mirai performing best for both. There was also a correlation in performance for risk and detection within algorithms by cancer size, with much greater accuracy for large cancers (30 mm+, detection AUC: 0.88-0.92; risk AUC: 0.64-0.74) than smaller cancers (0 to < 10 mm, detection AUC: 0.73-0.86, risk AUC: 0.54-0.64). Mirai was relatively strong for risk assessment of smaller cancers (0 to < 10 mm, risk, Mirai AUC: 0.64 (95% CI 0.57 to 0.70); other algorithms AUC 0.54-0.56). CONCLUSIONS: Improvements in risk assessment could stem from enhancing cancer detection capabilities of smaller cancers. Other state-of-the-art AI detection algorithms with high performance for smaller cancers might achieve relatively high performance for risk assessment.

An optimization framework to guide the choice of thresholds for risk-based cancer screening.

It is uncommon for risk groups defined by statistical or artificial intelligence (AI) models to be chosen by jointly considering model performance and potential interventions available. We develop a framework to rapidly guide choice of risk groups in this manner, and apply it to guide breast cancer screening intervals using an AI model. Linear programming is used to define risk groups that minimize expected advanced cancer incidence subject to resource constraints. In the application risk stratification performance is estimated from a case-control study (2044 cases, 1:1 matching), and other parameters are taken from screening trials and the screening programme in England. Under the model, re-screening in 1 year for the highest 4% AI model risk, in 3 years for the middle 64%, and in 4 years for 32% of the population at lowest risk, was expected to reduce the number of advanced cancers diagnosed by approximately 18 advanced cancers per 1000 diagnosed with triennial screening, for the same average number of screens in the population as triennial screening for all. Sensitivity analyses found the choice of thresholds was robust to model parameters, but the estimated reduction in advanced cancers was not precise and requires further evaluation. Our framework helps define thresholds with the greatest chance of success for reducing the population health burden of cancer when used in risk-adapted screening, which should be further evaluated such as in health-economic modelling based on computer simulation models, and real-world evaluations.

Evaluation of an AI Model to Assess Future Breast Cancer Risk.

Background Accurate breast cancer risk assessment after a negative screening result could enable better strategies for early detection. Purpose To evaluate a deep learning algorithm for risk assessment based on digital mammograms. Materials and Methods A retrospective observational matched case-control study was designed using the OPTIMAM Mammography Image Database from the National Health Service Breast Screening Programme in the United Kingdom from February 2010 to September 2019. Patients with breast cancer (cases) were diagnosed following a mammographic screening or between two triannual screening rounds. Controls were matched based on mammography device, screening site, and age. The artificial intelligence (AI) model only used mammograms at screening before diagnosis. The primary objective was to assess model performance, with a secondary objective to assess heterogeneity and calibration slope. The area under the receiver operating characteristic curve (AUC) was estimated for 3-year risk. Heterogeneity according to cancer subtype was assessed using a likelihood ratio interaction test. Statistical significance was set at P < .05. Results Analysis included patients with screen-detected (median age, 60 years [IQR, 55-65 years]; 2044 female, including 1528 with invasive cancer and 503 with ductal carcinoma in situ [DCIS]) or interval (median age, 59 years [IQR, 53-65 years]; 696 female, including 636 with invasive cancer and 54 with DCIS) breast cancer and 1:1 matched controls, each with a complete set of mammograms at the screening preceding diagnosis. The AI model had an overall AUC of 0.68 (95% CI: 0.66, 0.70), with no evidence of a significant difference between interval and screen-detected (AUC, 0.69 vs 0.67; P = .085) cancer. The calibration slope was 1.13 (95% CI: 1.01, 1.26). There was similar performance for the detection of invasive cancer versus DCIS (AUC, 0.68 vs 0.66; P = .057). The model had higher performance for advanced cancer risk (AUC, 0.72 ≥stage II vs 0.66