Analysis of tumor nuclear features using artificial intelligence to predict response to neoadjuvant chemotherapy in high-risk breast cancer patients.

PURPOSE: Neoadjuvant chemotherapy (NAC) is used to treat patients with high-risk breast cancer. The tumor response to NAC can be classified as either a pathological partial response (pPR) or pathological complete response (pCR), defined as complete eradication of invasive tumor cells, with a pCR conferring a significantly lower risk of recurrence. Predicting the response to NAC, however, remains a significant clinical challenge. The objective of this study was to determine if analysis of nuclear features on core biopsies using artificial intelligence (AI) can predict response to NAC. METHODS: Fifty-eight HER2-positive or triple-negative breast cancer patients were included in this study (pCR n = 37, pPR n = 21). Multiple deep convolutional neural networks were developed to automate tumor detection and nuclear segmentation. Nuclear count, area, and circularity, as well as image-based first- and second-order features including mean pixel intensity and correlation of the gray-level co-occurrence matrix (GLCM-COR) were determined. RESULTS: In univariate analysis, the pCR group had fewer multifocal/multicentric tumors, higher nuclear intensity, and lower GLCM-COR compared to the pPR group. In multivariate binary logistic regression, tumor multifocality/multicentricity (OR = 0.14, p = 0.012), nuclear intensity (OR = 1.23, p = 0.018), and GLCM-COR (OR = 0.96, p = 0.043) were each independently associated with likelihood of achieving a pCR, and the model was able to successful classify 79% of cases (62% for pPR and 89% for pCR). CONCLUSION: Analysis of tumor nuclear features using digital pathology/AI can significantly improve models to predict pathological response to NAC.

Computational Radiology in Breast Cancer Screening and Diagnosis Using Artificial Intelligence.

Breast cancer screening has been shown to significantly reduce mortality in women. The increased utilization of screening examinations has led to growing demands for rapid and accurate diagnostic reporting. In modern breast imaging centers, full-field digital mammography (FFDM) has replaced traditional analog mammography, and this has opened new opportunities for developing computational frameworks to automate detection and diagnosis. Artificial intelligence (AI), and its subdomain of deep learning, is showing promising results and improvements on diagnostic accuracy, compared to previous computer-based methods, known as computer-aided detection and diagnosis.In this commentary, we review the current status of computational radiology, with a focus on deep neural networks used in breast cancer screening and diagnosis. Recent studies are developing a new generation of computer-aided detection and diagnosis systems, as well as leveraging AI-driven tools to efficiently interpret digital mammograms, and breast tomosynthesis imaging. The use of AI in computational radiology necessitates transparency and rigorous testing. However, the overall impact of AI to radiology workflows will potentially yield more efficient and standardized processes as well as improve the level of care to patients with high diagnostic accuracy.

Predicting breast cancer response to neoadjuvant chemotherapy using pretreatment diffuse optical spectroscopic texture analysis.

BACKGROUND: Diffuse optical spectroscopy (DOS) has been demonstrated capable of monitoring response to neoadjuvant chemotherapy (NAC) in locally advanced breast cancer (LABC) patients. In this study, we evaluate texture features of pretreatment DOS functional maps for predicting LABC response to NAC. METHODS: Locally advanced breast cancer patients (n=37) underwent DOS breast imaging before starting NAC. Breast tissue parametric maps were constructed and texture analyses were performed based on grey-level co-occurrence matrices for feature extraction. Ground truth labels as responders (R) or non-responders (NR) were assigned to patients based on Miller-Payne pathological response criteria. The capability of DOS textural features computed on volumetric tumour data before the start of treatment (i.e., 'pretreatment') to predict patient responses to NAC was evaluated using a leave-one-out validation scheme at subject level. Data were analysed using a logistic regression, naive Bayes, and k-nearest neighbour classifiers. RESULTS: Data indicated that textural characteristics of pretreatment DOS parametric maps can differentiate between treatment response outcomes. The HbO2 homogeneity resulted in the highest accuracy among univariate parameters in predicting response to chemotherapy: sensitivity (%Sn) and specificity (%Sp) were 86.5% and 89.0%, respectively, and accuracy was 87.8%. The highest predictors using multivariate (binary) combination features were the Hb-contrast+HbO2-homogeneity, which resulted in a %Sn/%Sp=78.0/81.0% and an accuracy of 79.5%. CONCLUSIONS: This study demonstrated that the pretreatment DOS texture features can predict breast cancer response to NAC and potentially guide treatments.

Quantitative ultrasound radiomics guided adaptive neoadjuvant chemotherapy in breast cancer: early results from a randomized feasibility study.

Background: In patients with locally advanced breast cancer (LABC) receiving neoadjuvant chemotherapy (NAC), quantitative ultrasound (QUS) radiomics can predict final responses early within 4 of 16-18 weeks of treatment. The current study was planned to study the feasibility of a QUS-radiomics model-guided adaptive chemotherapy. Methods: The phase 2 open-label randomized controlled trial included patients with LABC planned for NAC. Patients were randomly allocated in 1:1 ratio to a standard arm or experimental arm stratified by hormonal receptor status. All patients were planned for standard anthracycline and taxane-based NAC as decided by their medical oncologist. Patients underwent QUS imaging using a clinical ultrasound device before the initiation of NAC and after the 1st and 4th weeks of treatment. A support vector machine-based radiomics model developed from an earlier cohort of patients was used to predict treatment response at the 4th week of NAC. In the standard arm, patients continued to receive planned chemotherapy with the treating oncologists blinded to results. In the experimental arm, the QUS-based prediction was conveyed to the responsible oncologist, and any changes to the planned chemotherapy for predicted non-responders were made by the responsible oncologist. All patients underwent surgery following NAC, and the final response was evaluated based on histopathological examination. Results: Between June 2018 and July 2021, 60 patients were accrued in the study arm, with 28 patients in each arm available for final analysis. In patients without a change in chemotherapy regimen (53 of 56 patients total), the QUS-radiomics model at week 4 of NAC that was used demonstrated an accuracy of 97%, respectively, in predicting the final treatment response. Seven patients were predicted to be non-responders (observational arm (n=2), experimental arm (n=5)). Three of 5 non-responders in the experimental arm had chemotherapy regimens adapted with an early initiation of taxane therapy or chemotherapy intensification, or early surgery and ended up as responders on final evaluation. Conclusion: The study demonstrates the feasibility of QUS-radiomics adapted guided NAC for patients with breast cancer. The ability of a QUS-based model in the early prediction of treatment response was prospectively validated in the current study. Clinical trial registration: clinicaltrials.gov, ID NCT04050228.

A novel tissue mechanics-based method for improved motion tracking in quasi-static ultrasound elastography.

PURPOSE: Most cancers are associated with biological and structural changes that lead to tissue stiffening. Therefore, imaging tissue stiffness using quasi-static ultrasound elastography (USE) can potentially be effective in cancer diagnosis. USE techniques developed for stiffness image reconstruction use noisy displacement data to obtain the stiffness images. In this study, we propose a technique to substantially improve the accuracy of the displacement data computed through ultrasound tissue motion tracking techniques, especially in the lateral direction. METHODS: The proposed technique uses mathematical constraints derived from fundamental tissue mechanics principles to regularize displacement and strain fields obtained using Global Ultrasound Elastography (GLUE) and Second-Order Ultrasound Elastography (SOUL) methods. The principles include a novel technique to enforce (1) tissue incompressibility using 3D Boussinesq model and (2) deformation compatibility using the compatibility differential equation. The technique was validated thoroughly using metrics pertaining to Signal-to-Noise-Ratio (SNR), Contrast-to-Noise-Ratio (CNR) and Normalized Cross Correlation (NCC) for four tissue-mimicking phantom models and two clinical breast ultrasound elastography cases. RESULTS: The results show substantial improvement in the displacement and strain images generated using the proposed technique. The tissue-mimicking phantom study results indicate that the proposed method is superior in improving image quality compared to the GLUE and SOUL techniques as it shows an average axial strain SNR and CNR improvement of 44% and 63%, and lateral strain SNR and CNR improvement of 130% and 435%, respectively. The results of the phantom study also indicate higher accuracy of displacement images obtained using the proposed technique, including improvement ranges of 7-84% and 26-140% for axial and lateral displacement images, respectively. For the clinical cases, the results indicate average improvement of 48% and 64% in SNR and CNR, respectively, in the axial strain images, and average improvement of 40% and 41% in SNR and CNR, respectively, in the lateral strain images. CONCLUSION: The proposed method is very effective in producing improved estimate of tissue displacement and strain images, especially with the lateral displacement and strain where the improvement is highly remarkable. While the method shows promise for clinical applications, further investigation is necessary for rigorous assessment of the method's performance in the clinic.

A hierarchical self-attention-guided deep learning framework to predict breast cancer response to chemotherapy using pre-treatment tumor biopsies.

BACKGROUND: Pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) has demonstrated a strong correlation to improved survival in breast cancer (BC) patients. However, pCR rates to NAC are less than 30%, depending on the BC subtype. Early prediction of NAC response would facilitate therapeutic modifications for individual patients, potentially improving overall treatment outcomes and patient survival. PURPOSE: This study, for the first time, proposes a hierarchical self-attention-guided deep learning framework to predict NAC response in breast cancer patients using digital histopathological images of pre-treatment biopsy specimens. METHODS: Digitized hematoxylin and eosin-stained slides of BC core needle biopsies were obtained from 207 patients treated with NAC, followed by surgery. The response to NAC for each patient was determined using the standard clinical and pathological criteria after surgery. The digital pathology images were processed through the proposed hierarchical framework consisting of patch-level and tumor-level processing modules followed by a patient-level response prediction component. A combination of convolutional layers and transformer self-attention blocks were utilized in the patch-level processing architecture to generate optimized feature maps. The feature maps were analyzed through two vision transformer architectures adapted for the tumor-level processing and the patient-level response prediction components. The feature map sequences for these transformer architectures were defined based on the patch positions within the tumor beds and the bed positions within the biopsy slide, respectively. A five-fold cross-validation at the patient level was applied on the training set (144 patients with 9430 annotated tumor beds and 1,559,784 patches) to train the models and optimize the hyperparameters. An unseen independent test set (63 patients with 3574 annotated tumor beds and 173,637 patches) was used to evaluate the framework. RESULTS: The obtained results on the test set showed an AUC of 0.89 and an F1-score of 90% for predicting pCR to NAC a priori by the proposed hierarchical framework. Similar frameworks with the patch-level, patch-level + tumor-level, and patch-level + patient-level processing components resulted in AUCs of 0.79, 0.81, and 0.84 and F1-scores of 86%, 87%, and 89%, respectively. CONCLUSIONS: The results demonstrate a high potential of the proposed hierarchical deep-learning methodology for analyzing digital pathology images of pre-treatment tumor biopsies to predict the pathological response of breast cancer to NAC.

A Self-Attention-Guided 3D Deep Residual Network With Big Transfer to Predict Local Failure in Brain Metastasis After Radiotherapy Using Multi-Channel MRI.

A noticeable proportion of larger brain metastases (BMs) are not locally controlled after stereotactic radiotherapy, and it may take months before local progression is apparent on standard follow-up imaging. This work proposes and investigates new explainable deep-learning models to predict the radiotherapy outcome for BM. A novel self-attention-guided 3D residual network is introduced for predicting the outcome of local failure (LF) after radiotherapy using the baseline treatment-planning MRI. The 3D self-attention modules facilitate capturing long-range intra/inter slice dependencies which are often overlooked by convolution layers. The proposed model was compared to a vanilla 3D residual network and 3D residual network with CBAM attention in terms of performance in outcome prediction. A training recipe was adapted for the outcome prediction models during pretraining and training the down-stream task based on the recently proposed big transfer principles. A novel 3D visualization module was coupled with the model to demonstrate the impact of various intra/peri-lesion regions on volumetric multi-channel MRI upon the network's prediction. The proposed self-attention-guided 3D residual network outperforms the vanilla residual network and the residual network with CBAM attention in accuracy, F1-score, and AUC. The visualization results show the importance of peri-lesional characteristics on treatment-planning MRI in predicting local outcome after radiotherapy. This study demonstrates the potential of self-attention-guided deep-learning features derived from volumetric MRI in radiotherapy outcome prediction for BM. The insights obtained via the developed visualization module for individual lesions can possibly be applied during radiotherapy planning to decrease the chance of LF.

Enhanced full-inversion-based ultrasound elastography for evaluating tumor response to neoadjuvant chemotherapy in patients with locally advanced breast cancer.

PURPOSE: An enhanced ultrasound elastography technique is proposed for early assessment of locally advanced breast cancer (LABC) response to neoadjuvant chemotherapy (NAC). METHODS: The proposed elastography technique inputs ultrasound radiofrequency data obtained through tissue quasi-static stimulation and adapts a strain refinement algorithm formulated based on fundamental principles of continuum mechanics, coupled with an iterative inverse finite element method to reconstruct the breast Young's modulus (E) images. The technique was explored for therapy response assessment using data acquired from 25 LABC patients before and at weeks 1, 2, and 4 after the NAC initiation (100 scans). The E ratio of tumor to the surrounding tissue was calculated at different scans and compared to the baseline for each patient. Patients' response to NAC was determined many months later using standard clinical and histopathological criteria. RESULTS: Reconstructed E ratio changes obtained as early as one week after the NAC onset demonstrate very good separation between the two cohorts of responders and non-responders to NAC. Statistically significant differences were observed in the E ratio changes between the two patient cohorts at weeks 1 to 4 after treatment (p-value < 0.001; statistical power greater than 97%). A significant difference in axial strain ratio changes was observed only at week 4 (p-value = 0.01; statistical power = 76%). No significant difference was observed in tumor size changes at weeks 1, 2 or 4. CONCLUSION: The proposed elastography technique demonstrates a high potential for chemotherapy response monitoring in LABC patients and superior performance compared to strain imaging.

Automatic Assessment of Stereotactic Radiation Therapy Outcome in Brain Metastasis Using Longitudinal Segmentation on Serial MRI.

The standard clinical approach to assess the radiotherapy outcome in brain metastasis is through monitoring the changes in tumour size on longitudinal MRI. This assessment requires contouring the tumour on many volumetric images acquired before and at several follow-up scans after the treatment that is routinely done manually by oncologists with a substantial burden on the clinical workflow. In this work, we introduce a novel system for automatic assessment of stereotactic radiation therapy (SRT) outcome in brain metastasis using standard serial MRI. At the heart of the proposed system is a deep learning-based segmentation framework to delineate tumours longitudinally on serial MRI with high precision. Longitudinal changes in tumour size are then analyzed automatically to assess the local response and detect possible adverse radiation effects (ARE) after SRT. The system was trained and optimized using the data acquired from 96 patients (130 tumours) and evaluated on an independent test set of 20 patients (22 tumours; 95 MRI scans). The comparison between automatic therapy outcome evaluation and manual assessments by expert oncologists demonstrates a good agreement with an accuracy, sensitivity, and specificity of 91%, 89%, and 92%, respectively, in detecting local control/failure and 91%, 100%, and 89% in detecting ARE on the independent test set. This study is a step forward towards automatic monitoring and evaluation of radiotherapy outcome in brain tumours that can streamline the radio-oncology workflow substantially.

Machine learning analysis of breast ultrasound to classify triple negative and HER2+ breast cancer subtypes.

OBJECTIVES: Early diagnosis of triple-negative (TN) and human epidermal growth factor receptor 2 positive (HER2+) breast cancer is important due to its increased risk of micrometastatic spread necessitating early treatment and for guiding targeted therapies. This study aimed to evaluate the diagnostic performance of machine learning (ML) classification of newly diagnosed breast masses into TN versus non-TN (NTN) and HER2+ versus HER2 negative (HER2-) breast cancer, using radiomic features extracted from grayscale ultrasound (US) b-mode images. MATERIALS AND METHODS: A retrospective chart review identified 88 female patients who underwent diagnostic breast US imaging, had confirmation of invasive malignancy on pathology and receptor status determined on immunohistochemistry available. The patients were classified as TN, NTN, HER2+ or HER2- for ground-truth labelling. For image analysis, breast masses were manually segmented by a breast radiologist. Radiomic features were extracted per image and used for predictive modelling. Supervised ML classifiers included: logistic regression, k-nearest neighbour, and Naïve Bayes. Classification performance measures were calculated on an independent (unseen) test set. The area under the receiver operating characteristic curve (AUC), sensitivity (%), and specificity (%) were reported for each classifier. RESULTS: The logistic regression classifier demonstrated the highest AUC: 0.824 (sensitivity: 81.8%, specificity: 74.2%) for the TN sub-group and 0.778 (sensitivity: 71.4%, specificity: 71.6%) for the HER2 sub-group. CONCLUSION: ML classifiers demonstrate high diagnostic accuracy in classifying TN versus NTN and HER2+ versus HER2- breast cancers using US images. Identification of more aggressive breast cancer subtypes early in the diagnostic process could help achieve better prognoses by prioritizing clinical referral and prompting adequate early treatment.

Predicting Patterns of Distant Metastasis in Breast Cancer Patients following Local Regional Therapy Using Machine Learning.

Up to 30% of breast cancer (BC) patients will develop distant metastases (DM), for which there is no cure. Here, statistical and machine learning (ML) models were developed to estimate the risk of site-specific DM following local-regional therapy. This retrospective study cohort included 175 patients diagnosed with invasive BC who later developed DM. Clinicopathological information was collected for analysis. Outcome variables were the first site of metastasis (brain, bone or visceral) and the time interval (months) to developing DM. Multivariate statistical analysis and ML-based multivariable gradient boosting machines identified factors associated with these outcomes. Machine learning models predicted the site of DM, demonstrating an area under the curve of 0.74, 0.75, and 0.73 for brain, bone and visceral sites, respectively. Overall, most patients (57%) developed bone metastases, with increased odds associated with estrogen receptor (ER) positivity. Human epidermal growth factor receptor-2 (HER2) positivity and non-anthracycline chemotherapy regimens were associated with a decreased risk of bone DM, while brain metastasis was associated with ER-negativity. Furthermore, non-anthracycline chemotherapy alone was a significant predictor of visceral metastasis. Here, clinicopathologic and treatment variables used in ML prediction models predict the first site of metastasis in BC. Further validation may guide focused patient-specific surveillance practices.

An Enhanced Method for Full-Inversion-Based Ultrasound Elastography of the Liver.

Similar to many other types of cancer, liver cancer is associated with biological changes that lead to tissue stiffening. An effective imaging technique that can be used for liver cancer detection through visualizing tissue stiffness is ultrasound elastography. In this paper, we show the effectiveness of an enhanced method of quasi-static ultrasound elastography for liver cancer assessment. The method utilizes initial estimates of axial and lateral displacement fields obtained using conventional time delay estimation (TDE) methods in conjunction with a recently proposed strain refinement algorithm to generate enhanced versions of the axial and lateral strain images. Another primary objective of this work is to investigate the sensitivity of the proposed method to the quality of these initial displacement estimates. The strain refinement algorithm is founded on the tissue mechanics principles of incompressibility and strain compatibility. Tissue strain images can serve as input for full-inversion-based elasticity image reconstruction algorithm. In this work, we use strain images generated by the proposed method with an iterative elasticity reconstruction algorithm. Ultrasound RF data collected from a tissue-mimicking phantom and in-vivo data of a liver cancer patient were used to evaluate the proposed method. Results show that while there is some sensitivity to the displacement field initial estimates, overall, the proposed method is robust to the quality of the initial estimates. Clinical Relevance- Improved elasticity images of the liver can aid in achieving more reliable diagnosis of liver cancer.

Data-Efficient Training of Pure Vision Transformers for the Task of Chest X-ray Abnormality Detection Using Knowledge Distillation.

It is generally believed that vision transformers (ViTs) require a huge amount of data to generalize well, which limits their adoption. The introduction of data-efficient algorithms such as data-efficient image transformers (DeiT) provided an opportunity to explore the application of ViTs in medical imaging, where data scarcity is a limiting factor. In this work, we investigated the possibility of using pure transformers for the task of chest x-ray abnormality detection on a small dataset. Our proposed framework is built on a DeiT structure benefiting from a teacher-student scheme for training, with a DenseNet with strong classification performance as the teacher and an adapted ViT as the student. The results show that the performance of transformers is on par with that of convolutional neural networks (CNNs). We achieved a test accuracy of 92.2% for the task of classifying chest x-ray images (normal/pneumonia/COVID-19) on a carefully selected dataset using pure transformers. The results show the capability of transformers to accompany or replace CNNs for achieving state-of-the-art in medical imaging applications. The code and models of this work are available at https://github.com/Ouantimb-Lab/DeiTCovid.

A Cascaded Deep Learning Framework for Segmentation of Nuclei in Digital Histology Images.

Accurate segmentation of nuclei is an essential step in analysis of digital histology images for diagnostic and prognostic applications. Despite recent advances in automated frameworks for nuclei segmentation, this task is still challenging. Specifically, detecting small nuclei in large-scale histology images and delineating the border of touching nuclei accurately is a complicated task even for advanced deep neural networks. In this study, a cascaded deep learning framework is proposed to segment nuclei accurately in digitized microscopy images of histology slides. A U-Net based model with customized pixel-wised weighted loss function is adapted in the proposed framework, followed by a U-Net based model with VGG16 backbone and a soft Dice loss function. The model was pretrained on the Post-NAT-BRCA public dataset before training and independent evaluation on the MoNuSeg dataset. The cascaded model could outperform the other state-of-the-art models with an AJI of 0.72 and a F1-score of 0.83 on the MoNuSeg test set.

Deep learning of quantitative ultrasound multi-parametric images at pre-treatment to predict breast cancer response to chemotherapy.

In this study, a novel deep learning-based methodology was investigated to predict breast cancer response to neo-adjuvant chemotherapy (NAC) using the quantitative ultrasound (QUS) multi-parametric imaging at pre-treatment. QUS multi-parametric images of breast tumors were generated using the data acquired from 181 patients diagnosed with locally advanced breast cancer and planned for NAC followed by surgery. The ground truth response to NAC was identified for each patient after the surgery using the standard clinical and pathological criteria. Two deep convolutional neural network (DCNN) architectures including the residual network and residual attention network (RAN) were explored for extracting optimal feature maps from the parametric images, with a fully connected network for response prediction. In different experiments, the features maps were derived from the tumor core only, as well as the core and its margin. Evaluation results on an independent test set demonstrate that the developed model with the RAN architecture to extract feature maps from the expanded parametric images of the tumor core and margin had the best performance in response prediction with an accuracy of 88% and an area under the receiver operating characteristic curve of 0.86. Ten-year survival analyses indicate statistically significant differences between the survival of the responders and non-responders identified based on the model prediction at pre-treatment and the standard criteria at post-treatment. The results of this study demonstrate the promising capability of DCNNs with attention mechanisms in predicting breast cancer response to NAC prior to the start of treatment using QUS multi-parametric images.

Impact of Tumour Segmentation Accuracy on Efficacy of Quantitative MRI Biomarkers of Radiotherapy Outcome in Brain Metastasis.

Significantly affecting patients' clinical course and quality of life, a growing number of cancer cases are diagnosed with brain metastasis (BM) annually. Stereotactic radiotherapy is now a major treatment option for patients with BM. However, it may take months before the local response of BM to stereotactic radiation treatment is apparent on standard follow-up imaging. While machine learning in conjunction with radiomics has shown great promise in predicting the local response of BM before or early after radiotherapy, further development and widespread application of such techniques has been hindered by their dependency on manual tumour delineation. In this study, we explored the impact of using less-accurate automatically generated segmentation masks on the efficacy of radiomic features for radiotherapy outcome prediction in BM. The findings of this study demonstrate that while the effect of tumour delineation accuracy is substantial for segmentation models with lower dice scores (dice score ≤ 0.85), radiomic features and prediction models are rather resilient to imperfections in the produced tumour masks. Specifically, the selected radiomic features (six shared features out of seven) and performance of the prediction model (accuracy of 80% versus 80%, AUC of 0.81 versus 0.78) were fairly similar for the ground-truth and automatically generated segmentation masks, with dice scores close to 0.90. The positive outcome of this work paves the way for adopting high-throughput automatically generated tumour masks for discovering diagnostic and prognostic imaging biomarkers in BM without sacrificing accuracy.

Predicting the outcome of radiotherapy in brain metastasis by integrating the clinical and MRI-based deep learning features.

BACKGROUND: A considerable proportion of metastatic brain tumors progress locally despite stereotactic radiation treatment, and it can take months before such local progression is evident on follow-up imaging. Prediction of radiotherapy outcome in terms of tumor local failure is crucial for these patients and can facilitate treatment adjustments or allow for early salvage therapies. PURPOSE: In this work, a novel deep learning architecture is introduced to predict the outcome of local control/failure in brain metastasis treated with stereotactic radiation therapy using treatment-planning magnetic resonance imaging (MRI) and standard clinical attributes. METHODS: At the core of the proposed architecture is an InceptionResentV2 network to extract distinct features from each MRI slice for local outcome prediction. A recurrent or transformer network is integrated into the architecture to incorporate spatial dependencies between MRI slices into the predictive modeling. A visualization method based on prediction difference analysis is coupled with the deep learning model to illustrate how different regions of each lesion on MRI contribute to the model's prediction. The model was trained and optimized using the data acquired from 99 patients (116 lesions) and evaluated on an independent test set of 25 patients (40 lesions). RESULTS: The results demonstrate the promising potential of the MRI deep learning features for outcome prediction, outperforming standard clinical variables. The prediction model with only clinical variables demonstrated an area under the receiver operating characteristic curve (AUC) of 0.68. The MRI deep learning models resulted in AUCs in the range of 0.72 to 0.83 depending on the mechanism to integrate information from MRI slices of each lesion. The best prediction performance (AUC = 0.86) was associated with the model that combined the MRI deep learning features with clinical variables and incorporated the inter-slice dependencies using a long short-term memory recurrent network. The visualization results highlighted the importance of tumor/lesion margins in local outcome prediction for brain metastasis. CONCLUSIONS: The promising results of this study show the possibility of early prediction of radiotherapy outcome for brain metastasis via deep learning of MRI and clinical attributes at pre-treatment and encourage future studies on larger groups of patients treated with other radiotherapy modalities.

Quantitative digital histopathology and machine learning to predict pathological complete response to chemotherapy in breast cancer patients using pre-treatment tumor biopsies.

Complete pathological response (pCR) to neoadjuvant chemotherapy (NAC) is a prognostic factor for breast cancer (BC) patients and is correlated with improved survival. However, pCR rates are variable to standard NAC, depending on BC subtype. This study investigates quantitative digital histopathology coupled with machine learning (ML) to predict NAC response a priori. Clinicopathologic data and digitized slides of BC core needle biopsies were collected from 149 patients treated with NAC. The nuclei within the tumor regions were segmented on the histology images of biopsy samples using a weighted U-Net model. Five pathomic feature subsets were extracted from segmented digitized samples, including the morphological, intensity-based, texture, graph-based and wavelet features. Seven ML experiments were conducted with different feature sets to develop a prediction model of therapy response using a gradient boosting machine with decision trees. The models were trained and optimized using a five-fold cross validation on the training data and evaluated using an unseen independent test set. The prediction model developed with the best clinical features (tumor size, tumor grade, age, and ER, PR, HER2 status) demonstrated an area under the ROC curve (AUC) of 0.73. Various pathomic feature subsets resulted in models with AUCs in the range of 0.67 and 0.87, with the best results associated with the graph-based and wavelet features. The selected features among all subsets of the pathomic and clinicopathologic features included four wavelet and three graph-based features and no clinical features. The predictive model developed with these features outperformed the other models, with an AUC of 0.90, a sensitivity of 85% and a specificity of 82% on the independent test set. The results demonstrated the potential of quantitative digital histopathology features integrated with ML methods in predicting BC response to NAC. This study is a step forward towards precision oncology for BC patients to potentially guide future therapies.

Characterizing intra-tumor regions on quantitative ultrasound parametric images to predict breast cancer response to chemotherapy at pre-treatment.

The efficacy of quantitative ultrasound (QUS) multi-parametric imaging in conjunction with unsupervised classification algorithms was investigated for the first time in characterizing intra-tumor regions to predict breast tumor response to chemotherapy before the start of treatment. QUS multi-parametric images of breast tumors were generated using the ultrasound radiofrequency data acquired from 181 patients diagnosed with locally advanced breast cancer and planned for neo-adjuvant chemotherapy followed by surgery. A hidden Markov random field (HMRF) expectation maximization (EM) algorithm was applied to identify distinct intra-tumor regions on QUS multi-parametric images. Several features were extracted from the segmented intra-tumor regions and tumor margin on different parametric images. A multi-step feature selection procedure was applied to construct a QUS biomarker consisting of four features for response prediction. Evaluation results on an independent test set indicated that the developed biomarker coupled with a decision tree model with adaptive boosting (AdaBoost) as the classifier could predict the treatment response of patient at pre-treatment with an accuracy of 85.4% and an area under the receiver operating characteristic (ROC) curve (AUC) of 0.89. In comparison, the biomarkers consisted of the features derived from the entire tumor core (without consideration of the intra-tumor regions), and the entire tumor core and the tumor margin could predict the treatment response of patients with an accuracy of 74.5% and 76.4%, and an AUC of 0.79 and 0.76, respectively. Standard clinical features could predict the therapy response with an accuracy of 69.1% and an AUC of 0.6. Long-term survival analyses indicated that the patients predicted by the developed model as responders had a significantly better survival compared to the non-responders. Similar findings were observed for the two response cohorts identified at post-treatment based on standard clinical and pathological criteria. The results obtained in this study demonstrated the potential of QUS multi-parametric imaging integrated with unsupervised learning methods in identifying distinct intra-tumor regions in breast cancer to characterize its responsiveness to chemotherapy prior to the start of treatment.

A priori prediction of local failure in brain metastasis after hypo-fractionated stereotactic radiotherapy using quantitative MRI and machine learning.

This study investigated the effectiveness of pre-treatment quantitative MRI and clinical features along with machine learning techniques to predict local failure in patients with brain metastasis treated with hypo-fractionated stereotactic radiation therapy (SRT). The predictive models were developed using the data from 100 patients (141 lesions) and evaluated on an independent test set with data from 20 patients (30 lesions). Quantitative MRI radiomic features were derived from the treatment-planning contrast-enhanced T1w and T2-FLAIR images. A multi-phase feature reduction and selection procedure was applied to construct an optimal quantitative MRI biomarker for predicting therapy outcome. The performance of standard clinical features in therapy outcome prediction was evaluated using a similar procedure. Survival analyses were conducted to compare the long-term outcome of the two patient cohorts (local control/failure) identified based on prediction at pre-treatment, and standard clinical criteria at last patient follow-up after SRT. The developed quantitative MRI biomarker consists of four features with two features quantifying heterogeneity in the edema region, one feature characterizing intra-tumour heterogeneity, and one feature describing tumour morphology. The predictive models with the radiomic and clinical feature sets yielded an AUC of 0.87 and 0.62, respectively on the independent test set. Incorporating radiomic features into the clinical predictive model improved the AUC of the model by up to 16%, relatively. A statistically significant difference was observed in survival of the two patient cohorts identified at pre-treatment using the radiomics-based predictive model, and at post-treatment using the the RANO-BM criteria. Results of this study revealed a good potential for quantitative MRI radiomic features at pre-treatment in predicting local failure in relatively large brain metastases undergoing SRT, and is a step forward towards a precision oncology paradigm for brain metastasis.