A Comparison of Computer-Aided Diagnosis Schemes Optimized Using Radiomics and Deep Transfer Learning Methods.

Objective: Radiomics and deep transfer learning are two popular technologies used to develop computer-aided detection and diagnosis (CAD) schemes of medical images. This study aims to investigate and to compare the advantages and the potential limitations of applying these two technologies in developing CAD schemes. Methods: A relatively large and diverse retrospective dataset including 3000 digital mammograms was assembled in which 1496 images depicted malignant lesions and 1504 images depicted benign lesions. Two CAD schemes were developed to classify breast lesions. The first scheme was developed using four steps namely, applying an adaptive multi-layer topographic region growing algorithm to segment lesions, computing initial radiomics features, applying a principal component algorithm to generate an optimal feature vector, and building a support vector machine classifier. The second CAD scheme was built based on a pre-trained residual net architecture (ResNet50) as a transfer learning model to classify breast lesions. Both CAD schemes were trained and tested using a 10-fold cross-validation method. Several score fusion methods were also investigated to classify breast lesions. CAD performances were evaluated and compared by the areas under the ROC curve (AUC). Results: The ResNet50 model-based CAD scheme yielded AUC = 0.85 ± 0.02, which was significantly higher than the radiomics feature-based CAD scheme with AUC = 0.77 ± 0.02 (p < 0.01). Additionally, the fusion of classification scores generated by the two CAD schemes did not further improve classification performance. Conclusion: This study demonstrates that using deep transfer learning is more efficient to develop CAD schemes and it enables a higher lesion classification performance than CAD schemes developed using radiomics-based technology.

Developing new quantitative CT image markers to predict prognosis of acute ischemic stroke patients.

BACKGROUND: Endovascular mechanical thrombectomy (EMT) is an effective method to treat acute ischemic stroke (AIS) patients due to large vessel occlusion (LVO). However, stratifying AIS patients who can and cannot benefit from EMT remains a clinical challenge. OBJECTIVE: To develop a new quantitative image marker computed from pre-intervention computed tomography perfusion (CTP) images and evaluate its feasibility to predict clinical outcome among AIS patients undergoing EMT after diagnosis of LVO. METHODS: A retrospective dataset of 31 AIS patients with pre-intervention CTP images is assembled. A computer-aided detection (CAD) scheme is developed to pre-process CTP images of different scanning series for each study case, perform image segmentation, quantify contrast-enhanced blood volumes in bilateral cerebral hemispheres, and compute features related to asymmetrical cerebral blood flow patterns based on the cumulative cerebral blood flow curves of two hemispheres. Next, image markers based on a single optimal feature and machine learning (ML) models fused with multi-features are developed and tested to classify AIS cases into two classes of good and poor prognosis based on the Modified Rankin Scale. Performance of image markers is evaluated using the area under the ROC curve (AUC) and accuracy computed from the confusion matrix. RESULTS: The ML model using the neuroimaging features computed from the slopes of the subtracted cumulative blood flow curves between two cerebral hemispheres yields classification performance of AUC = 0.878±0.077 with an overall accuracy of 90.3%. CONCLUSIONS: This study demonstrates feasibility of developing a new quantitative imaging method and marker to predict AIS patients' prognosis in the hyperacute stage, which can help clinicians optimally treat and manage AIS patients.

Applying Quantitative Radiographic Image Markers to Predict Clinical Complications After Aneurysmal Subarachnoid Hemorrhage: A Pilot Study.

Accurately predicting clinical outcome of aneurysmal subarachnoid hemorrhage (aSAH) patients is difficult. The purpose of this study was to develop and test a new fully-automated computer-aided detection (CAD) scheme of brain computed tomography (CT) images to predict prognosis of aSAH patients. A retrospective dataset of 59 aSAH patients was assembled. Each patient had 2 sets of CT images acquired at admission and prior-to-discharge. CAD scheme was applied to segment intracranial brain regions into four subregions, namely, cerebrospinal fluid (CSF), white matter (WM), gray matter (GM), and leaked extraparenchymal blood (EPB), respectively. CAD then detects sulci and computes 9 image features related to 5 volumes of the segmented sulci, EPB, CSF, WM, and GM and 4 volumetrical ratios to sulci. Subsequently, applying a leave-one-case-out cross-validation method embedded with a principal component analysis (PCA) algorithm to generate optimal feature vector, 16 support vector machine (SVM) models were built using CT images acquired either at admission or prior-to-discharge to predict each of eight clinically relevant parameters commonly used to assess patients' prognosis. Finally, a receiver operating characteristics (ROC) method was used to evaluate SVM model performance. Areas under ROC curves of 16 SVM models range from 0.62 ± 0.07 to 0.86 ± 0.07. In general, SVM models trained using CT images acquired at admission yielded higher accuracy to predict short-term clinical outcomes, while SVM models trained using CT images acquired prior-to-discharge demonstrated higher accuracy in predicting long-term clinical outcomes. This study demonstrates feasibility to predict prognosis of aSAH patients using new quantitative image markers generated by SVM models.

Applying a Random Projection Algorithm to Optimize Machine Learning Model for Breast Lesion Classification.

OBJECTIVE: Since computer-aided diagnosis (CAD) schemes of medical images usually computes large number of image features, which creates a challenge of how to identify a small and optimal feature vector to build robust machine learning models, the objective of this study is to investigate feasibility of applying a random projection algorithm (RPA) to build an optimal feature vector from the initially CAD-generated large feature pool and improve performance of machine learning model. METHODS: We assemble a retrospective dataset involving 1,487 cases of mammograms in which 644 cases have confirmed malignant mass lesions and 843 have benign lesions. A CAD scheme is first applied to segment mass regions and initially compute 181 features. Then, support vector machine (SVM) models embedded with several feature dimensionality reduction methods are built to predict likelihood of lesions being malignant. All SVM models are trained and tested using a leave-one-case-out cross-validation method. SVM generates a likelihood score of each segmented mass region depicting on one-view mammogram. By fusion of two scores of the same mass depicting on two-view mammograms, a case-based likelihood score is also evaluated. RESULTS: Comparing with the principle component analyses, nonnegative matrix factorization, and Chi-squared methods, SVM embedded with RPA yielded a significantly higher case-based lesion classification performance with the area under ROC curve of 0.84 ± 0.01 (p<0.02). CONCLUSION: The study demonstrates that RPA is a promising method to generate optimal feature vectors and improve SVM performance. SIGNIFICANCE: This study presents a new method to develop CAD schemes with significantly higher and robust performance.

Applying a random projection algorithm to optimize machine learning model for predicting peritoneal metastasis in gastric cancer patients using CT images.

BACKGROUND AND OBJECTIVE: Non-invasively predicting the risk of cancer metastasis before surgery can play an essential role in determining which patients can benefit from neoadjuvant chemotherapy. This study aims to investigate and test the advantages of applying a random projection algorithm to develop and optimize a radiomics-based machine learning model to predict peritoneal metastasis in gastric cancer patients using a small and imbalanced computed tomography (CT) image dataset. METHODS: A retrospective dataset involving CT images acquired from 159 patients is assembled, including 121 and 38 cases with and without peritoneal metastasis, respectively. A computer-aided detection scheme is first applied to segment primary gastric tumor volumes and initially compute 315 image features. Then, five gradients boosting machine (GBM) models embedded with five feature selection methods (including random projection algorithm, principal component analysis, least absolute shrinkage, and selection operator, maximum relevance and minimum redundancy, and recursive feature elimination) along with a synthetic minority oversampling technique, are built to predict the risk of peritoneal metastasis. All GBM models are trained and tested using a leave-one-case-out cross-validation method. RESULTS: Results show that the GBM model embedded with a random projection algorithm yields a significantly higher prediction accuracy (71.2%) than the other four GBM models (p<0.05). The precision, sensitivity, and specificity of this optimal GBM model are 65.78%, 43.10%, and 87.12%, respectively. CONCLUSIONS: This study demonstrates that CT images of the primary gastric tumors contain discriminatory information to predict the risk of peritoneal metastasis, and a random projection algorithm is a promising method to generate optimal feature vector, improving the performance of machine learning based prediction models.

Improving the performance of CNN to predict the likelihood of COVID-19 using chest X-ray images with preprocessing algorithms.

OBJECTIVE: This study aims to develop and test a new computer-aided diagnosis (CAD) scheme of chest X-ray images to detect coronavirus (COVID-19) infected pneumonia. METHOD: CAD scheme first applies two image preprocessing steps to remove the majority of diaphragm regions, process the original image using a histogram equalization algorithm, and a bilateral low-pass filter. Then, the original image and two filtered images are used to form a pseudo color image. This image is fed into three input channels of a transfer learning-based convolutional neural network (CNN) model to classify chest X-ray images into 3 classes of COVID-19 infected pneumonia, other community-acquired no-COVID-19 infected pneumonia, and normal (non-pneumonia) cases. To build and test the CNN model, a publicly available dataset involving 8474 chest X-ray images is used, which includes 415, 5179 and 2,880 cases in three classes, respectively. Dataset is randomly divided into 3 subsets namely, training, validation, and testing with respect to the same frequency of cases in each class to train and test the CNN model. RESULTS: The CNN-based CAD scheme yields an overall accuracy of 94.5 % (2404/2544) with a 95 % confidence interval of [0.93,0.96] in classifying 3 classes. CAD also yields 98.4 % sensitivity (124/126) and 98.0 % specificity (2371/2418) in classifying cases with and without COVID-19 infection. However, without using two preprocessing steps, CAD yields a lower classification accuracy of 88.0 % (2239/2544). CONCLUSION: This study demonstrates that adding two image preprocessing steps and generating a pseudo color image plays an important role in developing a deep learning CAD scheme of chest X-ray images to improve accuracy in detecting COVID-19 infected pneumonia.

Predicting Clinical Outcome After Mechanical Thrombectomy: The GADIS (Gender, Age, Diabetes Mellitus History, Infarct Volume, and Current Smoker [corrected]) Score.

OBJECTIVE: To investigate predictive factors and develop an outcome assessment tool to determine clinical outcome after endovascular mechanical thrombectomy (EMT) in patients presenting with large vessel occlusion (LVO). METHODS: A retrospective analysis was carried out of a prospective cohort of patients presenting with LVO who underwent EMT after adoption of an expanded time window of ≤24 hours. Final cerebral infarction volume (CIV) after EMT was estimated using magnetic resonance imaging segmentation software. Stepwise linear regression models were used to identify factors that determined clinical outcome and to develop a predictive scale. RESULTS: Ninety patients underwent EMT over 19 months (68 within 6 hours and 22 between 6 and 24 hours). Clinical outcome determined using modified Rankin Scale (mRS) score at discharge and 3 months was no different among these subcohorts. A threshold of 16.99 mL of CIV, using the Youden index, resulted in a sensitivity of 90.5% and specificity of 58.1% for predicting mRS score of 0-2. A regression model identified gender, age, diabetes mellitus status, CIV, and smoking status as outcome determinants, which were used to develop the GADIS (Gender, Age, Diabetes Mellitus History, Infarct Volume, and Sex) scoring system to predict good clinical outcome. Using the GADIS score, <6 predicted mRS score 0-2 at discharge with a sensitivity of 83.3% and specificity of 80.6%. CONCLUSIONS: The GADIS score for patients with LVO-related acute ischemic stroke includes CIV after EMT and helps in early short-term prognostication. It is not intended to predict preintervention patient selection or outcome prediction.

IDRiD: Diabetic Retinopathy - Segmentation and Grading Challenge.

Diabetic Retinopathy (DR) is the most common cause of avoidable vision loss, predominantly affecting the working-age population across the globe. Screening for DR, coupled with timely consultation and treatment, is a globally trusted policy to avoid vision loss. However, implementation of DR screening programs is challenging due to the scarcity of medical professionals able to screen a growing global diabetic population at risk for DR. Computer-aided disease diagnosis in retinal image analysis could provide a sustainable approach for such large-scale screening effort. The recent scientific advances in computing capacity and machine learning approaches provide an avenue for biomedical scientists to reach this goal. Aiming to advance the state-of-the-art in automatic DR diagnosis, a grand challenge on "Diabetic Retinopathy - Segmentation and Grading" was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI - 2018). In this paper, we report the set-up and results of this challenge that is primarily based on Indian Diabetic Retinopathy Image Dataset (IDRiD). There were three principal sub-challenges: lesion segmentation, disease severity grading, and localization of retinal landmarks and segmentation. These multiple tasks in this challenge allow to test the generalizability of algorithms, and this is what makes it different from existing ones. It received a positive response from the scientific community with 148 submissions from 495 registrations effectively entered in this challenge. This paper outlines the challenge, its organization, the dataset used, evaluation methods and results of top-performing participating solutions. The top-performing approaches utilized a blend of clinical information, data augmentation, and an ensemble of models. These findings have the potential to enable new developments in retinal image analysis and image-based DR screening in particular.

Developing a Quantitative Ultrasound Image Feature Analysis Scheme to Assess Tumor Treatment Efficacy Using a Mouse Model.

The aim of this study is to investigate the feasibility of identifying and applying quantitative imaging features computed from ultrasound images of athymic nude mice to predict tumor response to treatment at an early stage. A computer-aided detection (CAD) scheme with a graphic user interface was developed to conduct tumor segmentation and image feature analysis. A dataset involving ultrasound images of 23 athymic nude mice bearing C26 mouse adenocarcinomas was assembled. These mice were divided into 7 treatment groups utilizing a combination of thermal and nanoparticle-controlled drug delivery. Longitudinal ultrasound images of mice were taken prior and post-treatment in day 3 and day 6. After tumor segmentation, CAD scheme computed image features and created four feature pools including features computed from (1) prior treatment images only and (2) difference between prior and post-treatment images of day 3 and day 6, respectively. To predict tumor treatment efficacy, data analysis was performed to identify top image features and an optimal feature fusion method, which have a higher correlation to tumor size increase ratio (TSIR) determined at Day 10. Using image features computed from day 3, the highest Pearson Correlation coefficients between the top two features selected from two feature pools versus TSIR were 0.373 and 0.552, respectively. Using an equally weighted fusion method of two features computed from prior and post-treatment images, the correlation coefficient increased to 0.679. Meanwhile, using image features computed from day 6, the highest correlation coefficient was 0.680. Study demonstrated the feasibility of extracting quantitative image features from the ultrasound images taken at an early treatment stage to predict tumor response to therapies.

Developing global image feature analysis models to predict cancer risk and prognosis.

In order to develop precision or personalized medicine, identifying new quantitative imaging markers and building machine learning models to predict cancer risk and prognosis has been attracting broad research interest recently. Most of these research approaches use the similar concepts of the conventional computer-aided detection schemes of medical images, which include steps in detecting and segmenting suspicious regions or tumors, followed by training machine learning models based on the fusion of multiple image features computed from the segmented regions or tumors. However, due to the heterogeneity and boundary fuzziness of the suspicious regions or tumors, segmenting subtle regions is often difficult and unreliable. Additionally, ignoring global and/or background parenchymal tissue characteristics may also be a limitation of the conventional approaches. In our recent studies, we investigated the feasibility of developing new computer-aided schemes implemented with the machine learning models that are trained by global image features to predict cancer risk and prognosis. We trained and tested several models using images obtained from full-field digital mammography, magnetic resonance imaging, and computed tomography of breast, lung, and ovarian cancers. Study results showed that many of these new models yielded higher performance than other approaches used in current clinical practice. Furthermore, the computed global image features also contain complementary information from the features computed from the segmented regions or tumors in predicting cancer prognosis. Therefore, the global image features can be used alone to develop new case-based prediction models or can be added to current tumor-based models to increase their discriminatory power.

Classification of Breast Masses Using a Computer-Aided Diagnosis Scheme of Contrast Enhanced Digital Mammograms.

Contrast-enhanced digital mammography (CEDM) is a promising imaging modality in breast cancer diagnosis. This study aims to investigate how to optimally develop a computer-aided diagnosis (CAD) scheme of CEDM images to classify breast masses. A CEDM dataset of 111 patients was assembled, which includes 33 benign and 78 malignant cases. Each CEDM includes two types of images namely, low energy (LE) and dual-energy subtracted (DES) images. A CAD scheme was applied to segment mass regions depicting on LE and DES images separately. Optimal segmentation results generated from DES images were also mapped to LE images or vice versa. After computing image features, multilayer perceptron based machine learning classifiers that integrate with a correlation-based feature subset evaluator and leave-one-case-out cross-validation method were built to classify mass regions. When applying CAD to DES and LE images with original segmentation, areas under ROC curves (AUC) were 0.759 ± 0.053 and 0.753 ± 0.047, respectively. After mapping the mass regions optimally segmented on DES images to LE images, AUC significantly increased to 0.848 ± 0.038 (p < 0.01). Study demonstrated that DES images eliminated overlapping effect of dense breast tissue, which helps improve mass segmentation accuracy. The study demonstrated that applying a novel approach to optimally map mass region segmented from DES images to LE images enabled CAD to yield significantly improved performance.

Prediction of breast cancer risk using a machine learning approach embedded with a locality preserving projection algorithm.

In order to automatically identify a set of effective mammographic image features and build an optimal breast cancer risk stratification model, this study aims to investigate advantages of applying a machine learning approach embedded with a locally preserving projection (LPP) based feature combination and regeneration algorithm to predict short-term breast cancer risk. A dataset involving negative mammograms acquired from 500 women was assembled. This dataset was divided into two age-matched classes of 250 high risk cases in which cancer was detected in the next subsequent mammography screening and 250 low risk cases, which remained negative. First, a computer-aided image processing scheme was applied to segment fibro-glandular tissue depicted on mammograms and initially compute 44 features related to the bilateral asymmetry of mammographic tissue density distribution between left and right breasts. Next, a multi-feature fusion based machine learning classifier was built to predict the risk of cancer detection in the next mammography screening. A leave-one-case-out (LOCO) cross-validation method was applied to train and test the machine learning classifier embedded with a LLP algorithm, which generated a new operational vector with 4 features using a maximal variance approach in each LOCO process. Results showed a 9.7% increase in risk prediction accuracy when using this LPP-embedded machine learning approach. An increased trend of adjusted odds ratios was also detected in which odds ratios increased from 1.0 to 11.2. This study demonstrated that applying the LPP algorithm effectively reduced feature dimensionality, and yielded higher and potentially more robust performance in predicting short-term breast cancer risk.

Applying Quantitative CT Image Feature Analysis to Predict Response of Ovarian Cancer Patients to Chemotherapy.

RATIONALE AND OBJECTIVES: The study aimed to investigate the role of applying quantitative image features computed from computed tomography (CT) images for early prediction of tumor response to chemotherapy in the clinical trials for treating ovarian cancer patients. MATERIALS AND METHODS: A dataset involving 91 patients was retrospectively assembled. Each patient had two sets of pre- and post-therapy CT images. A computer-aided detection scheme was applied to segment metastatic tumors previously tracked by radiologists on CT images and computed image features. Two initial feature pools were built using image features computed from pre-therapy CT images only and image feature difference computed from both pre- and post-therapy images. A feature selection method was applied to select optimal features, and an equal-weighted fusion method was used to generate a new quantitative imaging marker from each pool to predict 6-month progression-free survival. The prediction accuracy between quantitative imaging markers and the Response Evaluation Criteria in Solid Tumors (RECIST) criteria was also compared. RESULTS: The highest areas under the receiver operating characteristic curve are 0.684 ± 0.056 and 0.771 ± 0.050 when using a single image feature computed from pre-therapy CT images and feature difference computed from pre- and post-therapy CT images, respectively. Using two corresponding fusion-based image markers, the areas under the receiver operating characteristic curve significantly increased to 0.810 ± 0.045 and 0.829 ± 0.043 (P < 0.05), respectively. Overall prediction accuracy levels are 71.4%, 80.2%, and 74.7% when using two imaging markers and RECIST, respectively. CONCLUSIONS: This study demonstrated the feasibility of predicting patients' response to chemotherapy using quantitative imaging markers computed from pre-therapy CT images. However, using image feature difference computed between pre- and post-therapy CT images yielded higher prediction accuracy.