Detecting microsatellite instability in colorectal cancer using Transformer-based colonoscopy image classification and retrieval.

Colorectal cancer (CRC) is a major global health concern, with microsatellite instability-high (MSI-H) being a defining characteristic of hereditary nonpolyposis colorectal cancer syndrome and affecting 15% of sporadic CRCs. Tumors with MSI-H have unique features and better prognosis compared to MSI-L and microsatellite stable (MSS) tumors. This study proposed establishing a MSI prediction model using more available and low-cost colonoscopy images instead of histopathology. The experiment utilized a database of 427 MSI-H and 1590 MSS colonoscopy images and vision Transformer (ViT) with different feature training approaches to establish the MSI prediction model. The accuracy of combining pre-trained ViT features was 84% with an area under the receiver operating characteristic curve of 0.86, which was better than that of DenseNet201 (80%, 0.80) in the experiment with support vector machine. The content-based image retrieval (CBIR) approach showed that ViT features can obtain a mean average precision of 0.81 compared to 0.79 of DenseNet201. ViT reduced the issues that occur in convolutional neural networks, including limited receptive field and gradient disappearance, and may be better at interpreting diagnostic information around tumors and surrounding tissues. By using CBIR, the presentation of similar images with the same MSI status would provide more convincing deep learning suggestions for clinical use.

Modeling the survival of colorectal cancer patients based on colonoscopic features in a feature ensemble vision transformer.

The prognosis of patients with colorectal cancer (CRC) mostly relies on the classic tumor node metastasis (TNM) staging classification. A more accurate and convenient prediction model would provide a better prognosis and assist in treatment. From May 2014 to December 2017, patients who underwent an operation for CRC were enrolled. The proposed feature ensemble vision transformer (FEViT) used ensemble classifiers to benefit the combinations of relevant colonoscopy features from the pretrained vision transformer and clinical features, including sex, age, family history of CRC, and tumor location, to establish the prognostic model. A total of 1729 colonoscopy images were enrolled in the current retrospective study. For the prediction of patient survival, FEViT achieved an accuracy of 94 % with an area under the receiver operating characteristic curve of 0.93, which was better than the TNM staging classification (90 %, 0.83) in the experiment. FEViT reduced the limited receptive field and gradient disappearance in the conventional convolutional neural network and was a relatively effective and efficient procedure. The promising accuracy of FEViT in modeling survival makes the prognosis of CRC patients more predictable and practical.

Rapid Polyp Classification in Colonoscopy Using Textural and Convolutional Features.

Colorectal cancer is the leading cause of cancer-associated morbidity and mortality worldwide. One of the causes of developing colorectal cancer is untreated colon adenomatous polyps. Clinically, polyps are detected in colonoscopy and the malignancies are determined according to the biopsy. To provide a quick and objective assessment to gastroenterologists, this study proposed a quantitative polyp classification via various image features in colonoscopy. The collected image database was composed of 1991 images including 1053 hyperplastic polyps and 938 adenomatous polyps and adenocarcinomas. From each image, textural features were extracted and combined in machine learning classifiers and machine-generated features were automatically selected in deep convolutional neural networks (DCNN). The DCNNs included AlexNet, Inception-V3, ResNet-101, and DenseNet-201. AlexNet trained from scratch achieved the best performance of 96.4% accuracy which is better than transfer learning and textural features. Using the prediction models, the malignancy level of polyps can be evaluated during a colonoscopy to provide a rapid treatment plan.

Deep Convolutional Neural Networks Detect Tumor Genotype from Pathological Tissue Images in Gastrointestinal Stromal Tumors.

Gastrointestinal stromal tumors (GIST) are common mesenchymal tumors, and their effective treatment depends upon the mutational subtype of the KIT/PDGFRA genes. We established deep convolutional neural network (DCNN) models to rapidly predict drug-sensitive mutation subtypes from images of pathological tissue. A total of 5153 pathological images of 365 different GISTs from three different laboratories were collected and divided into training and validation sets. A transfer learning mechanism based on DCNN was used with four different network architectures, to identify cases with drug-sensitive mutations. The accuracy ranged from 87% to 75%. Cross-institutional inconsistency, however, was observed. Using gray-scale images resulted in a 7% drop in accuracy (accuracy 80%, sensitivity 87%, specificity 73%). Using images containing only nuclei (accuracy 81%, sensitivity 87%, specificity 73%) or cytoplasm (accuracy 79%, sensitivity 88%, specificity 67%) produced 6% and 8% drops in accuracy rate, respectively, suggesting buffering effects across subcellular components in DCNN interpretation. The proposed DCNN model successfully inferred cases with drug-sensitive mutations with high accuracy. The contribution of image color and subcellular components was also revealed. These results will help to generate a cheaper and quicker screening method for tumor gene testing.

Computer-aided diagnosis of isocitrate dehydrogenase genotypes in glioblastomas from radiomic patterns.

World Health Organization tumor classifications of the central nervous system differentiate glioblastoma multiforme (GBM) into wild-type (WT) and mutant isocitrate dehydrogenase (IDH) genotypes. This study proposes a noninvasive computer-aided diagnosis to interpret the status of IDH in glioblastomas from transformed magnetic resonance imaging patterns. The collected image database was composed of 32 WT and 7 mutant IDH cases. For each image, a ranklet transformation which changed the original pixel values into relative coefficients was 1st applied to reduce the effects of different scanning parameters and machines on the underlying patterns. Extracting various textural features from the transformed ranklet images and combining them in a logistic regression classifier allowed an IDH prediction. We achieved an accuracy of 90%, a sensitivity of 57%, and a specificity of 97%. Four of the selected textural features in the classifier (homogeneity, difference entropy, information measure of correlation, and inverse difference normalized) were significant (P < .05), and the other 2 were close to being significant (P = .06). The proposed computer-aided diagnosis system based on radiomic textural features from ranklet-transformed images using relative rankings of pixel values as intensity-invariant coefficients is a promising noninvasive solution to provide recommendations about the IDH status in GBM across different healthcare institutions.

RNA editing-based classification of diffuse gliomas: predicting isocitrate dehydrogenase mutation and chromosome 1p/19q codeletion.

BACKGROUND: Accurate classification of diffuse gliomas, the most common tumors of the central nervous system in adults, is important for appropriate treatment. However, detection of isocitrate dehydrogenase (IDH) mutation and chromosome1p/19q codeletion, biomarkers to classify gliomas, is time- and cost-intensive and diagnostic discordance remains an issue. Adenosine to inosine (A-to-I) RNA editing has emerged as a novel cancer prognostic marker, but its value for glioma classification remains largely unexplored. We aim to (1) unravel the relationship between RNA editing and IDH mutation and 1p/19q codeletion and (2) predict IDH mutation and 1p/19q codeletion status using machine learning algorithms. RESULTS: By characterizing genome-wide A-to-I RNA editing signatures of 638 gliomas, we found that tumors without IDH mutation exhibited higher total editing level compared with those carrying it (Kolmogorov-Smirnov test, p < 0.0001). When tumor grade was considered, however, only grade IV tumors without IDH mutation exhibited higher total editing level. According to 10-fold cross-validation, support vector machines (SVM) outperformed random forest and AdaBoost (DeLong test, p < 0.05). The area under the receiver operating characteristic curve (AUC) of SVM in predicting IDH mutation and 1p/19q codeletion were 0.989 and 0.990, respectively. After performing feature selection, AUCs of SVM and AdaBoost in predicting IDH mutation were higher than that of random forest (0.985 and 0.983 vs. 0.977; DeLong test, p < 0.05), but AUCs of the three algorithms in predicting 1p/19q codeletion were similar (0.976-0.982). Furthermore, 67% of the six continuously misclassified samples by our 1p/19q codeletion prediction models were misclassifications in the original labelling after inspection of 1p/19q status and/or pathology report, highlighting the accuracy and clinical utility of our models. CONCLUSIONS: The study represents the first genome-wide analysis of glioma editome and identifies RNA editing as a novel prognostic biomarker for glioma. Our prediction models provide standardized, accurate, reproducible and objective classification of gliomas. Our models are not only useful in clinical decision-making, but also able to identify editing events that have the potential to serve as biomarkers and therapeutic targets in glioma management and treatment.

A machine learning texture model for classifying lung cancer subtypes using preliminary bronchoscopic findings.

PURPOSE: Bronchoscopy is useful in lung cancer detection, but cannot be used to differentiate cancer types. A computer-aided diagnosis (CAD) system was proposed to distinguish malignant cancer types to achieve objective diagnoses. METHODS: Bronchoscopic images of 12 adenocarcinoma and 10 squamous cell carcinoma patients were collected. The images were transformed from a red-blue-green (RGB) to a hue-saturation-value (HSV) color space to obtain more meaningful color textures. By combining significant textural features (P < 0.05) in a machine learning classifier, a prediction model of malignant types was established. RESULTS: The performance of the CAD system achieved an accuracy of 86% (19/22), a sensitivity of 90% (9/10), a specificity of 83% (10/12), a positive predictive value of 82% (9/11), and a negative predictive value of 91% (10/11) in distinguishing lung cancer types. The area under the receiver operating characteristic curve was 0.82. CONCLUSIONS: On the basis of extracted HSV textures of bronchoscopic images, the CAD system can provide recommendations for clinical diagnoses of lung cancer types.

Classification of lung cancer subtypes based on autofluorescence bronchoscopic pattern recognition: A preliminary study.

BACKGROUND AND OBJECTIVES: Lung cancer is the leading cause of cancer deaths worldwide. With current use of autofluorescent bronchoscopic imaging to detect early lung cancer and limitations of pathologic examinations, a computer-aided diagnosis (CAD) system based on autofluorescent bronchoscopy was proposed to distinguish different pathological cancer types to achieve objective and consistent diagnoses. METHODS: The collected database consisted of 12 adenocarcinomas and 11 squamous cell carcinomas. The corresponding autofluorescent bronchoscopic images were first transformed to a hue (H), saturation (S), and value (V) color space to obtain better interpretation of the color information. Color textural features were respectively extracted from the H, S, and V channels and combined in a logistic regression classifier to classify malignant types by machine learning. RESULTS: After feature selection, the proposed CAD system achieved an accuracy of 83% (19/23), a sensitivity of 73% (8/11), a specificity of 92% (11/12), a positive predictive value of 89% (8/9), a negative predictive value of 79% (11/14), and an area under the receiver operating characteristic curve of 0.81 for distinguishing lung cancer types. CONCLUSIONS: The proposed CAD system based on color textures of autofluorescent bronchoscopic images provides a diagnostic method of malignant types in clinical use.

Quantitative breast density analysis using tomosynthesis and comparison with MRI and digital mammography.

BACKGROUND AND OBJECTIVE: Breast density at mammography has been used as markers of breast cancer risk. However, newly introduced tomosynthesis and computer-aided quantitative method could provide more reliable breast density evaluation. METHODS: In the experiment, 98 tomosynthesis image volumes were obtained from 98 women. For each case, an automatic skin removal was used and followed by a fuzzy c-mean (FCM) classifier which separated the fibroglandular tissues from other tissues in breast area. Finally, percent of breast density and breast volume were calculated and the results were compared with MRI. In addition, the percent of breast density and breast area of digital mammography calculated using the software Cumulus (University of Toronto, Toronto, ON, Canada.) were also compared with 3-D modalities. RESULTS: Percent of breast density and breast volume, which were computed from tomosynthesis, MRI and digital mammography were 17.37% ±â€¯4.39% and 607.12 cm3 ±â€¯323.01 cm3, 20.3% ±â€¯8.6% and 537.59 cm3 ±â€¯287.74 cm3, and 12.03% ±â€¯4.08%, respectively. There were significant correlations on breast density as well as volume between tomosynthesis and MRI (R = 0.482 and R = 0.805), tomosynthesis and breast density with breast area of digital mammography (R = 0.789 and R = 0.877), and MRI and breast density with breast area of digital mammography (R = 0.482 and R = 0.857) (all P values < .001). CONCLUSIONS: Breast density and breast volume evaluated from tomosynthesis, MRI and breast density and breast area of digital mammographic images have significant correlations and indicate that tomosynthesis could provide useful 3-D information on breast density through proposed method.

Computer-aided diagnosis of liver tumors on computed tomography images.

BACKGROUND AND OBJECTIVE: Liver cancer is the tenth most common cancer in the USA, and its incidence has been increasing for several decades. Early detection, diagnosis, and treatment of the disease are very important. Computed tomography (CT) is one of the most common and robust imaging techniques for the detection of liver cancer. CT scanners can provide multiple-phase sequential scans of the whole liver. In this study, we proposed a computer-aided diagnosis (CAD) system to diagnose liver cancer using the features of tumors obtained from multiphase CT images. METHODS: A total of 71 histologically-proven liver tumors including 49 benign and 22 malignant lesions were evaluated with the proposed CAD system to evaluate its performance. Tumors were identified by the user and then segmented using a region growing algorithm. After tumor segmentation, three kinds of features were obtained for each tumor, including texture, shape, and kinetic curve. The texture was quantified using 3 dimensional (3-D) texture data of the tumor based on the grey level co-occurrence matrix (GLCM). Compactness, margin, and an elliptic model were used to describe the 3-D shape of the tumor. The kinetic curve was established from each phase of tumor and represented as variations in density between each phase. Backward elimination was used to select the best combination of features, and binary logistic regression analysis was used to classify the tumors with leave-one-out cross validation. RESULTS: The accuracy and sensitivity for the texture were 71.82% and 68.18%, respectively, which were better than for the shape and kinetic curve under closed specificity. Combining all of the features achieved the highest accuracy (58/71, 81.69%), sensitivity (18/22, 81.82%), and specificity (40/49, 81.63%). The Az value of combining all features was 0.8713. CONCLUSIONS: Combining texture, shape, and kinetic curve features may be able to differentiate benign from malignant tumors in the liver using our proposed CAD system.

Radiomic model for predicting mutations in the isocitrate dehydrogenase gene in glioblastomas.

The present study proposed a computer-aided diagnosis system based on radiomic features extracted through magnetic resonance imaging to determine the isocitrate dehydrogenase status in glioblastomas. Magnetic resonance imaging data were obtained from 32 patients with wild-typeisocitrate dehydrogenase and 7 patients with mutant isocitrate dehydrogenase in glioblastomas. Radiomic features, namely morphological, intensity, and textural features, were extracted from the tumor area of each patient. The feature sets were evaluated using a logistic regression classifier to develop a prediction model. The accuracy of the global morphological and intensity features was 51% (20/39) and 59% (23/39), respectively. The textural features describing local patterns yielded an accuracy of 85% (33/39), which is significantly higher than that yielded by the morphological and intensity features. The agreement level (κ) between the prediction results and biopsy-proven pathology was 0.60. The proposed diagnosis system based on radiomic textural features shows promise for application in providing suggestions to radiologists for distinguishing isocitrate dehydrogenase mutations in glioblastomas.

Whole-Breast Ultrasound for Breast Screening and Archiving.

The incidence of breast cancer is increasing worldwide, reinforcing the importance of breast screening. Conventional hand-held ultrasound (HHUS) for breast screening is efficient and relatively easy to perform; however, it lacks systematic recording and localization. This study investigated an electromagnetic tracking-based whole-breast ultrasound (WBUS) system to facilitate the use of HHUS for breast screening. One-hundred nine breast masses were collected, and the detection of suspicious breast lesions was compared between the WBUS system, HHUS and a commercial automated breast ultrasound (ABUS) system. The positioning error between WBUS and ABUS (1.39 ± 0.68 cm) was significantly smaller than that between HHUS and ABUS (1.62 ± 0.91 cm, p = 0.014) and HHUS and WBUS (1.63 ± 0.9 cm, p = 0.024). WBUS is a practical clinical tool for breast screening that can be used instead of the often unavailable and costly ABUS.

Quantitative glioma grading using transformed gray-scale invariant textures of MRI.

BACKGROUND: A computer-aided diagnosis (CAD) system based on intensity-invariant magnetic resonance (MR) imaging features was proposed to grade gliomas for general application to various scanning systems and settings. METHOD: In total, 34 glioblastomas and 73 lower-grade gliomas comprised the image database to evaluate the proposed CAD system. For each case, the local texture on MR images was transformed into a local binary pattern (LBP) which was intensity-invariant. From the LBP, quantitative image features, including the histogram moment and textures, were extracted and combined in a logistic regression classifier to establish a malignancy prediction model. The performance was compared to conventional texture features to demonstrate the improvement. RESULTS: The performance of the CAD system based on LBP features achieved an accuracy of 93% (100/107), a sensitivity of 97% (33/34), a negative predictive value of 99% (67/68), and an area under the receiver operating characteristic curve (Az) of 0.94, which were significantly better than the conventional texture features: an accuracy of 84% (90/107), a sensitivity of 76% (26/34), a negative predictive value of 89% (64/72), and an Az of 0.89 with respective p values of 0.0303, 0.0122, 0.0201, and 0.0334. CONCLUSIONS: More-robust texture features were extracted from MR images and combined into a significantly better CAD system for distinguishing glioblastomas from lower-grade gliomas. The proposed CAD system would be more practical in clinical use with various imaging systems and settings.

Computer-aided tumor diagnosis using shear wave breast elastography.

The shear wave elastography (SWE) uses the acoustic radiation force to measure the stiffness of tissues and is less operator dependent in data acquisition compared to strain elastography. However, the reproducibility of the result is still interpreter dependent. The purpose of this study is to develop a computer-aided diagnosis (CAD) method to differentiate benign from malignant breast tumors using SWE images. After applying the level set method to automatically segment the tumor contour and hue-saturation-value color transformation, SWE features including average tissue elasticity, sectional stiffness ratio, and normalized minimum distance for grouped stiffer pixels are calculated. Finally, the performance of CAD based on SWE features are compared with those based on B-mode ultrasound (morphologic and textural) features, and a combination of both feature sets to differentiate benign from malignant tumors. In this study, we use 109 biopsy-proved breast tumors composed of 57 benign and 52 malignant cases. The experimental results show that the sensitivity, specificity, accuracy and the area under the receiver operating characteristic ROC curve (Az value) of CAD are 86.5%, 93.0%, 89.9%, and 0.905 for SWE features whereas they are 86.5%, 80.7%, 83.5% and 0.893 for B-mode features and 90.4%, 94.7%, 92.3% and 0.961 for the combined features. The Az value of combined feature set is significantly higher compared to the B-mode and SWE feature sets (p=0.0296 and p=0.0204, respectively). Our results suggest that the CAD based on SWE features has the potential to improve the performance of classifying breast tumors with US.

Effect of a computer-aided diagnosis system on radiologists' performance in grading gliomas with MRI.

The effects of a computer-aided diagnosis (CAD) system based on quantitative intensity features with magnetic resonance (MR) imaging (MRI) were evaluated by examining radiologists' performance in grading gliomas. The acquired MRI database included 71 lower-grade gliomas and 34 glioblastomas. Quantitative image features were extracted from the tumor area and combined in a CAD system to generate a prediction model. The effect of the CAD system was evaluated in a two-stage procedure. First, a radiologist performed a conventional reading. A sequential second reading was determined with a malignancy estimation by the CAD system. Each MR image was regularly read by one radiologist out of a group of three radiologists. The CAD system achieved an accuracy of 87% (91/105), a sensitivity of 79% (27/34), a specificity of 90% (64/71), and an area under the receiver operating characteristic curve (Az) of 0.89. In the evaluation, the radiologists' Az values significantly improved from 0.81, 0.87, and 0.84 to 0.90, 0.90, and 0.88 with p = 0.0011, 0.0076, and 0.0167, respectively. Based on the MR image features, the proposed CAD system not only performed well in distinguishing glioblastomas from lower-grade gliomas but also provided suggestions about glioma grading to reinforce radiologists' confidence rating.

Computer-aided grading of gliomas based on local and global MRI features.

BACKGROUND AND OBJECTIVES: A computer-aided diagnosis (CAD) system based on quantitative magnetic resonance imaging (MRI) features was developed to evaluate the malignancy of diffuse gliomas, which are central nervous system tumors. METHODS: The acquired image database for the CAD performance evaluation was composed of 34 glioblastomas and 73 diffuse lower-grade gliomas. In each case, tissues enclosed in a delineated tumor area were analyzed according to their gray-scale intensities on MRI scans. Four histogram moment features describing the global gray-scale distributions of gliomas tissues and 14 textural features were used to interpret local correlations between adjacent pixel values. With a logistic regression model, the individual feature set and a combination of both feature sets were used to establish the malignancy prediction model. RESULTS: Performances of the CAD system using global, local, and the combination of both image feature sets achieved accuracies of 76%, 83%, and 88%, respectively. Compared to global features, the combined features had significantly better accuracy (p = 0.0213). With respect to the pathology results, the CAD classification obtained substantial agreement κ = 0.698, p < 0.001. CONCLUSIONS: Numerous proposed image features were significant in distinguishing glioblastomas from lower-grade gliomas. Combining them further into a malignancy prediction model would be promising in providing diagnostic suggestions for clinical use.