Artificial intelligence for the detection of esophageal and esophagogastric junctional adenocarcinoma.

BACKGROUND AND AIM: Conventional endoscopy for the early detection of esophageal and esophagogastric junctional adenocarcinoma (E/J cancer) is limited because early lesions are asymptomatic, and the associated changes in the mucosa are subtle. There are no reports on artificial intelligence (AI) diagnosis for E/J cancer from Asian countries. Therefore, we aimed to develop a computerized image analysis system using deep learning for the detection of E/J cancers. METHODS: A total of 1172 images from 166 pathologically proven superficial E/J cancer cases and 2271 images of normal mucosa in esophagogastric junctional from 219 cases were used as the training image data. A total of 232 images from 36 cancer cases and 43 non-cancerous cases were used as the validation test data. The same validation test data were diagnosed by 15 board-certified specialists (experts). RESULTS: The sensitivity, specificity, and accuracy of the AI system were 94%, 42%, and 66%, respectively, and that of the experts were 88%, 43%, and 63%, respectively. The sensitivity of the AI system was favorable, while its specificity for non-cancerous lesions was similar to that of the experts. Interobserver agreement among the experts for detecting superficial E/J was fair (Fleiss' kappa = 0.26, z = 20.4, P < 0.001). CONCLUSIONS: Our AI system achieved high sensitivity and acceptable specificity for the detection of E/J cancers and may be a good supporting tool for the screening of E/J cancers.

Stratification of gastric cancer risk using a deep neural network.

BACKGROUND AND AIM: Stratifying gastric cancer (GC) risk and endoscopy findings in high-risk individuals may provide effective surveillance for GC. We developed a computerized image- analysis system for endoscopic images to stratify the risk of GC. METHODS: The system was trained using images taken during endoscopic examinations with non-magnified white-light imaging. Patients were classified as high-risk (patients with GC), moderate-risk (patients with current or past Helicobacter pylori infection or gastric atrophy), or low-risk (patients with no history of H. pylori infection or gastric atrophy). After selection, 20,960, 17,404, and 68,920 images were collected as training images for the high-, moderate-, and low-risk groups, respectively. RESULTS: Performance of the artificial intelligence (AI) system was evaluated by the prevalence of GC in each group using an independent validation dataset of patients who underwent endoscopic examination and H. pylori serum antibody testing. In total, 12,824 images from 454 patients were included in the analysis. The time required for diagnosing all the images was 345 seconds. The AI system diagnosed 46, 250, and 158 patients as low-, moderate-, and high risk, respectively. The prevalence of GC in the low-, moderate-, and high-risk groups was 2.2, 8.8, and 16.4%, respectively (P = 0.0017). Three experienced endoscopists also successfully stratified the risk; however, interobserver agreement was not satisfactory (kappa value of 0.27, indicating fair agreement). CONCLUSION: The current AI system detected significant differences in the prevalence of GC among the low-, moderate-, and high-risk groups, suggesting its potential for stratifying GC risk.

Application of Convolutional Neural Networks for Detection of Superficial Nonampullary Duodenal Epithelial Tumors in Esophagogastroduodenoscopic Images.

OBJECTIVES: A superficial nonampullary duodenal epithelial tumor (SNADET) is defined as a mucosal or submucosal sporadic tumor of the duodenum that does not arise from the papilla of Vater. SNADETs rarely metastasize to the lymph nodes, and most can be treated endoscopically. However, SNADETs are sometimes missed during esophagogastroduodenoscopic examination. In this study, we constructed a convolutional neural network (CNN) and evaluated its ability to detect SNADETs. METHODS: A deep CNN was pretrained and fine-tuned using a training data set of the endoscopic images of SNADETs (duodenal adenomas [N = 65] and high-grade dysplasias [HGDs] [N = 31] [total 531 images]). The CNN evaluated a separate set of images from 26 adenomas, 8 HGDs, and 681 normal tissue (total 1,080 images). The gold standard for both the training data set and test data set was a "true diagnosis" made by board-certified endoscopists and pathologists. A detected tumor was marked with a rectangular frame on the endoscopic image. If it overlapped at least a part of the "true tumor" diagnosed by board-certified endoscopists, the CNN was considered to have "detected" the SNADET. RESULTS: The trained CNN detected 94.7% (378 of 399) of SNADETs on an image basis (94% [280 of 298] of adenomas and 100% [101 of 101] of HGDs) and 100% on a tumor basis. The time needed for screening the 399 images containing SNADETs and all 1,080 images (including normal images) was 12 and 31 seconds, respectively. DISCUSSION: We used a novel algorithm to construct a CNN for detecting SNADETs in a short time.

Automatic detection and classification of protruding lesions in wireless capsule endoscopy images based on a deep convolutional neural network.

BACKGROUND AND AIMS: Protruding lesions of the small bowel vary in wireless capsule endoscopy (WCE) images, and their automatic detection may be difficult. We aimed to develop and test a deep learning-based system to automatically detect protruding lesions of various types in WCE images. METHODS: We trained a deep convolutional neural network (CNN), using 30,584 WCE images of protruding lesions from 292 patients. We evaluated CNN performance by calculating the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity, using an independent set of 17,507 test images from 93 patients, including 7507 images of protruding lesions from 73 patients. RESULTS: The developed CNN analyzed 17,507 images in 530.462 seconds. The AUC for detection of protruding lesions was 0.911 (95% confidence interval [Cl], 0.9069-0.9155). The sensitivity and specificity of the CNN were 90.7% (95% CI, 90.0%-91.4%) and 79.8% (95% CI, 79.0%-80.6%), respectively, at the optimal cut-off value of 0.317 for probability score. In a subgroup analysis of the category of protruding lesions, the sensitivities were 86.5%, 92.0%, 95.8%, 77.0%, and 94.4% for the detection of polyps, nodules, epithelial tumors, submucosal tumors, and venous structures, respectively. In individual patient analyses (n = 73), the detection rate of protruding lesions was 98.6%. CONCLUSION: We developed and tested a new computer-aided system based on a CNN to automatically detect various protruding lesions in WCE images. Patient-level analyses with larger cohorts and efforts to achieve better diagnostic performance are necessary in further studies.

Application of artificial intelligence using convolutional neural networks in determining the invasion depth of esophageal squamous cell carcinoma.

OBJECTIVES: In Japan, endoscopic resection (ER) is often used to treat esophageal squamous cell carcinoma (ESCC) when invasion depths are diagnosed as EP-SM1, whereas ESCC cases deeper than SM2 are treated by surgical operation or chemoradiotherapy. Therefore, it is crucial to determine the invasion depth of ESCC via preoperative endoscopic examination. Recently, rapid progress in the utilization of artificial intelligence (AI) with deep learning in medical fields has been achieved. In this study, we demonstrate the diagnostic ability of AI to measure ESCC invasion depth. METHODS: We retrospectively collected 1751 training images of ESCC at the Cancer Institute Hospital, Japan. We developed an AI-diagnostic system of convolutional neural networks using deep learning techniques with these images. Subsequently, 291 test images were prepared and reviewed by the AI-diagnostic system and 13 board-certified endoscopists to evaluate the diagnostic accuracy. RESULTS: The AI-diagnostic system detected 95.5% (279/291) of the ESCC in test images in 10 s, analyzed the 279 images and correctly estimated the invasion depth of ESCC with a sensitivity of 84.1% and accuracy of 80.9% in 6 s. The accuracy score of this system exceeded those of 12 out of 13 board-certified endoscopists, and its area under the curve (AUC) was greater than the AUCs of all endoscopists. CONCLUSIONS: The AI-diagnostic system demonstrated a higher diagnostic accuracy for ESCC invasion depth than those of endoscopists and, therefore, can be potentially used in ESCC diagnostics.

Endoscopic detection and differentiation of esophageal lesions using a deep neural network.

BACKGROUND AND AIMS: Diagnosing esophageal squamous cell carcinoma (SCC) depends on individual physician expertise and may be subject to interobserver variability. Therefore, we developed a computerized image-analysis system to detect and differentiate esophageal SCC. METHODS: A total of 9591 nonmagnified endoscopy (non-ME) and 7844 ME images of pathologically confirmed superficial esophageal SCCs and 1692 non-ME and 3435 ME images from noncancerous lesions or normal esophagus were used as training image data. Validation was performed using 255 non-ME white-light images, 268 non-ME narrow-band images/blue-laser images, and 204 ME narrow-band images/blue-laser images from 135 patients. The same validation test data were diagnosed by 15 board-certified specialists (experienced endoscopists). RESULTS: Regarding diagnosis by non-ME with narrow-band imaging/blue-laser imaging, the sensitivity, specificity, and accuracy were 100%, 63%, and 77%, respectively, for the artificial intelligence (AI) system and 92%, 69%, and 78%, respectively, for the experienced endoscopists. Regarding diagnosis by non-ME with white-light imaging, the sensitivity, specificity, and accuracy were 90%, 76%, and 81%, respectively, for the AI system and 87%, 67%, and 75%, respectively, for the experienced endoscopists. Regarding diagnosis by ME, the sensitivity, specificity, and accuracy were 98%, 56%, and 77%, respectively, for the AI system and 83%, 70%, and 76%, respectively, for the experienced endoscopists. There was no significant difference in the diagnostic performance between the AI system and the experienced endoscopists. CONCLUSIONS: Our AI system showed high sensitivity for detecting SCC by non-ME and high accuracy for differentiating SCC from noncancerous lesions by ME.

Convolutional Neural Network for Differentiating Gastric Cancer from Gastritis Using Magnified Endoscopy with Narrow Band Imaging.

BACKGROUND: Early detection of early gastric cancer (EGC) allows for less invasive cancer treatment. However, differentiating EGC from gastritis remains challenging. Although magnifying endoscopy with narrow band imaging (ME-NBI) is useful for differentiating EGC from gastritis, this skill takes substantial effort. Since the development of the ability to convolve the image while maintaining the characteristics of the input image (convolution neural network: CNN), allowing the classification of the input image (CNN system), the image recognition ability of CNN has dramatically improved. AIMS: To explore the diagnostic ability of the CNN system with ME-NBI for differentiating between EGC and gastritis. METHODS: A 22-layer CNN system was pre-trained using 1492 EGC and 1078 gastritis images from ME-NBI. A separate test data set (151 EGC and 107 gastritis images based on ME-NBI) was used to evaluate the diagnostic ability [accuracy, sensitivity, positive predictive value (PPV), and negative predictive value (NPV)] of the CNN system. RESULTS: The accuracy of the CNN system with ME-NBI images was 85.3%, with 220 of the 258 images being correctly diagnosed. The method's sensitivity, specificity, PPV, and NPV were 95.4%, 71.0%, 82.3%, and 91.7%, respectively. Seven of the 151 EGC images were recognized as gastritis, whereas 31 of the 107 gastritis images were recognized as EGC. The overall test speed was 51.83 images/s (0.02 s/image). CONCLUSIONS: The CNN system with ME-NBI can differentiate between EGC and gastritis in a short time with high sensitivity and NPV. Thus, the CNN system may complement current clinical practice of diagnosis with ME-NBI.

Classification for invasion depth of esophageal squamous cell carcinoma using a deep neural network compared with experienced endoscopists.

BACKGROUND AND AIMS: Cancer invasion depth is a critical factor affecting the choice of treatment in patients with superficial squamous cell carcinoma (SCC). However, the diagnosis of invasion depth is currently subjective and liable to interobserver variability. METHODS: We developed a deep learning-based artificial intelligence (AI) system based on Single Shot MultiBox Detector architecture for the assessment of superficial esophageal SCC. We obtained endoscopic images from patients with superficial esophageal SCC at our facility between December 2005 and December 2016. RESULTS: After excluding poor-quality images, 8660 non-magnified endoscopic (non-ME) and 5678 ME images from 804 superficial esophageal SCCs with pathologic proof of cancer invasion depth were used as the training dataset, and 405 non-ME images and 509 ME images from 155 patients were selected for the validation set. Our system showed a sensitivity of 90.1%, specificity of 95.8%, positive predictive value of 99.2%, negative predictive value of 63.9%, and an accuracy of 91.0% for differentiating pathologic mucosal and submucosal microinvasive (SM1) cancers from submucosal deep invasive (SM2/3) cancers. Cancer invasion depth was diagnosed by 16 experienced endoscopists using the same validation set, with an overall sensitivity of 89.8%, specificity of 88.3%, positive predictive value of 97.9%, negative predictive value of 65.5%, and an accuracy of 89.6%. CONCLUSIONS: This newly developed AI system showed favorable performance for diagnosing invasion depth in patients with superficial esophageal SCC, with comparable performance to experienced endoscopists.

Application of convolutional neural networks for evaluating Helicobacter pylori infection status on the basis of endoscopic images.

BACKGROUND AND AIM: We recently reported the role of artificial intelligence in the diagnosis of Helicobacter pylori (H. pylori) gastritis on the basis of endoscopic images. However, that study included only H. pylori-positive and -negative patients, excluding patients after H. pylori-eradication. In this study, we constructed a convolutional neural network (CNN) and evaluated its ability to ascertain all H. pylori infection statuses. METHODS: A deep CNN was pre-trained and fine-tuned on a dataset of 98,564 endoscopic images from 5236 patients (742 H. pylori-positive, 3649 -negative, and 845 -eradicated). A separate test data set (23,699 images from 847 patients; 70 positive, 493 negative, and 284 eradicated) was evaluated by the CNN. RESULTS: The trained CNN outputs a continuous number between 0 and 1 as the probability index for H. pylori infection status per image (Pp, H. pylori-positive; Pn, negative; Pe, eradicated). The most probable (largest number) of the three infectious statuses was selected as the 'CNN diagnosis'. Among 23,699 images, the CNN diagnosed 418 images as positive, 23,034 as negative, and 247 as eradicated. Because of the large number of H. pylori negative findings, the probability of H. pylori-negative was artificially re-defined as Pn -0.9, after which 80% (465/582) of negative diagnoses were accurate, 84% (147/174) eradicated, and 48% (44/91) positive. The time needed to diagnose 23,699 images was 261 seconds. CONCLUSION: We used a novel algorithm to construct a CNN for diagnosing H. pylori infection status on the basis of endoscopic images very quickly. ABBREVIATIONS: H. pylori: Helicobacter pylori; CNN: convolutional neural network; AI: artificial intelligence; EGD: esophagogastroduodenoscopies.

Diagnostic outcomes of esophageal cancer by artificial intelligence using convolutional neural networks.

BACKGROUND AND AIMS: The prognosis of esophageal cancer is relatively poor. Patients are usually diagnosed at an advanced stage when it is often too late for effective treatment. Recently, artificial intelligence (AI) using deep learning has made remarkable progress in medicine. However, there are no reports on its application for diagnosing esophageal cancer. Here, we demonstrate the diagnostic ability of AI to detect esophageal cancer including squamous cell carcinoma and adenocarcinoma. METHODS: We retrospectively collected 8428 training images of esophageal cancer from 384 patients at the Cancer Institute Hospital, Japan. Using these, we developed deep learning through convolutional neural networks (CNNs). We also prepared 1118 test images for 47 patients with 49 esophageal cancers and 50 patients without esophageal cancer to evaluate the diagnostic accuracy. RESULTS: The CNN took 27 seconds to analyze 1118 test images and correctly detected esophageal cancer cases with a sensitivity of 98%. CNN could detect all 7 small cancer lesions less than 10 mm in size. Although the positive predictive value for each image was 40%, misdiagnosing shadows and normal structures led to a negative predictive value of 95%. The CNN could distinguish superficial esophageal cancer from advanced cancer with an accuracy of 98%. CONCLUSIONS: The constructed CNN system for detecting esophageal cancer can analyze stored endoscopic images in a short time with high sensitivity. However, more training would lead to higher diagnostic accuracy. This system can facilitate early detection in practice, leading to a better prognosis in the near future.

Diagnosis using deep-learning artificial intelligence based on the endocytoscopic observation of the esophagus.

BACKGROUND AND AIMS: The endocytoscopic system (ECS) helps in virtual realization of histology and can aid in confirming histological diagnosis in vivo. We propose replacing biopsy-based histology for esophageal squamous cell carcinoma (ESCC) by using the ECS. We applied deep-learning artificial intelligence (AI) to analyse ECS images of the esophagus to determine whether AI can support endoscopists for the replacement of biopsy-based histology. METHODS: A convolutional neural network-based AI was constructed based on GoogLeNet and trained using 4715 ECS images of the esophagus (1141 malignant and 3574 non-malignant images). To evaluate the diagnostic accuracy of the AI, an independent test set of 1520 ECS images, collected from 55 consecutive patients (27 ESCCs and 28 benign esophageal lesions) were examined. RESULTS: On the basis of the receiver-operating characteristic curve analysis, the areas under the curve of the total images, higher magnification pictures, and lower magnification pictures were 0.85, 0.90, and 0.72, respectively. The AI correctly diagnosed 25 of the 27 ESCC cases, with an overall sensitivity of 92.6%. Twenty-five of the 28 non-cancerous lesions were diagnosed as non-malignant, with a specificity of 89.3% and an overall accuracy of 90.9%. Two cases of malignant lesions, misdiagnosed as non-malignant by the AI, were correctly diagnosed as malignant by the endoscopist. Among the 3 cases of non-cancerous lesions diagnosed as malignant by the AI, 2 were of radiation-related esophagitis and one was of gastroesophageal reflux disease. CONCLUSION: AI is expected to support endoscopists in diagnosing ESCC based on ECS images without biopsy-based histological reference.

Novel computer-assisted diagnosis system for endoscopic disease activity in patients with ulcerative colitis.

BACKGROUND AND AIMS: Evaluation of endoscopic disease activity for patients with ulcerative colitis (UC) is important when determining the treatment of choice. However, endoscopists require a certain period of training to evaluate the activity of inflammation properly, and interobserver variability exists. Therefore, we constructed a computer-assisted diagnosis (CAD) system using a convolutional neural network (CNN) and evaluated its performance using a large dataset of endoscopic images from patients with UC. METHODS: A CNN-based CAD system was constructed based on GoogLeNet architecture. The CNN was trained using 26,304 colonoscopy images from a cumulative total of 841 patients with UC, which were tagged with anatomic locations and Mayo endoscopic scores. The performance of the CNN in identifying normal mucosa (Mayo 0) and mucosal healing state (Mayo 0-1) was evaluated in an independent test set of 3981 images from 114 patients with UC, by calculating the areas under the receiver operating characteristic curves (AUROCs). In addition, AUROCs in the right side of the colon, left side of the colon, and rectum were evaluated. RESULTS: The CNN-based CAD system showed a high level of performance with AUROCs of 0.86 and 0.98 to identify Mayo 0 and 0-1, respectively. The performance of the CNN was better for the rectum than for the right side and left side of the colon when identifying Mayo 0 (AUROC = 0.92, 0.83, and 0.83, respectively). CONCLUSIONS: The performance of the CNN-based CAD system was robust when used to identify endoscopic inflammation severity in patients with UC, highlighting its promising role in supporting less-experienced endoscopists and reducing interobserver variability.

Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images.

BACKGROUND: Image recognition using artificial intelligence with deep learning through convolutional neural networks (CNNs) has dramatically improved and been increasingly applied to medical fields for diagnostic imaging. We developed a CNN that can automatically detect gastric cancer in endoscopic images. METHODS: A CNN-based diagnostic system was constructed based on Single Shot MultiBox Detector architecture and trained using 13,584 endoscopic images of gastric cancer. To evaluate the diagnostic accuracy, an independent test set of 2296 stomach images collected from 69 consecutive patients with 77 gastric cancer lesions was applied to the constructed CNN. RESULTS: The CNN required 47 s to analyze 2296 test images. The CNN correctly diagnosed 71 of 77 gastric cancer lesions with an overall sensitivity of 92.2%, and 161 non-cancerous lesions were detected as gastric cancer, resulting in a positive predictive value of 30.6%. Seventy of the 71 lesions (98.6%) with a diameter of 6 mm or more as well as all invasive cancers were correctly detected. All missed lesions were superficially depressed and differentiated-type intramucosal cancers that were difficult to distinguish from gastritis even for experienced endoscopists. Nearly half of the false-positive lesions were gastritis with changes in color tone or an irregular mucosal surface. CONCLUSION: The constructed CNN system for detecting gastric cancer could process numerous stored endoscopic images in a very short time with a clinically relevant diagnostic ability. It may be well applicable to daily clinical practice to reduce the burden of endoscopists.

Application of Convolutional Neural Networks in the Diagnosis of Helicobacter pylori Infection Based on Endoscopic Images.

BACKGROUND AND AIMS: The role of artificial intelligence in the diagnosis of Helicobacter pylori gastritis based on endoscopic images has not been evaluated. We constructed a convolutional neural network (CNN), and evaluated its ability to diagnose H. pylori infection. METHODS: A 22-layer, deep CNN was pre-trained and fine-tuned on a dataset of 32,208 images either positive or negative for H. pylori (first CNN). Another CNN was trained using images classified according to 8 anatomical locations (secondary CNN). A separate test data set (11,481 images from 397 patients) was evaluated by the CNN, and 23 endoscopists, independently. RESULTS: The sensitivity, specificity, accuracy, and diagnostic time were 81.9%, 83.4%, 83.1%, and 198s, respectively, for the first CNN, and 88.9%, 87.4%, 87.7%, and 194s, respectively, for the secondary CNN. These values for the 23 endoscopists were 79.0%, 83.2%, 82.4%, and 230±65min (85.2%, 89.3%, 88.6%, and 253±92min by 6 board-certified endoscopists), respectively. The secondary CNN had a significantly higher accuracy than endoscopists (by 5.3%; 95% CI, 0.3-10.2). CONCLUSION: H. pylori gastritis could be diagnosed based on endoscopic images using CNN with higher accuracy and in a considerably shorter time compared to manual diagnosis by endoscopists.