Gaussian affinity and GIoU-based loss for perforation detection and localization from colonoscopy videos.

PURPOSE: Endoscopic submucosal dissection (ESD) is a minimally invasive treatment for early gastric cancer. However, perforations may happen and cause peritonitis during ESD. Thus, there is a potential demand for a computer-aided diagnosis system to support physicians in ESD. This paper presents a method to detect and localize perforations from colonoscopy videos to avoid perforation ignoring or enlarging by ESD physicians. METHOD: We proposed a training method for YOLOv3 by using GIoU and Gaussian affinity losses for perforation detection and localization in colonoscopic images. In this method, the object functional contains the generalized intersection over Union loss and Gaussian affinity loss. We propose a training method for the architecture of YOLOv3 with the presented loss functional to detect and localize perforations precisely. RESULTS: To qualitatively and quantitatively evaluate the presented method, we created a dataset from 49 ESD videos. The results of the presented method on our dataset revealed a state-of-the-art performance of perforation detection and localization, which achieved 0.881 accuracy, 0.869 AUC, and 0.879 mean average precision. Furthermore, the presented method is able to detect a newly appeared perforation in 0.1 s. CONCLUSIONS: The experimental results demonstrated that YOLOv3 trained by the presented loss functional were very effective in perforation detection and localization. The presented method can quickly and precisely remind physicians of perforation happening in ESD. We believe a future CAD system can be constructed for clinical applications with the proposed method.

Positive-gradient-weighted object activation mapping: visual explanation of object detector towards precise colorectal-polyp localisation.

PURPOSE: Precise polyp detection and localisation are essential for colonoscopy diagnosis. Statistical machine learning with a large-scale data set can contribute to the construction of a computer-aided diagnosis system for the prevention of overlooking and miss-localisation of a polyp in colonoscopy. We propose new visual explaining methods for a well-trained object detector, which achieves fast and accurate polyp detection with a bounding box towards a precise automated polyp localisation. METHOD: We refine gradient-weighted class activation mapping for more accurate highlighting of important patterns in processing a convolutional neural network. Extending the refined mapping into multiscaled processing, we define object activation mapping that highlights important object patterns in an image for a detection task. Finally, we define polyp activation mapping to achieve precise polyp localisation by integrating adaptive local thresholding into object activation mapping. We experimentally evaluate the proposed visual explaining methods with four publicly available databases. RESULTS: The refined mapping visualises important patterns in each convolutional layer more accurately than the original gradient-weighted class activation mapping. The object activation mapping clearly visualises important patterns in colonoscopic images for polyp detection. The polyp activation mapping localises the detected polyps in ETIS-Larib, CVC-Clinic and Kvasir-SEG database with mean Dice scores of 0.76, 0.72 and 0.72, respectively. CONCLUSIONS: We developed new visual explaining methods for a convolutional neural network by refining and extending gradient-weighted class activation mapping. Experimental results demonstrated the validity of the proposed methods by showing that accurate visualisation of important patterns and localisation of polyps in a colonoscopic image. The proposed visual explaining methods are useful for the interpreting and applying a trained polyp detector.

Impact of artificial intelligence on colorectal polyp detection for early-career endoscopists: an international comparative study.

BACKGROUND: Artificial intelligence (AI) for polyp detection is being introduced to colonoscopy, but there is uncertainty how this affects endoscopists' ability to detect polyps and neoplasms. We performed a video-based study to address whether AI improved the endoscopists' performance to detect polyps. METHODS: We established a dataset of 200 colonoscopy videos (length 5 s; 100 without polyps and 100 with one polyp). About 33 early-career endoscopists (50-400 colonoscopies performed) from 10 European countries classified each video as either 'polyp present' or 'polyp not present'. The video assessment was performed twice with a four-week interval. The first assessment was performed without any AI tool, whereas the second was performed with an AI tool for polyp detection. The primary endpoint was early-career endoscopists' sensitivity to detect polyps. Gold standard for presence and histology of polyps were confirmed by two expert endoscopists and pathologists, respectively. McNemar's test was used for statistical significance. RESULTS: There were 86 neoplastic and 14 non-neoplastic polyps (mean size 5.6 mm) in the 100 videos with polyps. Early-career endoscopists' sensitivity to detect polyps increased from 86.3% (95% confidence interval [CI]: 85.1-87.5%) to 91.7% (95%CI: 90.7-92.6%) with the AI aid (p < .0001). Their sensitivity to detect neoplastic polyps increased from 85.4% (95% CI: 84.0-86.7%) to 92.1% (95%CI: 91.1-93.1%) with the AI aid (p < .0001). CONCLUSION: The polyp detection AI tool helped early-career endoscopists to increase their sensitivity to identify all polyps and neoplastic polyps during colonoscopy.

Impact of the clinical use of artificial intelligence-assisted neoplasia detection for colonoscopy: a large-scale prospective, propensity score-matched study (with video).

BACKGROUND AND AIMS: Recently, the use of computer-aided detection (CADe) for colonoscopy has been investigated to improve the adenoma detection rate (ADR). We aimed to assess the efficacy of a regulatory-approved CADe in a large-scale study with high numbers of patients and endoscopists. METHODS: This was a propensity score-matched prospective study that took place at a university hospital between July 2020 and December 2020. We recruited patients aged ≥20 years who were scheduled for colonoscopy. Patients with polyposis, inflammatory bowel disease, or incomplete colonoscopy were excluded. We used a regulatory-approved CADe system and conducted a propensity score matching-based comparison of the ADR between patients examined with and without CADe as the primary outcome. RESULTS: During the study period, 2261 patients underwent colonoscopy with the CADe system or routine colonoscopy, and 172 patients were excluded in accordance with the exclusion criteria. Thirty endoscopists (9 nonexperts and 21 experts) were involved in this study. Propensity score matching was conducted using 5 factors, resulting in 1836 patients included in the analysis (918 patients in each group). The ADR was significantly higher in the CADe group than in the control group (26.4% vs 19.9%, respectively; relative risk, 1.32; 95% confidence interval, 1.12-1.57); however, there was no significant increase in the advanced neoplasia detection rate (3.7% vs 2.9%, respectively). CONCLUSIONS: The use of the CADe system for colonoscopy significantly increased the ADR in a large-scale prospective study including 30 endoscopists (Clinical trial registration number: UMIN000040677.).

Real-Time Artificial Intelligence-Based Optical Diagnosis of Neoplastic Polyps during Colonoscopy.

Real-Time AI-Based Diagnosis of Neoplastic PolypsColonoscopists diagnosed small colonic polyps as benign or malignant on the basis of their appearance. The results were compared in real time to see if CADx could distinguish among polyps requiring removal. For standard visual inspection versus CADx, we determined sensitivity for diagnosis (88.4% vs. 90.4%) and high confidence in assessment (74.2% vs. 92.6%).

Artificial intelligence and computer-aided diagnosis for colonoscopy: where do we stand now?

Computer-aided diagnosis (CAD) for colonoscopy with use of artificial intelligence (AI) is catching increased attention of endoscopists. CAD allows automated detection and pathological prediction, namely optical biopsy, of colorectal polyps during real-time endoscopy, which help endoscopists avoid missing and/or misdiagnosing colorectal lesions. With the increased number of publications in this field and emergence of the AI medical device that have already secured regulatory approval, CAD in colonoscopy is now being implemented into clinical practice. On the other side, drawbacks and weak points of CAD in colonoscopy have not been thoroughly discussed. In this review, we provide an overview of CAD for optical biopsy of colorectal lesions with a particular focus on its clinical applications and limitations.

Binary polyp-size classification based on deep-learned spatial information.

PURPOSE: The size information of detected polyps is an essential factor for diagnosis in colon cancer screening. For example, adenomas and sessile serrated polyps that are [Formula: see text] mm are considered advanced, and shorter surveillance intervals are recommended for smaller polyps. However, sometimes the subjective estimations of endoscopists are incorrect and overestimate the sizes. To circumvent these difficulties, we developed a method for automatic binary polyp-size classification between two polyp sizes: from 1 to 9 mm and [Formula: see text] mm. METHOD: We introduce a binary polyp-size classification method that estimates a polyp's three-dimensional spatial information. This estimation is comprised of polyp localisation and depth estimation. The combination of location and depth information expresses a polyp's three-dimensional shape. In experiments, we quantitatively and qualitatively evaluate the proposed method using 787 polyps of both protruded and flat types. RESULTS: The proposed method's best classification accuracy outperformed the fine-tuned state-of-the-art image classification methods. Post-processing of sequential voting increased the classification accuracy and achieved classification accuracy of 0.81 and 0.88 for polyps ranging from 1 to 9 mm and others that are [Formula: see text] mm. Qualitative analysis revealed the importance of polyp localisation even in polyp-size classification. CONCLUSIONS: We developed a binary polyp-size classification method by utilising the estimated three-dimensional shape of a polyp. Experiments demonstrated accurate classification for both protruded- and flat-type polyps, even though the flat type have ambiguous boundary between a polyp and colon wall.

Artificial intelligence-assisted colonic endocytoscopy for cancer recognition: a multicenter study.

Background and study aims Large adenomas are sometimes misidentified as cancers during colonoscopy and are surgically removed. To address this overtreatment, we developed an artificial intelligence (AI) tool that identified cancerous pathology in vivo with high specificity. We evaluated our AI tool under the supervision of a government agency to obtain regulatory approval. Patients and methods The AI tool outputted three pathological class predictions (cancer, adenoma, or non-neoplastic) for endocytoscopic images obtained at 520-fold magnification and previously trained on 68,082 images from six academic centers. A validation test was developed, employing 500 endocytoscopic images taken from various parts of randomly selected 50 large (≥ 20 mm) colorectal lesions (10 images per lesion). An expert board labelled each of the 500 images with a histopathological diagnosis, which was made using endoscopic and histopathological images. The validation test was performed using the AI tool under a controlled environment. The primary outcome measure was the specificity in identifying cancer. Results The validation test consisted of 30 cancers, 15 adenomas, and five non-neoplastic lesions. The AI tool could analyze 83.6 % of the images (418/500): 231 cancers, 152 adenomas, and 35 non-neoplastic lesions. Among the analyzable images, the AI tool identified the three pathological classes with an overall accuracy of 91.9 % (384/418, 95 % confidence interval [CI]: 88.8 %-94.3 %). Its sensitivity and specificity for differentiating cancer was 91.8 % (212/231, 95 % CI: 87.5 %-95.0 %) and 97.3 % (182/187, 95 % CI: 93.9 %-99.1 %), respectively. Conclusions The newly developed AI system designed for endocytoscopy showed excellent specificity in identifying colorectal cancer.

Unsupervised colonoscopic depth estimation by domain translations with a Lambertian-reflection keeping auxiliary task.

PURPOSE: A three-dimensional (3D) structure extraction technique viewed from a two-dimensional image is essential for the development of a computer-aided diagnosis (CAD) system for colonoscopy. However, a straightforward application of existing depth-estimation methods to colonoscopic images is impossible or inappropriate due to several limitations of colonoscopes. In particular, the absence of ground-truth depth for colonoscopic images hinders the application of supervised machine learning methods. To circumvent these difficulties, we developed an unsupervised and accurate depth-estimation method. METHOD: We propose a novel unsupervised depth-estimation method by introducing a Lambertian-reflection model as an auxiliary task to domain translation between real and virtual colonoscopic images. This auxiliary task contributes to accurate depth estimation by maintaining the Lambertian-reflection assumption. In our experiments, we qualitatively evaluate the proposed method by comparing it with state-of-the-art unsupervised methods. Furthermore, we present two quantitative evaluations of the proposed method using a measuring device, as well as a new 3D reconstruction technique and measured polyp sizes. RESULTS: Our proposed method achieved accurate depth estimation with an average estimation error of less than 1 mm for regions close to the colonoscope in both of two types of quantitative evaluations. Qualitative evaluation showed that the introduced auxiliary task reduces the effects of specular reflections and colon wall textures on depth estimation and our proposed method achieved smooth depth estimation without noise, thus validating the proposed method. CONCLUSIONS: We developed an accurate depth-estimation method with a new type of unsupervised domain translation with the auxiliary task. This method is useful for analysis of colonoscopic images and for the development of a CAD system since it can extract accurate 3D information.

Current status and future perspective on artificial intelligence for lower endoscopy.

The global incidence and mortality rate of colorectal cancer remains high. Colonoscopy is regarded as the gold standard examination for detecting and eradicating neoplastic lesions. However, there are some uncertainties in colonoscopy practice that are related to limitations in human performance. First, approximately one-fourth of colorectal neoplasms are missed on a single colonoscopy. Second, it is still difficult for non-experts to perform adequately regarding optical biopsy. Third, recording of some quality indicators (e.g. cecal intubation, bowel preparation, and withdrawal speed) which are related to adenoma detection rate, is sometimes incomplete. With recent improvements in machine learning techniques and advances in computer performance, artificial intelligence-assisted computer-aided diagnosis is being increasingly utilized by endoscopists. In particular, the emergence of deep-learning, data-driven machine learning techniques have made the development of computer-aided systems easier than that of conventional machine learning techniques, the former currently being considered the standard artificial intelligence engine of computer-aided diagnosis by colonoscopy. To date, computer-aided detection systems seem to have improved the rate of detection of neoplasms. Additionally, computer-aided characterization systems may have the potential to improve diagnostic accuracy in real-time clinical practice. Furthermore, some artificial intelligence-assisted systems that aim to improve the quality of colonoscopy have been reported. The implementation of computer-aided system clinical practice may provide additional benefits such as helping in educational poorly performing endoscopists and supporting real-time clinical decision-making. In this review, we have focused on computer-aided diagnosis during colonoscopy reported by gastroenterologists and discussed its status, limitations, and future prospects.

Development of a computer-aided detection system for colonoscopy and a publicly accessible large colonoscopy video database (with video).

BACKGROUND AND AIMS: Artificial intelligence (AI)-assisted polyp detection systems for colonoscopic use are currently attracting attention because they may reduce the possibility of missed adenomas. However, few systems have the necessary regulatory approval for use in clinical practice. We aimed to develop an AI-assisted polyp detection system and to validate its performance using a large colonoscopy video database designed to be publicly accessible. METHODS: To develop the deep learning-based AI system, 56,668 independent colonoscopy images were obtained from 5 centers for use as training images. To validate the trained AI system, consecutive colonoscopy videos taken at a university hospital between October 2018 and January 2019 were searched to construct a database containing polyps with unbiased variance. All images were annotated by endoscopists according to the presence or absence of polyps and the polyps' locations with bounding boxes. RESULTS: A total of 1405 videos acquired during the study period were identified for the validation database, 797 of which contained at least 1 polyp. Of these, 100 videos containing 100 independent polyps and 13 videos negative for polyps were randomly extracted, resulting in 152,560 frames (49,799 positive frames and 102,761 negative frames) for the database. The AI showed 90.5% sensitivity and 93.7% specificity for frame-based analysis. The per-polyp sensitivities for all, diminutive, protruded, and flat polyps were 98.0%, 98.3%, 98.5%, and 97.0%, respectively. CONCLUSIONS: Our trained AI system was validated with a new large publicly accessible colonoscopy database and could identify colorectal lesions with high sensitivity and specificity. (Clinical trial registration number: UMIN 000037064.).

Artificial Intelligence System to Determine Risk of T1 Colorectal Cancer Metastasis to Lymph Node.

BACKGROUND & AIMS: In accordance with guidelines, most patients with T1 colorectal cancers (CRC) undergo surgical resection with lymph node dissection, despite the low incidence (∼10%) of metastasis to lymph nodes. To reduce unnecessary surgical resections, we used artificial intelligence to build a model to identify T1 colorectal tumors at risk for metastasis to lymph node and validated the model in a separate set of patients. METHODS: We collected data from 3134 patients with T1 CRC treated at 6 hospitals in Japan from April 1997 through September 2017 (training cohort). We developed a machine-learning artificial neural network (ANN) using data on patients' age and sex, as well as tumor size, location, morphology, lymphatic and vascular invasion, and histologic grade. We then conducted the external validation on the ANN model using independent 939 patients at another hospital during the same period (validation cohort). We calculated areas under the receiver operator characteristics curves (AUCs) for the ability of the model and US guidelines to identify patients with lymph node metastases. RESULTS: Lymph node metastases were found in 319 (10.2%) of 3134 patients in the training cohort and 79 (8.4%) of /939 patients in the validation cohort. In the validation cohort, the ANN model identified patients with lymph node metastases with an AUC of 0.83, whereas the guidelines identified patients with lymph node metastases with an AUC of 0.73 (P < .001). When the analysis was limited to patients with initial endoscopic resection (n = 517), the ANN model identified patients with lymph node metastases with an AUC of 0.84 and the guidelines identified these patients with an AUC of 0.77 (P = .005). CONCLUSIONS: The ANN model outperformed guidelines in identifying patients with T1 CRCs who had lymph node metastases. This model might be used to determine which patients require additional surgery after endoscopic resection of T1 CRCs. UMIN Clinical Trials Registry no: UMIN000038609.

Robust endocytoscopic image classification based on higher-order symmetric tensor analysis and multi-scale topological statistics.

PURPOSE: An endocytoscope is a new type of endoscope that enables users to perform conventional endoscopic observation and ultramagnified observation at the cell level. Although endocytoscopy is expected to improve the cost-effectiveness of colonoscopy, endocytoscopic image diagnosis requires much knowledge and high-level experience for physicians. To circumvent this difficulty, we developed a robust endocytoscopic (EC) image classification method for the construction of a computer-aided diagnosis (CAD) system, since real-time CAD can resolve accuracy issues and reduce interobserver variability. METHOD: We propose a novel feature extraction method by introducing higher-order symmetric tensor analysis to the computation of multi-scale topological statistics on an image, and we integrate this feature extraction with EC image classification. We experimentally evaluate the classification accuracy of our proposed method by comparing it with three deep learning methods. We conducted this comparison by using our large-scale multi-hospital dataset of about 55,000 images of over 3800 patients. RESULTS: Our proposed method achieved an average 90% classification accuracy for all the images in four hospitals even though the best deep learning method achieved 95% classification accuracy for images in only one hospital. In the case with a rejection option, the proposed method achieved expert-level accurate classification. These results demonstrate the robustness of our proposed method against pit pattern variations, including differences of colours, contrasts, shapes, and hospitals. CONCLUSIONS: We developed a robust EC image classification method with novel feature extraction. This method is useful for the construction of a practical CAD system, since it has sufficient generalisation ability.

How Far Will Clinical Application of AI Applications Advance for Colorectal Cancer Diagnosis?

Integrating artificial intelligence (AI) applications into colonoscopy practice is being accelerated as deep learning technologies emerge. In this field, most of the preceding research has focused on polyp detection and characterization, which can mitigate inherent human errors accompanying colonoscopy procedures. On the other hand, more challenging research areas are currently capturing attention: the automated prediction of invasive cancers. Colorectal cancers (CRCs) harbor potential lymph node metastasis when they invade deeply into submucosal layers, which should be resected surgically rather than endoscopically. However, pretreatment discrimination of deeply invasive submucosal CRCs is considered difficult, according to previous prospective studies (e.g., <70% sensitivity), leading to an increased number of unnecessary surgeries for large adenomas or slightly invasive submucosal CRCs. AI is now expected to overcome this challenging hurdle because it is considered to provide better performance in predicting invasive cancer than non-expert endoscopists. In this review, we introduce five relevant publications in this area. Unfortunately, progress in this research area is in a very preliminary phase, compared to that of automated polyp detection and characterization, because of the lack of number of invasive CRCs used for machine learning. However, this issue will be overcome with more target images and cases. The research field of AI for invasive CRCs is just starting but could be a game changer of patient care in the near future, given rapidly growing technologies, and research will gradually increase.

Artificial Intelligence-assisted System Improves Endoscopic Identification of Colorectal Neoplasms.

BACKGROUND & AIMS: Precise optical diagnosis of colorectal polyps could improve the cost-effectiveness of colonoscopy and reduce polypectomy-related complications. However, it is difficult for community-based non-experts to obtain sufficient diagnostic performance. Artificial intelligence-based systems have been developed to analyze endoscopic images; they identify neoplasms with high accuracy and low interobserver variation. We performed a multi-center study to determine the diagnostic accuracy of EndoBRAIN, an artificial intelligence-based system that analyzes cell nuclei, crypt structure, and microvessels in endoscopic images, in identification of colon neoplasms. METHODS: The EndoBRAIN system was initially trained using 69,142 endocytoscopic images, taken at 520-fold magnification, from patients with colorectal polyps who underwent endoscopy at 5 academic centers in Japan from October 2017 through March 2018. We performed a retrospective comparative analysis of the diagnostic performance of EndoBRAIN vs that of 30 endoscopists (20 trainees and 10 experts); the endoscopists assessed images from 100 cases produced via white-light microscopy, endocytoscopy with methylene blue staining, and endocytoscopy with narrow-band imaging. EndoBRAIN was used to assess endocytoscopic, but not white-light, images. The primary outcome was the accuracy of EndoBrain in distinguishing neoplasms from non-neoplasms, compared with that of endoscopists, using findings from pathology analysis as the reference standard. RESULTS: In analysis of stained endocytoscopic images, EndoBRAIN identified colon lesions with 96.9% sensitivity (95% CI, 95.8%-97.8%), 100% specificity (95% CI, 99.6%-100%), 98% accuracy (95% CI, 97.3%-98.6%), a 100% positive-predictive value (95% CI, 99.8%-100%), and a 94.6% negative-predictive (95% CI, 92.7%-96.1%); these values were all significantly greater than those of the endoscopy trainees and experts. In analysis of narrow-band images, EndoBRAIN distinguished neoplastic from non-neoplastic lesions with 96.9% sensitivity (95% CI, 95.8-97.8), 94.3% specificity (95% CI, 92.3-95.9), 96.0% accuracy (95% CI, 95.1-96.8), a 96.9% positive-predictive value, (95% CI, 95.8-97.8), and a 94.3% negative-predictive value (95% CI, 92.3-95.9); these values were all significantly higher than those of the endoscopy trainees, sensitivity and negative-predictive value were significantly higher but the other values are comparable to those of the experts. CONCLUSIONS: EndoBRAIN accurately differentiated neoplastic from non-neoplastic lesions in stained endocytoscopic images and endocytoscopic narrow-band images, when pathology findings were used as the standard. This technology has been authorized for clinical use by the Japanese regulatory agency and should be used in endoscopic evaluation of small polyps more widespread clinical settings. UMIN clinical trial no: UMIN000028843.

Artificial intelligence and colonoscopy: Current status and future perspectives.

BACKGROUND AND AIM: Application of artificial intelligence in medicine is now attracting substantial attention. In the field of gastrointestinal endoscopy, computer-aided diagnosis (CAD) for colonoscopy is the most investigated area, although it is still in the preclinical phase. Because colonoscopy is carried out by humans, it is inherently an imperfect procedure. CAD assistance is expected to improve its quality regarding automated polyp detection and characterization (i.e. predicting the polyp's pathology). It could help prevent endoscopists from missing polyps as well as provide a precise optical diagnosis for those detected. Ultimately, these functions that CAD provides could produce a higher adenoma detection rate and reduce the cost of polypectomy for hyperplastic polyps. METHODS AND RESULTS: Currently, research on automated polyp detection has been limited to experimental assessments using an algorithm based on ex vivo videos or static images. Performance for clinical use was reported to have >90% sensitivity with acceptable specificity. In contrast, research on automated polyp characterization seems to surpass that for polyp detection. Prospective studies of in vivo use of artificial intelligence technologies have been reported by several groups, some of which showed a >90% negative predictive value for differentiating diminutive (≤5 mm) rectosigmoid adenomas, which exceeded the threshold for optical biopsy. CONCLUSION: We introduce the potential of using CAD for colonoscopy and describe the most recent conditions for regulatory approval for artificial intelligence-assisted medical devices.

Artificial intelligence and upper gastrointestinal endoscopy: Current status and future perspective.

With recent breakthroughs in artificial intelligence, computer-aided diagnosis (CAD) for upper gastrointestinal endoscopy is gaining increasing attention. Main research focuses in this field include automated identification of dysplasia in Barrett's esophagus and detection of early gastric cancers. By helping endoscopists avoid missing and mischaracterizing neoplastic change in both the esophagus and the stomach, these technologies potentially contribute to solving current limitations of gastroscopy. Currently, optical diagnosis of early-stage dysplasia related to Barrett's esophagus can be precisely achieved only by endoscopists proficient in advanced endoscopic imaging, and the false-negative rate for detecting gastric cancer is approximately 10%. Ideally, these novel technologies should work during real-time gastroscopy to provide on-site decision support for endoscopists regardless of their skill; however, previous studies of these topics remain ex vivo and experimental in design. Therefore, the feasibility, effectiveness, and safety of CAD for upper gastrointestinal endoscopy in clinical practice remain unknown, although a considerable number of pilot studies have been conducted by both engineers and medical doctors with excellent results. This review summarizes current publications relating to CAD for upper gastrointestinal endoscopy from the perspective of endoscopists and aims to indicate what is required for future research and implementation in clinical practice.

Stable polyp-scene classification via subsampling and residual learning from an imbalanced large dataset.

This Letter presents a stable polyp-scene classification method with low false positive (FP) detection. Precise automated polyp detection during colonoscopies is essential for preventing colon-cancer deaths. There is, therefore, a demand for a computer-assisted diagnosis (CAD) system for colonoscopies to assist colonoscopists. A high-performance CAD system with spatiotemporal feature extraction via a three-dimensional convolutional neural network (3D CNN) with a limited dataset achieved about 80% detection accuracy in actual colonoscopic videos. Consequently, further improvement of a 3D CNN with larger training data is feasible. However, the ratio between polyp and non-polyp scenes is quite imbalanced in a large colonoscopic video dataset. This imbalance leads to unstable polyp detection. To circumvent this, the authors propose an efficient and balanced learning technique for deep residual learning. The authors' method randomly selects a subset of non-polyp scenes whose number is the same number of still images of polyp scenes at the beginning of each epoch of learning. Furthermore, they introduce post-processing for stable polyp-scene classification. This post-processing reduces the FPs that occur in the practical application of polyp-scene classification. They evaluate several residual networks with a large polyp-detection dataset consisting of 1027 colonoscopic videos. In the scene-level evaluation, their proposed method achieves stable polyp-scene classification with 0.86 sensitivity and 0.97 specificity.

Real-Time Use of Artificial Intelligence in Identification of Diminutive Polyps During Colonoscopy: A Prospective Study.

Background: Computer-aided diagnosis (CAD) for colonoscopy may help endoscopists distinguish neoplastic polyps (adenomas) requiring resection from nonneoplastic polyps not requiring resection, potentially reducing cost. Objective: To evaluate the performance of real-time CAD with endocytoscopes (×520 ultramagnifying colonoscopes providing microvascular and cellular visualization of colorectal polyps after application of the narrow-band imaging [NBI] and methylene blue staining modes, respectively). Design: Single-group, open-label, prospective study. (UMIN [University hospital Medical Information Network] Clinical Trial Registry: UMIN000027360). Setting: University hospital. Participants: 791 consecutive patients undergoing colonoscopy and 23 endoscopists. Intervention: Real-time use of CAD during colonoscopy. Measurements: CAD-predicted pathology (neoplastic or nonneoplastic) of detected diminutive polyps (≤5 mm) on the basis of real-time outputs compared with pathologic diagnosis of the resected specimen (gold standard). The primary end point was whether CAD with the stained mode produced a negative predictive value (NPV) of 90% or greater for identifying diminutive rectosigmoid adenomas, the threshold required to "diagnose-and-leave" nonneoplastic polyps. Best- and worst-case scenarios assumed that polyps lacking either CAD diagnosis or pathology were true- or false-positive or true- or false-negative, respectively. Results: Overall, 466 diminutive (including 250 rectosigmoid) polyps from 325 patients were assessed by CAD, with a pathologic prediction rate of 98.1% (457 of 466). The NPVs of CAD for diminutive rectosigmoid adenomas were 96.4% (95% CI, 91.8% to 98.8%) (best-case scenario) and 93.7% (CI, 88.3% to 97.1%) (worst-case scenario) with stained mode and 96.5% (CI, 92.1% to 98.9%) (best-case scenario) and 95.2% (CI, 90.3% to 98.0%) (worst-case scenario) with NBI. Limitation: Two thirds of the colonoscopies were conducted by experts who had each experienced more than 200 endocytoscopies; 186 polyps not assessed by CAD were excluded. Conclusion: Real-time CAD can achieve the performance level required for a diagnose-and-leave strategy for diminutive, nonneoplastic rectosigmoid polyps. Primary Funding Source: Japan Society for the Promotion of Science.