Artificial Intelligence-assisted colonoscopy and colorectal cancer screening: Where are we going?

Colorectal cancer is a significant global health concern, necessitating effective screening strategies to reduce its incidence and mortality rates. Colonoscopy plays a crucial role in the detection and removal of colorectal neoplastic precursors. However, there are limitations and variations in the performance of endoscopists, leading to missed lesions and suboptimal outcomes. The emergence of artificial intelligence (AI) in endoscopy offers promising opportunities to improve the quality and efficacy of screening colonoscopies. In particular, AI applications, including computer-aided detection (CADe) and computer-aided characterization (CADx), have demonstrated the potential to enhance adenoma detection and optical diagnosis accuracy. Additionally, AI-assisted quality control systems aim to standardize the endoscopic examination process. This narrative review provides an overview of AI principles and discusses the current knowledge on AI-assisted endoscopy in the context of screening colonoscopies. It highlights the significant role of AI in improving lesion detection, characterization, and quality assurance during colonoscopy. However, further well-designed studies are needed to validate the clinical impact and cost-effectiveness of AI-assisted colonoscopy before its widespread implementation.

Direct comparison of multiple computer-aided polyp detection systems.

BACKGROUND AND STUDY AIMS: Artificial intelligence (AI)-based systems for computer-aided detection (CADe) of polyps receive regular updates and occasionally offer customizable detection thresholds, both of which impact their performance, but little is known about these effects. This study aimed to compare the performance of different CADe systems on the same benchmark dataset. METHODS: 101 colonoscopy videos were used as benchmark. Each video frame with a visible polyp was manually annotated with bounding boxes, resulting in 129 705 polyp images. The videos were then analyzed by three different CADe systems, representing five conditions: two versions of GI Genius, Endo-AID with detection Types A and B, and EndoMind, a freely available system. Evaluation included an analysis of sensitivity and false-positive rate, among other metrics. RESULTS: Endo-AID detection Type A, the earlier version of GI Genius, and EndoMind detected all 93 polyps. Both the later version of GI Genius and Endo-AID Type B missed 1 polyp. The mean per-frame sensitivities were 50.63 % and 67.85 %, respectively, for the earlier and later versions of GI Genius, 65.60 % and 52.95 %, respectively, for Endo-AID Types A and B, and 60.22 % for EndoMind. CONCLUSIONS: This study compares the performance of different CADe systems, different updates, and different configuration modes. This might help clinicians to select the most appropriate system for their specific needs.

The Role of Specialized Instruments for Advanced Endoscopic Resections in Gastrointestinal Disease.

INTRODUCTION: Advanced endoscopic therapy techniques have been developed and have created alternative treatment options to surgical therapy for several gastrointestinal diseases. This work will focus on new endoscopic tools for special indications of advanced endoscopic resections (ER), especially endoscopic submucosal dissection (ESD), which were developed in our institution. This paper aims to analyze these specialized instruments and identify their status. METHODS: Initially, the technical process of ESD was analyzed, and the following limitations of the different endoscopic steps and the necessary manipulations were determined: the problem of traction-countertraction, the grasping force needed to pull on tissue, the instrument tip maneuverability, the limited angulation/triangulation, and the mobility of the scope and instruments. Five instruments developed by our team were used: the Endo-dissector, additional working channel system, external independent next-to-the-scope grasper, 3D overtube working station, and over-the-scope grasper. The instruments were used and applied according to their special functions in dry lab, experimental in vivo, and clinical conditions by the members of our team. RESULTS: The Endo-dissector has a two-fold function: (1) grasping submucosal tissue with enough precision and strength to pull it off the surrounding mucosa and muscle, avoiding damage during energy application and (2) effectively dividing tissue using monopolar energy. The AWC system quickly fulfills the lack of a second working channel as needed to complete the endoscopic task on demand. The EINTS grasper can deliver a serious grasping force, which may be necessary for a traction-countertraction situation during endoscopic resection for lifting a larger specimen. The 3D overtube multifunctional platform provides surgical-like work with bimanual-operated instruments at the tip of the scope, which allows for a coordinated approach during lesion treatment. The OTSG is a grasping tool with very special features for cleaning cavities with debris. CONCLUSIONS: The research and development of instruments with special features can solve unmet needs in advanced endoscopic procedures. The latter may help to increase indications for the endoscopic resections of gut lesions in the future.

Preclinical Evaluation of a Microwave-Based Accessory Device for Colonoscopy in an In Vivo Porcine Model with Colorectal Polyps.

BACKGROUND AND AIMS: Colonoscopy is currently the most effective way of detecting colorectal cancer and removing polyps, but it has some drawbacks and can miss up to 22% of polyps. Microwave imaging has the potential to provide a 360° view of the colon and addresses some of the limitations of conventional colonoscopy. This study evaluates the feasibility of a microwave-based colonoscopy in an in vivo porcine model. METHODS: A prototype device with microwave antennas attached to a conventional endoscope was tested on four healthy pigs and three gene-targeted pigs with mutations in the adenomatous polyposis coli gene. The first four animals were used to evaluate safety and maneuverability and compatibility with endoscopic tools. The ability to detect polyps was tested in a series of three gene-targeted pigs. RESULTS: the microwave-based device did not affect endoscopic vision or cause any adverse events such as deep mural injuries. The microwave system was stable during the procedures, and the detection algorithm showed a maximum detection signal for adenomas compared with healthy mucosa. CONCLUSIONS: Microwave-based colonoscopy is feasible and safe in a preclinical model, and it has the potential to improve polyp detection. Further investigations are required to assess the device's efficacy in humans.

New Concept of External and Independent Next-to-the-Scope Intraluminal Grasping Device (EINTS).

Background: Interventional endoscopic procedures require complex manipulations and precise maneuvering of end-effectors. One focus in research on improved endoscopic instrument function was based on surgical experience to gain additional traction. The idea has emerged using assisting instruments by applying external tools next-to-the endoscope to follow surgical concepts. The aim of this study is the assessment of flexible endoscopic grasping instruments regarding their function and working-radius introducing the concept of an intraluminal "next-to the-scope" endoscopic grasper. Methods: In this study endoscopic graspers are evaluated (1:through-the-scope-grasper, TTSG; 2:additional-working-channel-system AWC-S;3:external-independent-next-to-the-scope-grasper EINTS-G) regarding their working-radius, grasping abilities, maneuverability and the ability to expose tissue with varying angulation. Results: The working radius of the tools attached or within the endoscope (TTS-G and AWC-S) benefit from the steering abilities of the scope reaching 180-210 degrees in retroflexion; EINTS-G is limited to 110-degrees. The robust EINTS-grasper has the advantage of stronger grip for grasping and pulling force, which enables manipulation of larger objects. The independent maneuverability during ESD-dissection provides better tissue-exposure by changing the traction-angulation. Conclusion: The working radius of tools attached to the endoscope benefit from scope- steering. The EINTS-grasper has the advantage of stronger grasping force and pulling within the GI-tract and independent maneuverability enables improved tissue-exposure. WC200.

Artificial intelligence-based polyp size measurement in gastrointestinal endoscopy using the auxiliary waterjet as a reference.

BACKGROUND: Measurement of colorectal polyp size during endoscopy is mainly performed visually. In this work, we propose a novel polyp size measurement system (Poseidon) based on artificial intelligence (AI) using the auxiliary waterjet as a measurement reference. METHODS: Visual estimation, biopsy forceps-based estimation, and Poseidon were compared using a computed tomography colonography-based silicone model with 28 polyps of defined sizes. Four experienced gastroenterologists estimated polyp sizes visually and with biopsy forceps. Furthermore, the gastroenterologists recorded images of each polyp with the waterjet in proximity for the application of Poseidon. Additionally, Poseidon's measurements of 29 colorectal polyps during routine clinical practice were compared with visual estimates. RESULTS: In the silicone model, visual estimation had the largest median percentage error of 25.1 % (95 %CI 19.1 %-30.4 %), followed by biopsy forceps-based estimation: median 20.0 % (95 %CI 14.4 %-25.6 %). Poseidon gave a significantly lower median percentage error of 7.4 % (95 %CI 5.0 %-9.4 %) compared with other methods. During routine colonoscopies, Poseidon presented a significantly lower median percentage error (7.7 %, 95 %CI 6.1 %-9.3 %) than visual estimation (22.1 %, 95 %CI 15.1 %-26.9 %). CONCLUSION: In this work, we present a novel AI-based method for measuring colorectal polyp size with significantly higher accuracy than other common sizing methods.

A Real-Time Polyp-Detection System with Clinical Application in Colonoscopy Using Deep Convolutional Neural Networks.

Colorectal cancer (CRC) is a leading cause of cancer-related deaths worldwide. The best method to prevent CRC is with a colonoscopy. During this procedure, the gastroenterologist searches for polyps. However, there is a potential risk of polyps being missed by the gastroenterologist. Automated detection of polyps helps to assist the gastroenterologist during a colonoscopy. There are already publications examining the problem of polyp detection in the literature. Nevertheless, most of these systems are only used in the research context and are not implemented for clinical application. Therefore, we introduce the first fully open-source automated polyp-detection system scoring best on current benchmark data and implementing it ready for clinical application. To create the polyp-detection system (ENDOMIND-Advanced), we combined our own collected data from different hospitals and practices in Germany with open-source datasets to create a dataset with over 500,000 annotated images. ENDOMIND-Advanced leverages a post-processing technique based on video detection to work in real-time with a stream of images. It is integrated into a prototype ready for application in clinical interventions. We achieve better performance compared to the best system in the literature and score a F1-score of 90.24% on the open-source CVC-VideoClinicDB benchmark.

A video based benchmark data set (ENDOTEST) to evaluate computer-aided polyp detection systems.

BACKGROUND AND AIMS: Computer-aided polyp detection (CADe) may become a standard for polyp detection during colonoscopy. Several systems are already commercially available. We report on a video-based benchmark technique for the first preclinical assessment of such systems before comparative randomized trials are to be undertaken. Additionally, we compare a commercially available CADe system with our newly developed one. METHODS: ENDOTEST consisted in the combination of two datasets. The validation dataset contained 48 video-snippets with 22,856 manually annotated images of which 53.2% contained polyps. The performance dataset contained 10 full-length screening colonoscopies with 230,898 manually annotated images of which 15.8% contained a polyp. Assessment parameters were accuracy for polyp detection and time delay to first polyp detection after polyp appearance (FDT). Two CADe systems were assessed: a commercial CADe system (GI-Genius, Medtronic), and a self-developed new system (ENDOMIND). The latter being a convolutional neuronal network trained on 194,983 manually labeled images extracted from colonoscopy videos recorded in mainly six different gastroenterologic practices. RESULTS: On the ENDOTEST, both CADe systems detected all polyps in at least one image. The per-frame sensitivity and specificity in full colonoscopies was 48.1% and 93.7%, respectively for GI-Genius; and 54% and 92.7%, respectively for ENDOMIND. Median FDT of ENDOMIND with 217 ms (Inter-Quartile Range(IQR)8-1533) was significantly faster than GI-Genius with 1050 ms (IQR 358-2767, p = 0.003). CONCLUSIONS: Our benchmark ENDOTEST may be helpful for preclinical testing of new CADe devices. There seems to be a correlation between a shorter FDT with a higher sensitivity and a lower specificity for polyp detection.

Frame-by-Frame Analysis of a Commercially Available Artificial Intelligence Polyp Detection System in Full-Length Colonoscopies.

INTRODUCTION: Computer-aided detection (CADe) helps increase colonoscopic polyp detection. However, little is known about other performance metrics like the number and duration of false-positive (FP) activations or how stable the detection of a polyp is. METHODS: 111 colonoscopy videos with total 1,793,371 frames were analyzed on a frame-by-frame basis using a commercially available CADe system (GI-Genius, Medtronic Inc.). Primary endpoint was the number and duration of FP activations per colonoscopy. Additionally, we analyzed other CADe performance parameters, including per-polyp sensitivity, per-frame sensitivity, and first detection time of a polyp. We additionally investigated whether a threshold for withholding CADe activations can be set to suppress short FP activations and how this threshold alters the CADe performance parameters. RESULTS: A mean of 101 ± 88 FPs per colonoscopy were found. Most of the FPs consisted of less than three frames with a maximal 66-ms duration. The CADe system detected all 118 polyps and achieved a mean per-frame sensitivity of 46.6 ± 26.6%, with the lowest value for flat polyps (37.6 ± 24.8%). Withholding CADe detections up to 6 frames length would reduce the number of FPs by 87.97% (p < 0.001) without a significant impact on CADe performance metrics. CONCLUSIONS: The CADe system works reliable but generates many FPs as a side effect. Since most FPs are very short, withholding short-term CADe activations could substantially reduce the number of FPs without impact on other performance metrics. Clinical practice would benefit from the implementation of customizable CADe thresholds.

Pilot study of a new freely available computer-aided polyp detection system in clinical practice.

PURPOSE: Computer-aided polyp detection (CADe) systems for colonoscopy are already presented to increase adenoma detection rate (ADR) in randomized clinical trials. Those commercially available closed systems often do not allow for data collection and algorithm optimization, for example regarding the usage of different endoscopy processors. Here, we present the first clinical experiences of a, for research purposes publicly available, CADe system. METHODS: We developed an end-to-end data acquisition and polyp detection system named EndoMind. Examiners of four centers utilizing four different endoscopy processors used EndoMind during their clinical routine. Detected polyps, ADR, time to first detection of a polyp (TFD), and system usability were evaluated (NCT05006092). RESULTS: During 41 colonoscopies, EndoMind detected 29 of 29 adenomas in 66 of 66 polyps resulting in an ADR of 41.5%. Median TFD was 130 ms (95%-CI, 80-200 ms) while maintaining a median false positive rate of 2.2% (95%-CI, 1.7-2.8%). The four participating centers rated the system using the System Usability Scale with a median of 96.3 (95%-CI, 70-100). CONCLUSION: EndoMind's ability to acquire data, detect polyps in real-time, and high usability score indicate substantial practical value for research and clinical practice. Still, clinical benefit, measured by ADR, has to be determined in a prospective randomized controlled trial.

Fast machine learning annotation in the medical domain: a semi-automated video annotation tool for gastroenterologists.

BACKGROUND: Machine learning, especially deep learning, is becoming more and more relevant in research and development in the medical domain. For all the supervised deep learning applications, data is the most critical factor in securing successful implementation and sustaining the progress of the machine learning model. Especially gastroenterological data, which often involves endoscopic videos, are cumbersome to annotate. Domain experts are needed to interpret and annotate the videos. To support those domain experts, we generated a framework. With this framework, instead of annotating every frame in the video sequence, experts are just performing key annotations at the beginning and the end of sequences with pathologies, e.g., visible polyps. Subsequently, non-expert annotators supported by machine learning add the missing annotations for the frames in-between. METHODS: In our framework, an expert reviews the video and annotates a few video frames to verify the object's annotations for the non-expert. In a second step, a non-expert has visual confirmation of the given object and can annotate all following and preceding frames with AI assistance. After the expert has finished, relevant frames will be selected and passed on to an AI model. This information allows the AI model to detect and mark the desired object on all following and preceding frames with an annotation. Therefore, the non-expert can adjust and modify the AI predictions and export the results, which can then be used to train the AI model. RESULTS: Using this framework, we were able to reduce workload of domain experts on average by a factor of 20 on our data. This is primarily due to the structure of the framework, which is designed to minimize the workload of the domain expert. Pairing this framework with a state-of-the-art semi-automated AI model enhances the annotation speed further. Through a prospective study with 10 participants, we show that semi-automated annotation using our tool doubles the annotation speed of non-expert annotators compared to a well-known state-of-the-art annotation tool. CONCLUSION: In summary, we introduce a framework for fast expert annotation for gastroenterologists, which reduces the workload of the domain expert considerably while maintaining a very high annotation quality. The framework incorporates a semi-automated annotation system utilizing trained object detection models. The software and framework are open-source.

Development and evaluation of a deep learning model to improve the usability of polyp detection systems during interventions.

BACKGROUND: The efficiency of artificial intelligence as computer-aided detection (CADe) systems for colorectal polyps has been demonstrated in several randomized trials. However, CADe systems generate many distracting detections, especially during interventions such as polypectomies. Those distracting CADe detections are often induced by the introduction of snares or biopsy forceps as the systems have not been trained for such situations. In addition, there are a significant number of non-false but not relevant detections, since the polyp has already been previously detected. All these detections have the potential to disturb the examiner's work. OBJECTIVES: Development and evaluation of a convolutional neuronal network that recognizes instruments in the endoscopic image, suppresses distracting CADe detections, and reliably detects endoscopic interventions. METHODS: A total of 580 different examination videos from 9 different centers using 4 different processor types were screened for instruments and represented the training dataset (519,856 images in total, 144,217 contained a visible instrument). The test dataset included 10 full-colonoscopy videos that were analyzed for the recognition of visible instruments and detections by a commercially available CADe system (GI Genius, Medtronic). RESULTS: The test dataset contained 153,623 images, 8.84% of those presented visible instruments (12 interventions, 19 instruments used). The convolutional neuronal network reached an overall accuracy in the detection of visible instruments of 98.59%. Sensitivity and specificity were 98.55% and 98.92%, respectively. A mean of 462.8 frames containing distracting CADe detections per colonoscopy were avoided using the convolutional neuronal network. This accounted for 95.6% of all distracting CADe detections. CONCLUSIONS: Detection of endoscopic instruments in colonoscopy using artificial intelligence technology is reliable and achieves high sensitivity and specificity. Accordingly, the new convolutional neuronal network could be used to reduce distracting CADe detections during endoscopic procedures. Thus, our study demonstrates the great potential of artificial intelligence technology beyond mucosal assessment.

The influence of computer-aided polyp detection systems on reaction time for polyp detection and eye gaze.

BACKGROUND: Multiple computer-aided systems for polyp detection (CADe) have been introduced into clinical practice, with an unclear effect on examiner behavior. This study aimed to measure the influence of a CADe system on reaction time, mucosa misinterpretation, and changes in visual gaze pattern. METHODS: Participants with variable levels of colonoscopy experience viewed video sequences (n = 29) while eye movement was tracked. Using a crossover design, videos were presented in two assessments, with and without CADe support. Reaction time for polyp detection and eye-tracking metrics were evaluated. RESULTS: 21 participants performed 1218 experiments. CADe was significantly faster in detecting polyps compared with participants (median 1.16 seconds [99 %CI 0.40-3.43] vs. 2.97 seconds [99 %CI 2.53-3.77], respectively). However, the reaction time of participants when using CADe (median 2.90 seconds [99 %CI 2.55-3.38]) was similar to that without CADe. CADe increased misinterpretation of normal mucosa and reduced the eye travel distance. CONCLUSIONS: Results confirm that CADe systems detect polyps faster than humans. However, use of CADe did not improve human reaction times. It increased misinterpretation of normal mucosa and decreased the eye travel distance. Possible consequences of these findings might be prolonged examination time and deskilling.

New concept for colonoscopy including side optics and artificial intelligence.

BACKGROUND AND AIMS: Adenoma detection rate is the crucial parameter for colorectal cancer screening. Increasing the field of view with additional side optics has been reported to detect flat adenomas hidden behind folds. Furthermore, artificial intelligence (AI) has also recently been introduced to detect more adenomas. We therefore aimed to combine both technologies in a new prototypic colonoscopy concept. METHODS: A 3-dimensional-printed cap including 2 microcameras was attached to a conventional endoscope. The prototype was applied in 8 gene-targeted pigs with mutations in the adenomatous polyposis coli gene. The first 4 animals were used to train an AI system based on the images generated by microcameras. Thereafter, the conceptual prototype for detecting adenomas was tested in a further series of 4 pigs. RESULTS: Using our prototype, we detected, with side optics, adenomas that might have been missed conventionally. Furthermore, the newly developed AI could detect, mark, and present adenomas visualized with side optics outside of the conventional field of view. CONCLUSIONS: Combining AI with side optics might help detect adenomas that otherwise might have been missed.

Current status and limitations of artificial intelligence in colonoscopy.

BACKGROUND: Artificial intelligence (AI) using deep learning methods for polyp detection (CADe) and characterization (CADx) is on the verge of clinical application. CADe already implied its potential use in randomized controlled trials. Further efforts are needed to take CADx to the next level of development. AIM: This work aims to give an overview of the current status of AI in colonoscopy, without going into too much technical detail. METHODS: A literature search to identify important studies exploring the use of AI in colonoscopy was performed. RESULTS: This review focuses on AI performance in screening colonoscopy summarizing the first prospective trials for CADe, the state of research in CADx as well as current limitations of those systems and legal issues.