The effect of incorporating domain knowledge with deep learning in identifying benign and malignant gastric whitish lesions: A retrospective study.

BACKGROUND AND AIM: Early whitish gastric neoplasms can be easily misdiagnosed; differential diagnosis of gastric whitish lesions remains a challenge. We aim to build a deep learning (DL) model to diagnose whitish gastric neoplasms and explore the effect of adding domain knowledge in model construction. METHODS: We collected 4558 images from two institutions to train and test models. We first developed two sole DL models (1 and 2) using supervised and semi-supervised algorithms. Then we selected diagnosis-related features through literature research and developed feature-extraction models to determine features including boundary, surface, roundness, depression, and location. Then predictions of the five feature-extraction models and sole DL model were combined and inputted into seven machine-learning (ML) based fitting-diagnosis models. The optimal model was selected as ENDOANGEL-WD (whitish-diagnosis) and compared with endoscopists. RESULTS: Sole DL 2 had higher sensitivity (83.12% vs 68.67%, Bonferroni adjusted P = 0.024) than sole DL 1. Adding domain knowledge, the decision tree performed best among the seven ML models, achieving higher specificity than DL 1 (84.38% vs 72.27%, Bonferroni adjusted P < 0.05) and higher accuracy than DL 2 (80.47%, Bonferroni adjusted P < 0.001) and was selected as ENDOANGEL-WD. ENDOANGEL-WD showed better accuracy compared with 10 endoscopists (75.70%, P < 0.001). CONCLUSIONS: We developed a novel system ENDOANGEL-WD combining domain knowledge and traditional DL to detect gastric whitish neoplasms. Adding domain knowledge improved the performance of traditional DL, which provided a novel solution for establishing diagnostic models for other rare diseases potentially.

An artificial intelligence system for chronic atrophic gastritis diagnosis and risk stratification under white light endoscopy.

BACKGROUND AND AIMS: The diagnosis and stratification of gastric atrophy (GA) predict patients' gastric cancer progression risk and determine endoscopy surveillance interval. We aimed to construct an artificial intelligence (AI) system for GA endoscopic identification and risk stratification based on the Kimura-Takemoto classification. METHODS: We constructed the system using two trained models and verified its performance. First, we retrospectively collected 869 images and 119 videos to compare its performance with that of endoscopists in identifying GA. Then, we included original image cases of 102 patients to validate the system for stratifying GA and comparing it with endoscopists with different experiences. RESULTS: The sensitivity of model 1 was higher than that of endoscopists (92.72% vs. 76.85 %) at image level and also higher than that of experts (94.87% vs. 85.90 %) at video level. The system outperformed experts in stratifying GA (overall accuracy: 81.37 %, 73.04 %, p = 0.045). The accuracy of this system in classifying non-GA, mild GA, moderate GA, and severe GA was 80.00 %, 77.42 %, 83.33 %, and 85.71 %, comparable to that of experts and better than that of seniors and novices. CONCLUSIONS: We established an expert-level system for GA endoscopic identification and risk stratification. It has great potential for endoscopic assessment and surveillance determinations.

Development and Evaluation of a Surveillance System for Follow-Up After Colorectal Polypectomy.

Importance: The adherence of physicians and patients to published colorectal postpolypectomy surveillance guidelines varies greatly, and patient follow-up is critical but time consuming. Objectives: To evaluate the accuracy of an automatic surveillance (AS) system in identifying patients after polypectomy, assigning surveillance intervals for different risks of patients, and proactively following up with patients on time. Design, Setting, and Participants: In this diagnostic/prognostic study, endoscopic and pathological reports of 47 544 patients undergoing colonoscopy at 3 hospitals between January 1, 2017, and June 30, 2022, were collected to develop an AS system based on natural language processing. The performance of the AS system was fully evaluated in internal and external tests according to 5 guidelines worldwide and compared with that of physicians. A multireader, multicase (MRMC) trial was conducted to evaluate use of the AS system and physician guideline adherence, and prospective data were collected to evaluate the success rate in contacting patients and the association with reduced human workload. Data analysis was conducted from July to September 2022. Exposures: Assistance of the AS system. Main Outcomes and Measures: The accuracy of the system in identifying patients after polypectomy, stratifying patient risk levels, and assigning surveillance intervals in internal (Renmin Hospital of Wuhan University), external 1 (Wenzhou Central Hospital), and external 2 (The First People's Hospital of Yichang) test sets; the accuracy of physicians and their time burden with and without system assistance; and the rate of successfully informed patients of the system were evaluated. Results: Test sets for 16 106 patients undergoing colonoscopy (mean [SD] age, 51.90 [13.40] years; 7690 females [47.75%]) were evaluated. In internal, external 1, and external 2 test sets, the system had an overall accuracy of 99.91% (95% CI, 99.83%-99.95%), 99.54% (95% CI, 99.30%-99.70%), and 99.77% (95% CI, 99.41%-99.91%), respectively, for identifying types of patients and achieved an overall accuracy of at least 99.30% (95% CI, 98.67%-99.63%) in the internal test set, 98.89% (95% CI, 98.33%-99.27%) in external test set 1, and 98.56% (95% CI, 95.86%-99.51%) in external test set 2 for stratifying patient risk levels and assigning surveillance intervals according to 5 guidelines. The system was associated with increased mean (SD) accuracy among physicians vs no AS system in 105 patients (98.67% [1.28%] vs 78.10% [18.01%]; P = .04) in the MRMC trial. In a prospective trial, the AS system successfully informed 82 of 88 patients (93.18%) and was associated with reduced burden of follow-up time vs no AS system (0 vs 2.86 h). Conclusions and Relevance: This study found that an AS system was associated with improved adherence to guidelines among physicians and reduced workload among physicians and nurses.

Exploring the challenge of early gastric cancer diagnostic AI system face in multiple centers and its potential solutions.

BACKGROUND: Artificial intelligence (AI) performed variously among test sets with different diversity due to sample selection bias, which can be stumbling block for AI applications. We previously tested AI named ENDOANGEL, diagnosing early gastric cancer (EGC) on single-center videos in man-machine competition. We aimed to re-test ENDOANGEL on multi-center videos to explore challenges applying AI in multiple centers, then upgrade ENDOANGEL and explore solutions to the challenge. METHODS: ENDOANGEL was re-tested on multi-center videos retrospectively collected from 12 institutions and compared with performance in previously reported single-center videos. We then upgraded ENDOANGEL to ENDOANGEL-2022 with more training samples and novel algorithms and conducted competition between ENDOANGEL-2022 and endoscopists. ENDOANGEL-2022 was then tested on single-center videos and compared with performance in multi-center videos; the two AI systems were also compared with each other and endoscopists. RESULTS: Forty-six EGCs and 54 non-cancers were included in multi-center video cohort. On diagnosing EGCs, compared with single-center videos, ENDOANGEL showed stable sensitivity (97.83% vs. 100.00%) while sharply decreased specificity (61.11% vs. 82.54%); ENDOANGEL-2022 showed similar tendency while achieving significantly higher specificity (79.63%, p < 0.01) making fewer mistakes on typical lesions than ENDOANGEL. On detecting gastric neoplasms, both AI showed stable sensitivity while sharply decreased specificity. Nevertheless, both AI outperformed endoscopists in the two competitions. CONCLUSIONS: Great increase of false positives is a prominent challenge for applying EGC diagnostic AI in multiple centers due to high heterogeneity of negative cases. Optimizing AI by adding samples and using novel algorithms is promising to overcome this challenge.

The effectiveness of a computer-aided system in improving the detection rate of gastric neoplasm and early gastric cancer: study protocol for a multi-centre, randomized controlled trial.

BACKGROUND: This protocol is for a multi-centre randomised controlled trial to determine whether the computer-aided system ENDOANGEL-GC improves the detection rates of gastric neoplasms and early gastric cancer (EGC) in routine oesophagogastroduodenoscopy (EGD). METHODS: Study design: Prospective, single-blind, parallel-group, multi-centre randomised controlled trial. SETTINGS: The computer-aided system ENDOANGEL-GC was used to monitor blind spots, detect gastric abnormalities, and identify gastric neoplasms during EGD. PARTICIPANTS: Adults who underwent screening, diagnosis, or surveillance EGD. Randomisation groups: 1. Experiment group, EGD examinations with the assistance of the ENDOANGEL-GC; 2. Control group, EGD examinations without the assistance of the ENDOANGEL-GC. RANDOMISATION: Block randomisation, stratified by centre. PRIMARY OUTCOMES: Detection rates of gastric neoplasms and EGC. SECONDARY OUTCOMES: Detection rate of premalignant gastric lesions, biopsy rate, observation time, and number of blind spots on EGD. BLINDING: Outcomes are undertaken by blinded assessors. SAMPLE SIZE: Based on the previously published findings and our pilot study, the detection rate of gastric neoplasms in the control group is estimated to be 2.5%, and that of the experimental group is expected to be 4.0%. With a two-sided α level of 0.05 and power of 80%, allowing for a 10% drop-out rate, the sample size is calculated as 4858. The detection rate of EGC in the control group is estimated to be 20%, and that of the experiment group is expected to be 35%. With a two-sided α level of 0.05 and power of 80%, a total of 270 cases of gastric cancer are needed. Assuming the proportion of gastric cancer to be 1% in patients undergoing EGD and allowing for a 10% dropout rate, the sample size is calculated as 30,000. Considering the larger sample size calculated from the two primary endpoints, the required sample size is determined to be 30,000. DISCUSSION: The results of this trial will help determine the effectiveness of the ENDOANGEL-GC in clinical settings. TRIAL REGISTRATION: ChiCTR (Chinese Clinical Trial Registry), ChiCTR2100054449, registered 17 December 2021.

Explainable artificial intelligence incorporated with domain knowledge diagnosing early gastric neoplasms under white light endoscopy.

White light endoscopy is the most pivotal tool for detecting early gastric neoplasms. Previous artificial intelligence (AI) systems were primarily unexplainable, affecting their clinical credibility and acceptability. We aimed to develop an explainable AI named ENDOANGEL-ED (explainable diagnosis) to solve this problem. A total of 4482 images and 296 videos with focal lesions from 3279 patients from eight hospitals were used for training, validating, and testing ENDOANGEL-ED. A traditional sole deep learning (DL) model was trained using the same dataset. The performance of ENDOANGEL-ED and sole DL was evaluated on six levels: internal and external images, internal and external videos, consecutive videos, and man-machine comparison with 77 endoscopists in videos. Furthermore, a multi-reader, multi-case study was conducted to evaluate the ENDOANGEL-ED's effectiveness. A scale was used to compare the overall acceptance of endoscopists to traditional and explainable AI systems. The ENDOANGEL-ED showed high performance in the image and video tests. In man-machine comparison, the accuracy of ENDOANGEL-ED was significantly higher than that of all endoscopists in internal (81.10% vs. 70.61%, p < 0.001) and external videos (88.24% vs. 78.49%, p < 0.001). With ENDOANGEL-ED's assistance, the accuracy of endoscopists significantly improved (70.61% vs. 79.63%, p < 0.001). Compared with the traditional AI, the explainable AI increased the endoscopists' trust and acceptance (4.42 vs. 3.74, p < 0.001; 4.52 vs. 4.00, p < 0.001). In conclusion, we developed a real-time explainable AI that showed high performance, higher clinical credibility, and acceptance than traditional DL models and greatly improved the diagnostic ability of endoscopists.

A deep-learning based system using multi-modal data for diagnosing gastric neoplasms in real-time (with video).

BACKGROUND: White light (WL) and weak-magnifying (WM) endoscopy are both important methods for diagnosing gastric neoplasms. This study constructed a deep-learning system named ENDOANGEL-MM (multi-modal) aimed at real-time diagnosing gastric neoplasms using WL and WM data. METHODS: WL and WM images of a same lesion were combined into image-pairs. A total of 4201 images, 7436 image-pairs, and 162 videos were used for model construction and validation. Models 1-5 including two single-modal models (WL, WM) and three multi-modal models (data fusion on task-level, feature-level, and input-level) were constructed. The models were tested on three levels including images, videos, and prospective patients. The best model was selected for constructing ENDOANGEL-MM. We compared the performance between the models and endoscopists and conducted a diagnostic study to explore the ENDOANGEL-MM's assistance ability. RESULTS: Model 4 (ENDOANGEL-MM) showed the best performance among five models. Model 2 performed better in single-modal models. The accuracy of ENDOANGEL-MM was higher than that of Model 2 in still images, real-time videos, and prospective patients. (86.54 vs 78.85%, P = 0.134; 90.00 vs 85.00%, P = 0.179; 93.55 vs 70.97%, P < 0.001). Model 2 and ENDOANGEL-MM outperformed endoscopists on WM data (85.00 vs 71.67%, P = 0.002) and multi-modal data (90.00 vs 76.17%, P = 0.002), significantly. With the assistance of ENDOANGEL-MM, the accuracy of non-experts improved significantly (85.75 vs 70.75%, P = 0.020), and performed no significant difference from experts (85.75 vs 89.00%, P = 0.159). CONCLUSIONS: The multi-modal model constructed by feature-level fusion showed the best performance. ENDOANGEL-MM identified gastric neoplasms with good accuracy and has a potential role in real-clinic.