ABSTRACT
Pre-training deep learning models with large data sets of natural images, such as ImageNet, has become the standard for endoscopic image analysis. This approach is generally superior to training from scratch, due to the scarcity of high-quality medical imagery and labels. However, it is still unknown whether the learned features on natural imagery provide an optimal starting point for the downstream medical endoscopic imaging tasks. Intuitively, pre-training with imagery closer to the target domain could lead to better-suited feature representations. This study evaluates whether leveraging in-domain pre-training in gastrointestinal endoscopic image analysis has potential benefits compared to pre-training on natural images. To this end, we present a dataset comprising of 5,014,174 gastrointestinal endoscopic images from eight different medical centers (GastroNet-5M), and exploit self-supervised learning with SimCLRv2, MoCov2 and DINO to learn relevant features for in-domain downstream tasks. The learned features are compared to features learned on natural images derived with multiple methods, and variable amounts of data and/or labels (e.g. Billion-scale semi-weakly supervised learning and supervised learning on ImageNet-21k). The effects of the evaluation is performed on five downstream data sets, particularly designed for a variety of gastrointestinal tasks, for example, GIANA for angiodyplsia detection and Kvasir-SEG for polyp segmentation. The findings indicate that self-supervised domain-specific pre-training, specifically using the DINO framework, results into better performing models compared to any supervised pre-training on natural images. On the ResNet50 and Vision-Transformer-small architectures, utilizing self-supervised in-domain pre-training with DINO leads to an average performance boost of 1.63% and 4.62%, respectively, on the downstream datasets. This improvement is measured against the best performance achieved through pre-training on natural images within any of the evaluated frameworks. Moreover, the in-domain pre-trained models also exhibit increased robustness against distortion perturbations (noise, contrast, blur, etc.), where the in-domain pre-trained ResNet50 and Vision-Transformer-small with DINO achieved on average 1.28% and 3.55% higher on the performance metrics, compared to the best performance found for pre-trained models on natural images. Overall, this study highlights the importance of in-domain pre-training for improving the generic nature, scalability and performance of deep learning for medical image analysis. The GastroNet-5M pre-trained weights are made publicly available in our repository: huggingface.co/tgwboers/GastroNet-5M_Pretrained_Weights.
ABSTRACT
BACKGROUND AND AIMS: Characterization of visible abnormalities in patients with Barrett's esophagus (BE) can be challenging, especially for inexperienced endoscopists. This results in suboptimal diagnostic accuracy and poor interobserver agreement. Computer-aided diagnosis (CADx) systems may assist endoscopists. We aimed to develop, validate, and benchmark a CADx system for BE neoplasia. METHODS: The CADx system received pretraining with ImageNet and then consecutive domain-specific pretraining with GastroNet, which includes 5 million endoscopic images. It was subsequently trained and internally validated using 1758 narrow-band imaging (NBI) images of early BE neoplasia (352 patients) and 1838 NBI images of nondysplastic BE (173 patients) from 8 international centers. CADx was tested prospectively on corresponding image and video test sets with 30 cases (20 patients) of BE neoplasia and 60 cases (31 patients) of nondysplastic BE. The test set was benchmarked by 44 general endoscopists in 2 phases (phase 1, no CADx assistance; phase 2, with CADx assistance). Ten international BE experts provided additional benchmark performance. RESULTS: Stand-alone sensitivity and specificity of the CADx system were 100% and 98% for images and 93% and 96% for videos, respectively. CADx outperformed general endoscopists without CADx assistance in terms of sensitivity (P = .04). Sensitivity and specificity of general endoscopists increased from 84% to 96% and 90% to 98% with CAD assistance (P < .001). CADx assistance increased endoscopists' confidence in characterization (P < .001). CADx performance was similar to that of the BE experts. CONCLUSIONS: CADx assistance significantly increased characterization performance of BE neoplasia by general endoscopists to the level of expert endoscopists. The use of this CADx system may thereby improve daily Barrett surveillance.
ABSTRACT
BACKGROUND AND AIMS: This pilot study evaluated the performance of a recently developed computer-aided detection (CADe) system for Barrett's neoplasia during live endoscopic procedures. METHODS: Fifteen patients with a visible lesion and 15 without were included in this study. A CAD-assisted workflow was used that included a slow pullback video recording of the entire Barrett's segment with live CADe assistance, followed by CADe-assisted level-based video recordings every 2 cm of the Barrett's segment. Outcomes were per-patient and per-level diagnostic accuracy of the CAD-assisted workflow, in which the primary outcome was per-patient in vivo CADe sensitivity. RESULTS: In the per-patient analyses, the CADe system detected all visible lesions (sensitivity 100%). Per-patient CADe specificity was 53%. Per-level sensitivity and specificity of the CADe-assisted workflow were 100% and 73%, respectively. CONCLUSIONS: In this pilot study, detection by the CADe system of all potentially neoplastic lesions in Barrett's esophagus was comparable to that of an expert endoscopist. Continued refinement of the system may improve specificity. External validation in larger multicenter studies is planned. (Clinical trial registration number: NCT05628441.).
Subject(s)
Barrett Esophagus , Diagnosis, Computer-Assisted , Esophageal Neoplasms , Esophagoscopy , Aged , Aged, 80 and over , Female , Humans , Male , Middle Aged , Adenocarcinoma/pathology , Barrett Esophagus/pathology , Barrett Esophagus/surgery , Esophageal Neoplasms/pathology , Esophagoscopy/methods , Pilot Projects , Sensitivity and Specificity , Video RecordingABSTRACT
INTRODUCTION: Endoscopic detection of early neoplasia in Barrett's esophagus is difficult. Computer Aided Detection (CADe) systems may assist in neoplasia detection. The aim of this study was to report the first steps in the development of a CADe system for Barrett's neoplasia and to evaluate its performance when compared with endoscopists. METHODS: This CADe system was developed by a consortium, consisting of the Amsterdam University Medical Center, Eindhoven University of Technology, and 15 international hospitals. After pretraining, the system was trained and validated using 1.713 neoplastic (564 patients) and 2.707 non-dysplastic Barrett's esophagus (NDBE; 665 patients) images. Neoplastic lesions were delineated by 14 experts. The performance of the CADe system was tested on three independent test sets. Test set 1 (50 neoplastic and 150 NDBE images) contained subtle neoplastic lesions representing challenging cases and was benchmarked by 52 general endoscopists. Test set 2 (50 neoplastic and 50 NDBE images) contained a heterogeneous case-mix of neoplastic lesions, representing distribution in clinical practice. Test set 3 (50 neoplastic and 150 NDBE images) contained prospectively collected imagery. The main outcome was correct classification of the images in terms of sensitivity. RESULTS: The sensitivity of the CADe system on test set 1 was 84%. For general endoscopists, sensitivity was 63%, corresponding to a neoplasia miss-rate of one-third of neoplastic lesions and a potential relative increase in neoplasia detection of 33% for CADe-assisted detection. The sensitivity of the CADe system on test sets 2 and 3 was 100% and 88%, respectively. The specificity of the CADe system varied for the three test sets between 64% and 66%. CONCLUSION: This study describes the first steps towards the establishment of an unprecedented data infrastructure for using machine learning to improve the endoscopic detection of Barrett's neoplasia. The CADe system detected neoplasia reliably and outperformed a large group of endoscopists in terms of sensitivity.
Subject(s)
Barrett Esophagus , Deep Learning , Esophageal Neoplasms , Humans , Barrett Esophagus/diagnosis , Barrett Esophagus/pathology , Esophageal Neoplasms/diagnosis , Esophageal Neoplasms/pathology , Esophagoscopy/methods , Retrospective Studies , Sensitivity and SpecificityABSTRACT
Patients with Barrett's esophagus are at an increased risk to develop esophageal cancer and, therefore, undergo regular endoscopic surveillance. Early detection of neoplasia enables endoscopic treatment, which improves outcomes. However, early Barrett's neoplasia is easily missed during endoscopic surveillance. This study investigates multidiameter single fiber reflectance spectroscopy (MDSFR) to improve Barrett's surveillance. Based on the concept of field cancerization, it may be possible to identify the presence of a neoplastic lesion from measurements elsewhere in the esophagus or even the oral cavity. In this study, MDSFR measurements are performed on non-dysplastic Barrett's mucosa, squamous mucosa, oral mucosa, and the neoplastic lesion (if present). Based on logistic regression analysis on the scattering parameters measured by MDSFR, a classifier is developed that can predict the presence of neoplasia elsewhere in the Barrett's segment from measurements on the non-dysplastic Barrett's mucosa (sensitivity 91%, specificity 71%, AUC = 0.77). Classifiers obtained from logistic regression analysis for the squamous and oral mucosa do not result in an AUC significantly different from 0.5.
Subject(s)
Barrett Esophagus , Esophageal Neoplasms , Barrett Esophagus/diagnostic imaging , Esophageal Neoplasms/diagnostic imaging , Esophagoscopy , Humans , Spectrum AnalysisABSTRACT
BACKGROUND AND AIMS: Endoscopic recognition of early Barrett's neoplasia is challenging. Blue-light imaging (BLI) and linked-color imaging (LCI) may assist endoscopists in appreciation of neoplasia. Our aim was to evaluate BLI and LCI for visualization of Barrett's neoplasia in comparison with white-light endoscopy (WLE) alone, when assessed by nonexpert endoscopists. METHODS: In this web-based assessment, corresponding WLE, BLI, and LCI images of 30 neoplastic Barrett's lesions were delineated by 3 expert endoscopists to establish ground truth. These images were then scored and delineated by 76 nonexpert endoscopists from 3 countries and with different levels of expertise, in 4 separate assessment phases with a washout period of 2 weeks. Assessments were as follows: assessment 1, WLE only; assessment 2, WLE + BLI; assessment 3, WLE + LCI; assessment 4, WLE + BLI + LCI. The outcomes were (1) appreciation of macroscopic appearance and ability to delineate lesions (visual analog scale [VAS] scores); (2) preferred technique (ordinal scores); and (3) assessors' delineation performance in terms of overlap with expert ground truth. RESULTS: Median VAS scores for phases 2 to 4 were significantly higher than in phase 1 (P < .001). Assessors preferred BLI and LCI over WLE for appreciation of macroscopic appearance (P < .001) and delineation (P < .001). Linear mixed-effect models showed that delineation performance increased significantly in phase 4. CONCLUSIONS: The use of BLI and LCI has significant additional value for the visualization of Barrett's neoplasia when used by nonexpert endoscopists. Assessors appreciated the addition of BLI and LCI better than the use of WLE alone. Furthermore, this addition led to improved delineation performance, thereby allowing for better acquisition of targeted biopsy samples. (The Netherlands Trial Registry number: NL7541.).
Subject(s)
Barrett Esophagus , Esophageal Neoplasms , Barrett Esophagus/diagnostic imaging , Color , Esophageal Neoplasms/diagnostic imaging , Esophagoscopy , Humans , Light , NetherlandsABSTRACT
BACKGROUND AND AIMS: We assessed the preliminary diagnostic accuracy of a recently developed computer-aided detection (CAD) system for detection of Barrett's neoplasia during live endoscopic procedures. METHODS: The CAD system was tested during endoscopic procedures in 10 patients with nondysplastic Barrett's esophagus (NDBE) and 10 patients with confirmed Barrett's neoplasia. White-light endoscopy images were obtained at every 2-cm level of the Barrett's segment and immediately analyzed by the CAD system, providing instant feedback to the endoscopist. At every level, 3 images were evaluated by the CAD system. Outcome measures were diagnostic performance of the CAD system per level and per patient, defined as accuracy, sensitivity, and specificity (ground truth was established by expert assessment and corresponding histopathology), and concordance of 3 sequential CAD predictions per level. RESULTS: Accuracy, sensitivity, and specificity of the CAD system in a per-level analyses were 90%, 91%, and 89%, respectively. Nine of 10 neoplastic patients were correctly diagnosed. The single lesion not detected by CAD showed NDBE in the endoscopic resection specimen. In only 1 NDBE patient, the CAD system produced false-positive predictions. In 75% of all levels, the CAD system produced 3 concordant predictions. CONCLUSIONS: This is one of the first studies to evaluate a CAD system for Barrett's neoplasia during live endoscopic procedures. The system detected neoplasia with high accuracy, with only a small number of false-positive predictions and with a high concordance rate between separate predictions. The CAD system is thereby ready for testing in larger, multicenter trials. (Clinical trial registration number: NL7544.).
Subject(s)
Barrett Esophagus , Deep Learning , Esophageal Neoplasms , Barrett Esophagus/diagnostic imaging , Esophageal Neoplasms/diagnostic imaging , Esophagoscopy , Humans , Video RecordingABSTRACT
BACKGROUND & AIMS: We aimed to develop and validate a deep-learning computer-aided detection (CAD) system, suitable for use in real time in clinical practice, to improve endoscopic detection of early neoplasia in patients with Barrett's esophagus (BE). METHODS: We developed a hybrid ResNet-UNet model CAD system using 5 independent endoscopy data sets. We performed pretraining using 494,364 labeled endoscopic images collected from all intestinal segments. Then, we used 1704 unique esophageal high-resolution images of rigorously confirmed early-stage neoplasia in BE and nondysplastic BE, derived from 669 patients. System performance was assessed by using data sets 4 and 5. Data set 5 was also scored by 53 general endoscopists with a wide range of experience from 4 countries to benchmark CAD system performance. Coupled with histopathology findings, scoring of images that contained early-stage neoplasia in data sets 2-5 were delineated in detail for neoplasm position and extent by multiple experts whose evaluations served as the ground truth for segmentation. RESULTS: The CAD system classified images as containing neoplasms or nondysplastic BE with 89% accuracy, 90% sensitivity, and 88% specificity (data set 4, 80 patients and images). In data set 5 (80 patients and images) values for the CAD system vs those of the general endoscopists were 88% vs 73% accuracy, 93% vs 72% sensitivity, and 83% vs 74% specificity. The CAD system achieved higher accuracy than any of the individual 53 nonexpert endoscopists, with comparable delineation performance. CAD delineations of the area of neoplasm overlapped with those from the BE experts in all detected neoplasia in data sets 4 and 5. The CAD system identified the optimal site for biopsy of detected neoplasia in 97% and 92% of cases (data sets 4 and 5, respectively). CONCLUSIONS: We developed, validated, and benchmarked a deep-learning computer-aided system for primary detection of neoplasia in patients with BE. The system detected neoplasia with high accuracy and near-perfect delineation performance. The Netherlands National Trials Registry, Number: NTR7072.