RESUMO
Pre-training deep learning models with large data sets of natural images, such as ImageNet, has become the standard for endoscopic image analysis. This approach is generally superior to training from scratch, due to the scarcity of high-quality medical imagery and labels. However, it is still unknown whether the learned features on natural imagery provide an optimal starting point for the downstream medical endoscopic imaging tasks. Intuitively, pre-training with imagery closer to the target domain could lead to better-suited feature representations. This study evaluates whether leveraging in-domain pre-training in gastrointestinal endoscopic image analysis has potential benefits compared to pre-training on natural images. To this end, we present a dataset comprising of 5,014,174 gastrointestinal endoscopic images from eight different medical centers (GastroNet-5M), and exploit self-supervised learning with SimCLRv2, MoCov2 and DINO to learn relevant features for in-domain downstream tasks. The learned features are compared to features learned on natural images derived with multiple methods, and variable amounts of data and/or labels (e.g. Billion-scale semi-weakly supervised learning and supervised learning on ImageNet-21k). The effects of the evaluation is performed on five downstream data sets, particularly designed for a variety of gastrointestinal tasks, for example, GIANA for angiodyplsia detection and Kvasir-SEG for polyp segmentation. The findings indicate that self-supervised domain-specific pre-training, specifically using the DINO framework, results into better performing models compared to any supervised pre-training on natural images. On the ResNet50 and Vision-Transformer-small architectures, utilizing self-supervised in-domain pre-training with DINO leads to an average performance boost of 1.63% and 4.62%, respectively, on the downstream datasets. This improvement is measured against the best performance achieved through pre-training on natural images within any of the evaluated frameworks. Moreover, the in-domain pre-trained models also exhibit increased robustness against distortion perturbations (noise, contrast, blur, etc.), where the in-domain pre-trained ResNet50 and Vision-Transformer-small with DINO achieved on average 1.28% and 3.55% higher on the performance metrics, compared to the best performance found for pre-trained models on natural images. Overall, this study highlights the importance of in-domain pre-training for improving the generic nature, scalability and performance of deep learning for medical image analysis. The GastroNet-5M pre-trained weights are made publicly available in our repository: huggingface.co/tgwboers/GastroNet-5M_Pretrained_Weights.
RESUMO
BACKGROUND AND AIMS: This pilot study evaluated the performance of a recently developed computer-aided detection (CADe) system for Barrett's neoplasia during live endoscopic procedures. METHODS: Fifteen patients with a visible lesion and 15 without were included in this study. A CAD-assisted workflow was used that included a slow pullback video recording of the entire Barrett's segment with live CADe assistance, followed by CADe-assisted level-based video recordings every 2 cm of the Barrett's segment. Outcomes were per-patient and per-level diagnostic accuracy of the CAD-assisted workflow, in which the primary outcome was per-patient in vivo CADe sensitivity. RESULTS: In the per-patient analyses, the CADe system detected all visible lesions (sensitivity 100%). Per-patient CADe specificity was 53%. Per-level sensitivity and specificity of the CADe-assisted workflow were 100% and 73%, respectively. CONCLUSIONS: In this pilot study, detection by the CADe system of all potentially neoplastic lesions in Barrett's esophagus was comparable to that of an expert endoscopist. Continued refinement of the system may improve specificity. External validation in larger multicenter studies is planned. (Clinical trial registration number: NCT05628441.).
Assuntos
Esôfago de Barrett , Diagnóstico por Computador , Neoplasias Esofágicas , Esofagoscopia , Idoso , Idoso de 80 Anos ou mais , Feminino , Humanos , Masculino , Pessoa de Meia-Idade , Adenocarcinoma/patologia , Esôfago de Barrett/patologia , Esôfago de Barrett/cirurgia , Neoplasias Esofágicas/patologia , Esofagoscopia/métodos , Projetos Piloto , Sensibilidade e Especificidade , Gravação em VídeoRESUMO
BACKGROUND AND AIMS: Characterization of visible abnormalities in patients with Barrett's esophagus (BE) can be challenging, especially for inexperienced endoscopists. This results in suboptimal diagnostic accuracy and poor interobserver agreement. Computer-aided diagnosis (CADx) systems may assist endoscopists. We aimed to develop, validate, and benchmark a CADx system for BE neoplasia. METHODS: The CADx system received pretraining with ImageNet and then consecutive domain-specific pretraining with GastroNet, which includes 5 million endoscopic images. It was subsequently trained and internally validated using 1758 narrow-band imaging (NBI) images of early BE neoplasia (352 patients) and 1838 NBI images of nondysplastic BE (173 patients) from 8 international centers. CADx was tested prospectively on corresponding image and video test sets with 30 cases (20 patients) of BE neoplasia and 60 cases (31 patients) of nondysplastic BE. The test set was benchmarked by 44 general endoscopists in 2 phases (phase 1, no CADx assistance; phase 2, with CADx assistance). Ten international BE experts provided additional benchmark performance. RESULTS: Stand-alone sensitivity and specificity of the CADx system were 100% and 98% for images and 93% and 96% for videos, respectively. CADx outperformed general endoscopists without CADx assistance in terms of sensitivity (P = .04). Sensitivity and specificity of general endoscopists increased from 84% to 96% and 90% to 98% with CAD assistance (P < .001). CADx assistance increased endoscopists' confidence in characterization (P < .001). CADx performance was similar to that of the BE experts. CONCLUSIONS: CADx assistance significantly increased characterization performance of BE neoplasia by general endoscopists to the level of expert endoscopists. The use of this CADx system may thereby improve daily Barrett surveillance.
RESUMO
Computer-aided detection and diagnosis systems (CADe/CADx) in endoscopy are commonly trained using high-quality imagery, which is not representative for the heterogeneous input typically encountered in clinical practice. In endoscopy, the image quality heavily relies on both the skills and experience of the endoscopist and the specifications of the system used for screening. Factors such as poor illumination, motion blur, and specific post-processing settings can significantly alter the quality and general appearance of these images. This so-called domain gap between the data used for developing the system and the data it encounters after deployment, and the impact it has on the performance of deep neural networks (DNNs) supportive endoscopic CAD systems remains largely unexplored. As many of such systems, for e.g. polyp detection, are already being rolled out in clinical practice, this poses severe patient risks in particularly community hospitals, where both the imaging equipment and experience are subject to considerable variation. Therefore, this study aims to evaluate the impact of this domain gap on the clinical performance of CADe/CADx for various endoscopic applications. For this, we leverage two publicly available data sets (KVASIR-SEG and GIANA) and two in-house data sets. We investigate the performance of commonly-used DNN architectures under synthetic, clinically calibrated image degradations and on a prospectively collected dataset including 342 endoscopic images of lower subjective quality. Additionally, we assess the influence of DNN architecture and complexity, data augmentation, and pretraining techniques for improved robustness. The results reveal a considerable decline in performance of 11.6% (±1.5) as compared to the reference, within the clinically calibrated boundaries of image degradations. Nevertheless, employing more advanced DNN architectures and self-supervised in-domain pre-training effectively mitigate this drop to 7.7% (±2.03). Additionally, these enhancements yield the highest performance on the manually collected test set including images with lower subjective quality. By comprehensively assessing the robustness of popular DNN architectures and training strategies across multiple datasets, this study provides valuable insights into their performance and limitations for endoscopic applications. The findings highlight the importance of including robustness evaluation when developing DNNs for endoscopy applications and propose strategies to mitigate performance loss.
Assuntos
Diagnóstico por Computador , Redes Neurais de Computação , Humanos , Diagnóstico por Computador/métodos , Endoscopia Gastrointestinal , Processamento de Imagem Assistida por Computador/métodosRESUMO
INTRODUCTION: Endoscopic detection of early neoplasia in Barrett's esophagus is difficult. Computer Aided Detection (CADe) systems may assist in neoplasia detection. The aim of this study was to report the first steps in the development of a CADe system for Barrett's neoplasia and to evaluate its performance when compared with endoscopists. METHODS: This CADe system was developed by a consortium, consisting of the Amsterdam University Medical Center, Eindhoven University of Technology, and 15 international hospitals. After pretraining, the system was trained and validated using 1.713 neoplastic (564 patients) and 2.707 non-dysplastic Barrett's esophagus (NDBE; 665 patients) images. Neoplastic lesions were delineated by 14 experts. The performance of the CADe system was tested on three independent test sets. Test set 1 (50 neoplastic and 150 NDBE images) contained subtle neoplastic lesions representing challenging cases and was benchmarked by 52 general endoscopists. Test set 2 (50 neoplastic and 50 NDBE images) contained a heterogeneous case-mix of neoplastic lesions, representing distribution in clinical practice. Test set 3 (50 neoplastic and 150 NDBE images) contained prospectively collected imagery. The main outcome was correct classification of the images in terms of sensitivity. RESULTS: The sensitivity of the CADe system on test set 1 was 84%. For general endoscopists, sensitivity was 63%, corresponding to a neoplasia miss-rate of one-third of neoplastic lesions and a potential relative increase in neoplasia detection of 33% for CADe-assisted detection. The sensitivity of the CADe system on test sets 2 and 3 was 100% and 88%, respectively. The specificity of the CADe system varied for the three test sets between 64% and 66%. CONCLUSION: This study describes the first steps towards the establishment of an unprecedented data infrastructure for using machine learning to improve the endoscopic detection of Barrett's neoplasia. The CADe system detected neoplasia reliably and outperformed a large group of endoscopists in terms of sensitivity.