Self-supervised out-of-distribution detection in wireless capsule endoscopy images.

While deep learning has displayed excellent performance in a broad spectrum of application areas, neural networks still struggle to recognize what they have not seen, i.e., out-of-distribution (OOD) inputs. In the medical field, building robust models that are able to detect OOD images is highly critical, as these rare images could show diseases or anomalies that should be detected. In this study, we use wireless capsule endoscopy (WCE) images to present a novel patch-based self-supervised approach comprising three stages. First, we train a triplet network to learn vector representations of WCE image patches. Second, we cluster the patch embeddings to group patches in terms of visual similarity. Third, we use the cluster assignments as pseudolabels to train a patch classifier and use the Out-of-Distribution Detector for Neural Networks (ODIN) for OOD detection. The system has been tested on the Kvasir-capsule, a publicly released WCE dataset. Empirical results show an OOD detection improvement compared to baseline methods. Our method can detect unseen pathologies and anomalies such as lymphangiectasia, foreign bodies and blood with AUROC>0.6. This work presents an effective solution for OOD detection models without needing labeled images.

Study of capsule endoscopy delivery at scale through enhanced artificial intelligence-enabled analysis (the CESCAIL study).

AIM: Lower gastrointestinal (GI) diagnostics have been facing relentless capacity constraints for many years, even before the COVID-19 era. Restrictions from the COVID pandemic have resulted in a significant backlog in lower GI diagnostics. Given recent developments in deep neural networks (DNNs) and the application of artificial intelligence (AI) in endoscopy, automating capsule video analysis is now within reach. Comparable to the efficiency and accuracy of AI applications in small bowel capsule endoscopy, AI in colon capsule analysis will also improve the efficiency of video reading and address the relentless demand on lower GI services. The aim of the CESCAIL study is to determine the feasibility, accuracy and productivity of AI-enabled analysis tools (AiSPEED) for polyp detection compared with the 'gold standard': a conventional care pathway with clinician analysis. METHOD: This multi-centre, diagnostic accuracy study aims to recruit 674 participants retrospectively and prospectively from centres conducting colon capsule endoscopy (CCE) as part of their standard care pathway. After the study participants have undergone CCE, the colon capsule videos will be uploaded onto two different pathways: AI-enabled video analysis and the gold standard conventional clinician analysis pathway. The reports generated from both pathways will be compared for accuracy (sensitivity and specificity). The reading time can only be compared in the prospective cohort. In addition to validating the AI tool, this study will also provide observational data concerning its use to assess the pathway execution in real-world performance. RESULTS: The study is currently recruiting participants at multiple centres within the United Kingdom and is at the stage of collecting data. CONCLUSION: This standard diagnostic accuracy study carries no additional risk to patients as it does not affect the standard care pathway, and hence patient care remains unaffected.

Anatomical landmarks localization for capsule endoscopy studies.

Wireless Capsule Endoscopy is a medical procedure that uses a small, wireless camera to capture images of the inside of the digestive tract. The identification of the entrance and exit of the small bowel and of the large intestine is one of the first tasks that need to be accomplished to read a video. This paper addresses the design of a clinical decision support tool to detect these anatomical landmarks. We have developed a system based on deep learning that combines images, timestamps, and motion data to achieve state-of-the-art results. Our method does not only classify the images as being inside or outside the studied organs, but it is also able to identify the entrance and exit frames. The experiments performed with three different datasets (one public and two private) show that our system is able to approximate the landmarks while achieving high accuracy on the classification problem (inside/outside of the organ). When comparing the entrance and exit of the studied organs, the distance between predicted and real landmarks is reduced from 1.5 to 10 times with respect to previous state-of-the-art methods.

Clinicians' Guide to Artificial Intelligence in Colon Capsule Endoscopy-Technology Made Simple.

Artificial intelligence (AI) applications have become widely popular across the healthcare ecosystem. Colon capsule endoscopy (CCE) was adopted in the NHS England pilot project following the recent COVID pandemic's impact. It demonstrated its capability to relieve the national backlog in endoscopy. As a result, AI-assisted colon capsule video analysis has become gastroenterology's most active research area. However, with rapid AI advances, mastering these complex machine learning concepts remains challenging for healthcare professionals. This forms a barrier for clinicians to take on this new technology and embrace the new era of big data. This paper aims to bridge the knowledge gap between the current CCE system and the future, fully integrated AI system. The primary focus is on simplifying the technical terms and concepts in machine learning. This will hopefully address the general "fear of the unknown in AI" by helping healthcare professionals understand the basic principle of machine learning in capsule endoscopy and apply this knowledge in their future interactions and adaptation to AI technology. It also summarises the evidence of AI in CCE and its impact on diagnostic pathways. Finally, it discusses the unintended consequences of using AI, ethical challenges, potential flaws, and bias within clinical settings.

Key research questions for implementation of artificial intelligence in capsule endoscopy.

Background: Artificial intelligence (AI) is rapidly infiltrating multiple areas in medicine, with gastrointestinal endoscopy paving the way in both research and clinical applications. Multiple challenges associated with the incorporation of AI in endoscopy are being addressed in recent consensus documents. Objectives: In the current paper, we aimed to map future challenges and areas of research for the incorporation of AI in capsule endoscopy (CE) practice. Design: Modified three-round Delphi consensus online survey. Methods: The study design was based on a modified three-round Delphi consensus online survey distributed to a group of CE and AI experts. Round one aimed to map out key research statements and challenges for the implementation of AI in CE. All queries addressing the same questions were merged into a single issue. The second round aimed to rank all generated questions during round one and to identify the top-ranked statements with the highest total score. Finally, the third round aimed to redistribute and rescore the top-ranked statements. Results: Twenty-one (16 gastroenterologists and 5 data scientists) experts participated in the survey. In the first round, 48 statements divided into seven themes were generated. After scoring all statements and rescoring the top 12, the question of AI use for identification and grading of small bowel pathologies was scored the highest (mean score 9.15), correlation of AI and human expert reading-second (9.05), and real-life feasibility-third (9.0). Conclusion: In summary, our current study points out a roadmap for future challenges and research areas on our way to fully incorporating AI in CE reading.

Cardiac aging synthesis from cross-sectional data with conditional generative adversarial networks.

Age has important implications for health, and understanding how age manifests in the human body is the first step for a potential intervention. This becomes especially important for cardiac health, since age is the main risk factor for development of cardiovascular disease. Data-driven modeling of age progression has been conducted successfully in diverse applications such as face or brain aging. While longitudinal data is the preferred option for training deep learning models, collecting such a dataset is usually very costly, especially in medical imaging. In this work, a conditional generative adversarial network is proposed to synthesize older and younger versions of a heart scan by using only cross-sectional data. We train our model with more than 14,000 different scans from the UK Biobank. The induced modifications focused mainly on the interventricular septum and the aorta, which is consistent with the existing literature in cardiac aging. We evaluate the results by measuring image quality, the mean absolute error for predicted age using a pre-trained regressor, and demonstrate the application of synthetic data for counter-balancing biased datasets. The results suggest that the proposed approach is able to model realistic changes in the heart using only cross-sectional data and that these data can be used to correct age bias in a dataset.

Artificial intelligence to improve polyp detection and screening time in colon capsule endoscopy.

Colon Capsule Endoscopy (CCE) is a minimally invasive procedure which is increasingly being used as an alternative to conventional colonoscopy. Videos recorded by the capsule cameras are long and require one or more experts' time to review and identify polyps or other potential intestinal problems that can lead to major health issues. We developed and tested a multi-platform web application, AI-Tool, which embeds a Convolution Neural Network (CNN) to help CCE reviewers. With the help of artificial intelligence, AI-Tool is able to detect images with high probability of containing a polyp and prioritize them during the reviewing process. With the collaboration of 3 experts that reviewed 18 videos, we compared the classical linear review method using RAPID Reader Software v9.0 and the new software we present. Applying the new strategy, reviewing time was reduced by a factor of 6 and polyp detection sensitivity was increased from 81.08 to 87.80%.

Minimising multi-centre radiomics variability through image normalisation: a pilot study.

Radiomics is an emerging technique for the quantification of imaging data that has recently shown great promise for deeper phenotyping of cardiovascular disease. Thus far, the technique has been mostly applied in single-centre studies. However, one of the main difficulties in multi-centre imaging studies is the inherent variability of image characteristics due to centre differences. In this paper, a comprehensive analysis of radiomics variability under several image- and feature-based normalisation techniques was conducted using a multi-centre cardiovascular magnetic resonance dataset. 218 subjects divided into healthy (n = 112) and hypertrophic cardiomyopathy (n = 106, HCM) groups from five different centres were considered. First and second order texture radiomic features were extracted from three regions of interest, namely the left and right ventricular cavities and the left ventricular myocardium. Two methods were used to assess features' variability. First, feature distributions were compared across centres to obtain a distribution similarity index. Second, two classification tasks were proposed to assess: (1) the amount of centre-related information encoded in normalised features (centre identification) and (2) the generalisation ability for a classification model when trained on these features (healthy versus HCM classification). The results showed that the feature-based harmonisation technique ComBat is able to remove the variability introduced by centre information from radiomic features, at the expense of slightly degrading classification performance. Piecewise linear histogram matching normalisation gave features with greater generalisation ability for classification ( balanced accuracy in between 0.78 ± 0.08 and 0.79 ± 0.09). Models trained with features from images without normalisation showed the worst performance overall ( balanced accuracy in between 0.45 ± 0.28 and 0.60 ± 0.22). In conclusion, centre-related information removal did not imply good generalisation ability for classification.

Time-based self-supervised learning for Wireless Capsule Endoscopy.

State-of-the-art machine learning models, and especially deep learning ones, are significantly data-hungry; they require vast amounts of manually labeled samples to function correctly. However, in most medical imaging fields, obtaining said data can be challenging. Not only the volume of data is a problem, but also the imbalances within its classes; it is common to have many more images of healthy patients than of those with pathology. Computer-aided diagnostic systems suffer from these issues, usually over-designing their models to perform accurately. This work proposes using self-supervised learning for wireless endoscopy videos by introducing a custom-tailored method that does not initially need labels or appropriate balance. We prove that using the inferred inherent structure learned by our method, extracted from the temporal axis, improves the detection rate on several domain-specific applications even under severe imbalance. State-of-the-art results are achieved in polyp detection, with 95.00 ± 2.09% Area Under the Curve, and 92.77 ± 1.20% accuracy in the CAD-CAP dataset.

Mathematical Abilities in School-Aged Children: A Structural Magnetic Resonance Imaging Analysis With Radiomics.

Structural magnetic resonance imaging (sMRI) studies have shown that children that differ in some mathematical abilities show differences in gray matter volume mainly in parietal and frontal regions that are involved in number processing, attentional control, and memory. In the present study, a structural neuroimaging analysis based on radiomics and machine learning models is presented with the aim of identifying the brain areas that better predict children's performance in a variety of mathematical tests. A sample of 77 school-aged children from third to sixth grade were administered four mathematical tests: Math fluency, Calculation, Applied problems and Quantitative concepts as well as a structural brain imaging scan. By extracting radiomics related to the shape, intensity, and texture of specific brain areas, we observed that areas from the frontal, parietal, temporal, and occipital lobes, basal ganglia, and limbic system, were differentially related to children's performance in the mathematical tests. sMRI-based analyses in the context of mathematical performance have been mainly focused on volumetric measures. However, the results for radiomics-based analysis showed that for these areas, texture features were the most important for the regression models, while volume accounted for less than 15% of the shape importance. These findings highlight the potential of radiomics for more in-depth analysis of medical images for the identification of brain areas related to mathematical abilities.

Sequential Models for Endoluminal Image Classification.

Wireless Capsule Endoscopy (WCE) is a procedure to examine the human digestive system for potential mucosal polyps, tumours, or bleedings using an encapsulated camera. This work focuses on polyp detection within WCE videos through Machine Learning. When using Machine Learning in the medical field, scarce and unbalanced datasets often make it hard to receive a satisfying performance. We claim that using Sequential Models in order to take the temporal nature of the data into account improves the performance of previous approaches. Thus, we present a bidirectional Long Short-Term Memory Network (BLSTM), a sequential network that is particularly designed for temporal data. We find the BLSTM Network outperforms non-sequential architectures and other previous models, receiving a final Area under the Curve of 93.83%. Experiments show that our method of extracting spatial and temporal features yields better performance and could be a possible method to decrease the time needed by physicians to analyse the video material.

Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation: The M&Ms Challenge.

The emergence of deep learning has considerably advanced the state-of-the-art in cardiac magnetic resonance (CMR) segmentation. Many techniques have been proposed over the last few years, bringing the accuracy of automated segmentation close to human performance. However, these models have been all too often trained and validated using cardiac imaging samples from single clinical centres or homogeneous imaging protocols. This has prevented the development and validation of models that are generalizable across different clinical centres, imaging conditions or scanner vendors. To promote further research and scientific benchmarking in the field of generalizable deep learning for cardiac segmentation, this paper presents the results of the Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation (M&Ms) Challenge, which was recently organized as part of the MICCAI 2020 Conference. A total of 14 teams submitted different solutions to the problem, combining various baseline models, data augmentation strategies, and domain adaptation techniques. The obtained results indicate the importance of intensity-driven data augmentation, as well as the need for further research to improve generalizability towards unseen scanner vendors or new imaging protocols. Furthermore, we present a new resource of 375 heterogeneous CMR datasets acquired by using four different scanner vendors in six hospitals and three different countries (Spain, Canada and Germany), which we provide as open-access for the community to enable future research in the field.

Artificial intelligence for the detection of polyps or cancer with colon capsule endoscopy.

Colorectal cancer is common and can be devastating, with long-term survival rates vastly improved by early diagnosis. Colon capsule endoscopy (CCE) is increasingly recognised as a reliable option for colonic surveillance, but widespread adoption has been slow for several reasons, including the time-consuming reading process of the CCE recording. Automated image recognition and artificial intelligence (AI) are appealing solutions in CCE. Through a review of the currently available and developmental technologies, we discuss how AI is poised to deliver at the forefront of CCE in the coming years. Current practice for CCE reporting often involves a two-step approach, with a 'pre-reader' and 'validator'. This requires skilled and experienced readers with a significant time commitment. Therefore, CCE is well-positioned to reap the benefits of the ongoing digital innovation. This is likely to initially involve an automated AI check of finished CCE evaluations as a quality control measure. Once felt reliable, AI could be used in conjunction with a 'pre-reader', before adopting more of this role by sending provisional results and abnormal frames to the validator. With time, AI would be able to evaluate the findings more thoroughly and reduce the input required from human readers and ultimately autogenerate a highly accurate report and recommendation of therapy, if required, for any pathology identified. As with many medical fields reliant on image recognition, AI will be a welcome aid in CCE. Initially, this will be as an adjunct to 'double-check' that nothing has been missed, but with time will hopefully lead to a faster, more convenient diagnostic service for the screening population.

WCE polyp detection with triplet based embeddings.

Wireless capsule endoscopy is a medical procedure used to visualize the entire gastrointestinal tract and to diagnose intestinal conditions, such as polyps or bleeding. Current analyses are performed by manually inspecting nearly each one of the frames of the video, a tedious and error-prone task. Automatic image analysis methods can be used to reduce the time needed for physicians to evaluate a capsule endoscopy video. However these methods are still in a research phase. In this paper we focus on computer-aided polyp detection in capsule endoscopy images. This is a challenging problem because of the diversity of polyp appearance, the imbalanced dataset structure and the scarcity of data. We have developed a new polyp computer-aided decision system that combines a deep convolutional neural network and metric learning. The key point of the method is the use of the Triplet Loss function with the aim of improving feature extraction from the images when having small dataset. The Triplet Loss function allows to train robust detectors by forcing images from the same category to be represented by similar embedding vectors while ensuring that images from different categories are represented by dissimilar vectors. Empirical results show a meaningful increase of AUC values compared to state-of-the-art methods. A good performance is not the only requirement when considering the adoption of this technology to clinical practice. Trust and explainability of decisions are as important as performance. With this purpose, we also provide a method to generate visual explanations of the outcome of our polyp detector. These explanations can be used to build a physician's trust in the system and also to convey information about the inner working of the method to the designer for debugging purposes.

Motor dysfunction of the gut in cystic fibrosis.

BACKGROUND: Cystic fibrosis (CF) is a multisystem disease caused by mutations in the cystic fibrosis transmembrane conductance regulator (CFTR) protein. Cystic fibrosis transmembrane conductance regulator is extensively expressed in the intestine and has an important role in the regulation of the viscosity and pH of gut secretions. Several studies have reported a delay in small bowel and colonic transit times in patients with CF which have been attributed to the secretory dysfunction. Our aim was to determine whether intestinal contractility is affected in these patients. METHODS: Consecutive patients with CF referred to our institution between 2014 and 2017 (n = 16) were prospectively investigated using automated non-invasive techniques for morpho-functional evaluation of the gut developed in our laboratory. On separate days, intraluminal images of the gut were obtained by capsule endoscopy and external images by abdominal MRI. Analysis of images (endoluminal and external) was performed with original, previously validated programs based on computer vision and machine learning techniques and compared with age- and sex-matched controls. KEY RESULTS: Patients with CF exhibited important reduction in contractile activity and increased retention of static turbid luminal content in the small bowel by endoluminal image analysis. Morpho-volumetric analysis of MRI images found increased ileo-colonic volumes in CF. Significant correlations between abnormalities detected by intraluminal and external imaging techniques were found. The presence and severity of digestive symptoms were not related to abnormal gut function. CONCLUSION AND INFERENCES: Impaired transit and pooling of gut contents in patients with CF is associated with impaired intestinal motility.

Generic feature learning for wireless capsule endoscopy analysis.

The interpretation and analysis of wireless capsule endoscopy (WCE) recordings is a complex task which requires sophisticated computer aided decision (CAD) systems to help physicians with video screening and, finally, with the diagnosis. Most CAD systems used in capsule endoscopy share a common system design, but use very different image and video representations. As a result, each time a new clinical application of WCE appears, a new CAD system has to be designed from the scratch. This makes the design of new CAD systems very time consuming. Therefore, in this paper we introduce a system for small intestine motility characterization, based on Deep Convolutional Neural Networks, which circumvents the laborious step of designing specific features for individual motility events. Experimental results show the superiority of the learned features over alternative classifiers constructed using state-of-the-art handcrafted features. In particular, it reaches a mean classification accuracy of 96% for six intestinal motility events, outperforming the other classifiers by a large margin (a 14% relative performance increase).

Classification of functional bowel disorders by objective physiological criteria based on endoluminal image analysis.

We have previously developed an original method to evaluate small bowel motor function based on computer vision analysis of endoluminal images obtained by capsule endoscopy. Our aim was to demonstrate intestinal motor abnormalities in patients with functional bowel disorders by endoluminal vision analysis. Patients with functional bowel disorders (n = 205) and healthy subjects (n = 136) ingested the endoscopic capsule (Pillcam-SB2, Given-Imaging) after overnight fast and 45 min after gastric exit of the capsule a liquid meal (300 ml, 1 kcal/ml) was administered. Endoluminal image analysis was performed by computer vision and machine learning techniques to define the normal range and to identify clusters of abnormal function. After training the algorithm, we used 196 patients and 48 healthy subjects, completely naive, as test set. In the test set, 51 patients (26%) were detected outside the normal range (P < 0.001 vs. 3 healthy subjects) and clustered into hypo- and hyperdynamic subgroups compared with healthy subjects. Patients with hypodynamic behavior (n = 38) exhibited less luminal closure sequences (41 ± 2% of the recording time vs. 61 ± 2%; P < 0.001) and more static sequences (38 ± 3 vs. 20 ± 2%; P < 0.001); in contrast, patients with hyperdynamic behavior (n = 13) had an increased proportion of luminal closure sequences (73 ± 4 vs. 61 ± 2%; P = 0.029) and more high-motion sequences (3 ± 1 vs. 0.5 ± 0.1%; P < 0.001). Applying an original methodology, we have developed a novel classification of functional gut disorders based on objective, physiological criteria of small bowel function.

Motility bar: A new tool for motility analysis of endoluminal videos.

Wireless Capsule Endoscopy (WCE) provides a new perspective of the small intestine, since it enables, for the first time, visualization of the entire organ. However, the long visual video analysis time, due to the large number of data in a single WCE study, was an important factor impeding the widespread use of the capsule as a tool for intestinal abnormalities detection. Therefore, the introduction of WCE triggered a new field for the application of computational methods, and in particular, of computer vision. In this paper, we follow the computational approach and come up with a new perspective on the small intestine motility problem. Our approach consists of three steps: first, we review a tool for the visualization of the motility information contained in WCE video; second, we propose algorithms for the characterization of two motility building-blocks: contraction detector and lumen size estimation; finally, we introduce an approach to detect segments of stable motility behavior. Our claims are supported by an evaluation performed with 10 WCE videos, suggesting that our methods ably capture the intestinal motility information.

Detection of wrinkle frames in endoluminal videos using betweenness centrality measures for images.

Intestinal contractions are one of the most important events to diagnose motility pathologies of the small intestine. When visualized by wireless capsule endoscopy (WCE), the sequence of frames that represents a contraction is characterized by a clear wrinkle structure in the central frames that corresponds to the folding of the intestinal wall. In this paper, we present a new method to robustly detect wrinkle frames in full WCE videos by using a new mid-level image descriptor that is based on a centrality measure proposed for graphs. We present an extended validation, carried out in a very large database, that shows that the proposed method achieves state-of-the-art performance for this task.

Adaptable image cuts for motility inspection using WCE.

The Wireless Capsule Endoscopy (WCE) technology allows the visualization of the whole small intestine tract. Since the capsule is freely moving, mainly by the means of peristalsis, the data acquired during the study gives a lot of information about the intestinal motility. However, due to: (1) huge amount of frames, (2) complex intestinal scene appearance and (3) intestinal dynamics that make difficult the visualization of the small intestine physiological phenomena, the analysis of the WCE data requires computer-aided systems to speed up the analysis. In this paper, we propose an efficient algorithm for building a novel representation of the WCE video data, optimal for motility analysis and inspection. The algorithm transforms the 3D video data into 2D longitudinal view by choosing the most informative, from the intestinal motility point of view, part of each frame. This step maximizes the lumen visibility in its longitudinal extension. The task of finding "the best longitudinal view" has been defined as a cost function optimization problem which global minimum is obtained by using Dynamic Programming. Validation on both synthetic data and WCE data shows that the adaptive longitudinal view is a good alternative to the traditional motility analysis done by video analysis. The proposed novel data representation a new, holistic insight into the small intestine motility, allowing to easily define and analyze motility events that are difficult to spot by analyzing WCE video. Moreover, the visual inspection of small intestine motility is 4 times faster then by means of video skimming of the WCE.