Artificial intelligence in small bowel capsule endoscopy - current status, challenges and future promise.

Neural network-based solutions are under development to alleviate physicians from the tedious task of small-bowel capsule endoscopy reviewing. Computer-assisted detection is a critical step, aiming to reduce reading times while maintaining accuracy. Weakly supervised solutions have shown promising results; however, video-level evaluations are scarce, and no prospective studies have been conducted yet. Automated characterization (in terms of diagnosis and pertinence) by supervised machine learning solutions is the next step. It relies on large, thoroughly labeled databases, for which preliminary "ground truth" definitions by experts are of tremendous importance. Other developments are under ways, to assist physicians in localizing anatomical landmarks and findings in the small bowel, in measuring lesions, and in rating bowel cleanliness. It is still questioned whether artificial intelligence will enter the market with proprietary, built-in or plug-in software, or with a universal cloud-based service, and how it will be accepted by physicians and patients.

Sensors, Signal and Image Processing in Biomedicine and Assisted Living.

Sensor technologies are crucial in biomedicine, as the biomedical systems and devices used for screening and diagnosis rely on their efficiency and effectiveness [...].

Uncertainty-Aware Visual Perception System for Outdoor Navigation of the Visually Challenged.

Every day, visually challenged people (VCP) face mobility restrictions and accessibility limitations. A short walk to a nearby destination, which for other individuals is taken for granted, becomes a challenge. To tackle this problem, we propose a novel visual perception system for outdoor navigation that can be evolved into an everyday visual aid for VCP. The proposed methodology is integrated in a wearable visual perception system (VPS). The proposed approach efficiently incorporates deep learning, object recognition models, along with an obstacle detection methodology based on human eye fixation prediction using Generative Adversarial Networks. An uncertainty-aware modeling of the obstacle risk assessment and spatial localization has been employed, following a fuzzy logic approach, for robust obstacle detection. The above combination can translate the position and the type of detected obstacles into descriptive linguistic expressions, allowing the users to easily understand their location in the environment and avoid them. The performance and capabilities of the proposed method are investigated in the context of safe navigation of VCP in outdoor environments of cultural interest through obstacle recognition and detection. Additionally, a comparison between the proposed system and relevant state-of-the-art systems for the safe navigation of VCP, focused on design and user-requirements satisfaction, is performed.

Automatic lesion detection in capsule endoscopy based on color saliency: closer to an essential adjunct for reviewing software.

BACKGROUND: The advent of wireless capsule endoscopy (WCE) has revolutionized the diagnostic approach to small-bowel disease. However, the task of reviewing WCE video sequences is laborious and time-consuming; software tools offering automated video analysis would enable a timelier and potentially a more accurate diagnosis. OBJECTIVE: To assess the validity of innovative, automatic lesion-detection software in WCE. DESIGN/INTERVENTION: A color feature-based pattern recognition methodology was devised and applied to the aforementioned image group. SETTING: This study was performed at the Royal Infirmary of Edinburgh, United Kingdom, and the Technological Educational Institute of Central Greece, Lamia, Greece. MATERIALS: A total of 137 deidentified WCE single images, 77 showing pathology and 60 normal images. RESULTS: The proposed methodology, unlike state-of-the-art approaches, is capable of detecting several different types of lesions. The average performance, in terms of the area under the receiver-operating characteristic curve, reached 89.2 ± 0.9%. The best average performance was obtained for angiectasias (97.5 ± 2.4%) and nodular lymphangiectasias (96.3 ± 3.6%). LIMITATIONS: Single expert for annotation of pathologies, single type of WCE model, use of single images instead of entire WCE videos. CONCLUSION: A simple, yet effective, approach allowing automatic detection of all types of abnormalities in capsule endoscopy is presented. Based on color pattern recognition, it outperforms previous state-of-the-art approaches. Moreover, it is robust in the presence of luminal contents and is capable of detecting even very small lesions.

Interpretable EEG-Based Emotion Recognition Using Fuzzy Cognitive Maps.

The brain is one of the most complex parts of the human body, consisting of billions of neurons and it is involved in almost all vital functions. To study the brain functionality, Electroencephalography (EEG) is used to record the electrical activity generated by the brain through electrodes placed on the scalp surface. In this paper, an auto-constructed Fuzzy Cognitive Map (FCM) model is used for interpretable emotion recognition, based on EEG signals. The introduced model constitutes the first FCM that automatically detects the cause-and-effects relations existing among brain regions and emotions induced by movies watched by volunteers. In addition, it is simple to implement and earns the trust of the user, while providing interpretable results. The effectiveness of the model over other baseline and state-of-the-art methods is examined using a publicly available dataset.

E pluribus unum interpretable convolutional neural networks.

The adoption of convolutional neural network (CNN) models in high-stake domains is hindered by their inability to meet society's demand for transparency in decision-making. So far, a growing number of methodologies have emerged for developing CNN models that are interpretable by design. However, such models are not capable of providing interpretations in accordance with human perception, while maintaining competent performance. In this paper, we tackle these challenges with a novel, general framework for instantiating inherently interpretable CNN models, named E pluribus unum interpretable CNN (EPU-CNN). An EPU-CNN model consists of CNN sub-networks, each of which receives a different representation of an input image expressing a perceptual feature, such as color or texture. The output of an EPU-CNN model consists of the classification prediction and its interpretation, in terms of relative contributions of perceptual features in different regions of the input image. EPU-CNN models have been extensively evaluated on various publicly available datasets, as well as a contributed benchmark dataset. Medical datasets are used to demonstrate the applicability of EPU-CNN for risk-sensitive decisions in medicine. The experimental results indicate that EPU-CNN models can achieve a comparable or better classification performance than other CNN architectures while providing humanly perceivable interpretations.

A semantic model for multimodal data mining in healthcare information systems.

Electronic health records (EHRs) are representative examples of multimodal/multisource data collections; including measurements, images and free texts. The diversity of such information sources and the increasing amounts of medical data produced by healthcare institutes annually, pose significant challenges in data mining. In this paper we present a novel semantic model that describes knowledge extracted from the lowest-level of a data mining process, where information is represented by multiple features i.e. measurements or numerical descriptors extracted from measurements, images, texts or other medical data, forming multidimensional feature spaces. Knowledge collected by manual annotation or extracted by unsupervised data mining from one or more feature spaces is modeled through generalized qualitative spatial semantics. This model enables a unified representation of knowledge across multimodal data repositories. It contributes to bridging the semantic gap, by enabling direct links between low-level features and higher-level concepts e.g. describing body parts, anatomies and pathological findings. The proposed model has been developed in web ontology language based on description logics (OWL-DL) and can be applied to a variety of data mining tasks in medical informatics. It utility is demonstrated for automatic annotation of medical data.

Constructive Fuzzy Cognitive Map for Depression Severity Estimation.

Depression is a common and serious medical disorder that negatively affects the mood and the emotions of people, especially adolescents. In this paper, a novel framework for automatically creating Fuzzy Cognitive Maps (FCMs) is proposed. It is applied for the estimation of the severity of depression among adolescents, based on their electroencephalogram (EEG). The introduced Constructive FCM (CFCM) utilizes features extracted by a Constructive Fuzzy Representation Model (CFRM), which conduces to detect in a more intuitive way the cause-and-effect relationships between the brain activity and depression. CFCM contributes to limiting the participation of experts, and the manual interventions in the traditional construction of FCMs, it provides an embedded mechanism for dimensionality reduction, and it constitutes an inherently interpretable approach to decision making, while being uncertainty-aware and simple to implement. The results of the experiments, using a recent publicly available dataset, demonstrate the effectiveness of the proposed framework and highlight its advantages.

Artificial Intelligence in Cardiology-A Narrative Review of Current Status.

Artificial intelligence (AI) is an integral part of clinical decision support systems (CDSS), offering methods to approximate human reasoning and computationally infer decisions. Such methods are generally based on medical knowledge, either directly encoded with rules or automatically extracted from medical data using machine learning (ML). ML techniques, such as Artificial Neural Networks (ANNs) and support vector machines (SVMs), are based on mathematical models with parameters that can be optimally tuned using appropriate algorithms. The ever-increasing computational capacity of today's computer systems enables more complex ML systems with millions of parameters, bringing AI closer to human intelligence. With this objective, the term deep learning (DL) has been introduced to characterize ML based on deep ANN (DNN) architectures with multiple layers of artificial neurons. Despite all of these promises, the impact of AI in current clinical practice is still limited. However, this could change shortly, as the significantly increased papers in AI, machine learning and deep learning in cardiology show. We highlight the significant achievements of recent years in nearly all areas of cardiology and underscore the mounting evidence suggesting how AI will take a central stage in the field.

Key research questions for implementation of artificial intelligence in capsule endoscopy.

Background: Artificial intelligence (AI) is rapidly infiltrating multiple areas in medicine, with gastrointestinal endoscopy paving the way in both research and clinical applications. Multiple challenges associated with the incorporation of AI in endoscopy are being addressed in recent consensus documents. Objectives: In the current paper, we aimed to map future challenges and areas of research for the incorporation of AI in capsule endoscopy (CE) practice. Design: Modified three-round Delphi consensus online survey. Methods: The study design was based on a modified three-round Delphi consensus online survey distributed to a group of CE and AI experts. Round one aimed to map out key research statements and challenges for the implementation of AI in CE. All queries addressing the same questions were merged into a single issue. The second round aimed to rank all generated questions during round one and to identify the top-ranked statements with the highest total score. Finally, the third round aimed to redistribute and rescore the top-ranked statements. Results: Twenty-one (16 gastroenterologists and 5 data scientists) experts participated in the survey. In the first round, 48 statements divided into seven themes were generated. After scoring all statements and rescoring the top 12, the question of AI use for identification and grading of small bowel pathologies was scored the highest (mean score 9.15), correlation of AI and human expert reading-second (9.05), and real-life feasibility-third (9.0). Conclusion: In summary, our current study points out a roadmap for future challenges and research areas on our way to fully incorporating AI in CE reading.

A Constructive Fuzzy Representation Model for Heart Data Classification.

The early detection of Heart Disease (HD) and the prediction of Heart Failure (HF) via telemonitoring and can contribute to the reduction of patients' mortality and morbidity as well as to the reduction of respective treatment costs. In this study we propose a novel classification model based on fuzzy logic applied in the context of HD detection and HF prediction. The proposed model considers that data can be represented by fuzzy phrases constructed from fuzzy words, which are fuzzy sets derived from data. Advantages of this approach include the robustness of data classification, as well as an intuitive way for feature selection. The accuracy of the proposed model is investigated on real home telemonitoring data and a publicly available dataset from UCI.

M3G: maximum margin microarray gridding.

BACKGROUND: Complementary DNA (cDNA) microarrays are a well established technology for studying gene expression. A microarray image is obtained by laser scanning a hybridized cDNA microarray, which consists of thousands of spots representing chains of cDNA sequences, arranged in a two-dimensional array. The separation of the spots into distinct cells is widely known as microarray image gridding. METHODS: In this paper we propose M3G, a novel method for automatic gridding of cDNA microarray images based on the maximization of the margin between the rows and the columns of the spots. Initially the microarray image rotation is estimated and then a pre-processing algorithm is applied for a rough spot detection. In order to diminish the effect of artefacts, only a subset of the detected spots is selected by matching the distribution of the spot sizes to the normal distribution. Then, a set of grid lines is placed on the image in order to separate each pair of consecutive rows and columns of the selected spots. The optimal positioning of the lines is determined by maximizing the margin between these rows and columns by using a maximum margin linear classifier, effectively facilitating the localization of the spots. RESULTS: The experimental evaluation was based on a reference set of microarray images containing more than two million spots in total. The results show that M3G outperforms state of the art methods, demonstrating robustness in the presence of noise and artefacts. More than 98% of the spots reside completely inside their respective grid cells, whereas the mean distance between the spot center and the grid cell center is 1.2 pixels. CONCLUSIONS: The proposed method performs highly accurate gridding in the presence of noise and artefacts, while taking into account the input image rotation. Thus, it provides the potential of achieving perfect gridding for the vast majority of the spots.

The future of capsule endoscopy in clinical practice: from diagnostic to therapeutic experimental prototype capsules.

Capsule endoscopy (CE) is indicated as a first-line clinical examination for the detection of small-bowel pathology, and there is an ever-growing drive for it to become a method for the screening of the entire gastrointestinal tract (GI). Although CE's main function is diagnosis, the research for therapeutic capabilities has intensified to make therapeutic capsule endoscopy (TCE) a target within reach. This manuscript presents the research evolution of CE and TCE through the last 5 years and describes notable problems, as well as clinical and technological challenges to overcome. This review also reports the state-of-the-art of capsule devices with a focus on CE research prototypes promising an enhanced diagnostic yield (DY) and treatment. Lastly, this article provides an overview of the research progress made in software for enhancing DY by increasing the accuracy of abnormality detection and lesion localisation.

Deep Endoscopic Visual Measurements.

Robotic endoscopic systems offer a minimally invasive approach to the examination of internal body structures, and their application is rapidly extending to cover the increasing needs for accurate therapeutic interventions. In this context, it is essential for such systems to be able to perform measurements, such as measuring the distance traveled by a wireless capsule endoscope, so as to determine the location of a lesion in the gastrointestinal tract, or to measure the size of lesions for diagnostic purposes. In this paper, we investigate the feasibility of performing contactless measurements using a computer vision approach based on neural networks. The proposed system integrates a deep convolutional image registration approach and a multilayer feed-forward neural network into a novel architecture. The main advantage of this system, with respect to the state-of-the-art ones, is that it is more generic in the sense that it is 1) unconstrained by specific models, 2) more robust to nonrigid deformations, and 3) adaptable to most of the endoscopic systems and environment, while enabling measurements of enhanced accuracy. The performance of this system is evaluated under ex vivo conditions using a phantom experimental model and a robotically assisted test bench. The results obtained promise a wider applicability and impact in endoscopy in the era of big data.

Use of Hyperspectral/Multispectral Imaging in Gastroenterology. Shedding Some⁻Different⁻Light into the Dark.

Hyperspectral/Multispectral imaging (HSI/MSI) technologies are able to sample from tens to hundreds of spectral channels within the electromagnetic spectrum, exceeding the capabilities of human vision. These spectral techniques are based on the principle that every material has a different response (reflection and absorption) to different wavelengths. Thereby, this technology facilitates the discrimination between different materials. HSI has demonstrated good discrimination capabilities for materials in fields, for instance, remote sensing, pollution monitoring, field surveillance, food quality, agriculture, astronomy, geological mapping, and currently, also in medicine. HSI technology allows tissue observation beyond the limitations of the human eye. Moreover, many researchers are using HSI as a new diagnosis tool to analyze optical properties of tissue. Recently, HSI has shown good performance in identifying human diseases in a non-invasive manner. In this paper, we show the potential use of these technologies in the medical domain, with emphasis in the current advances in gastroenterology. The main aim of this review is to provide an overview of contemporary concepts regarding HSI technology together with state-of-art systems and applications in gastroenterology. Finally, we discuss the current limitations and upcoming trends of HSI in gastroenterology.

Follow-up on: optimizing lesion detection in small bowel capsule endoscopy and beyond: from present problems to future solutions.

INTRODUCTION: This review presents noteworthy advances in clinical and experimental Capsule Endoscopy (CE), focusing on the progress that has been reported over the last 5 years since our previous review on the subject. Areas covered: This study presents the commercially available CE platforms, as well as the advances made in optimizing the diagnostic capabilities of CE. The latter includes recent concept and prototype capsule endoscopes, medical approaches to improve diagnostic yield, and progress in software for enhancing visualization, abnormality detection, and lesion localization. Expert commentary: Currently, moving through the second decade of CE evolution, there are still several open issues and remarkable challenges to overcome.

DINOSARC: Color Features Based on Selective Aggregation of Chromatic Image Components for Wireless Capsule Endoscopy.

Wireless Capsule Endoscopy (WCE) is a noninvasive diagnostic technique enabling the inspection of the whole gastrointestinal (GI) tract by capturing and wirelessly transmitting thousands of color images. Proprietary software "stitches" the images into videos for examination by accredited readers. However, the videos produced are of large length and consequently the reading task becomes harder and more prone to human errors. Automating the WCE reading process could contribute in both the reduction of the examination time and the improvement of its diagnostic accuracy. In this paper, we present a novel feature extraction methodology for automated WCE image analysis. It aims at discriminating various kinds of abnormalities from the normal contents of WCE images, in a machine learning-based classification framework. The extraction of the proposed features involves an unsupervised color-based saliency detection scheme which, unlike current approaches, combines both point and region-level saliency information and the estimation of local and global image color descriptors. The salient point detection process involves estimation of DIstaNces On Selective Aggregation of chRomatic image Components (DINOSARC). The descriptors are extracted from superpixels by coevaluating both point and region-level information. The main conclusions of the experiments performed on a publicly available dataset of WCE images are (a) the proposed salient point detection scheme results in significantly less and more relevant salient points; (b) the proposed descriptors are more discriminative than relevant state-of-the-art descriptors, promising a wider adoption of the proposed approach for computer-aided diagnosis in WCE.

Detecting and Locating Gastrointestinal Anomalies Using Deep Learning and Iterative Cluster Unification.

This paper proposes a novel methodology for automatic detection and localization of gastrointestinal (GI) anomalies in endoscopic video frame sequences. Training is performed with weakly annotated images, using only image-level, semantic labels instead of detailed, and pixel-level annotations. This makes it a cost-effective approach for the analysis of large videoendoscopy repositories. Other advantages of the proposed methodology include its capability to suggest possible locations of GI anomalies within the video frames, and its generality, in the sense that abnormal frame detection is based on automatically derived image features. It is implemented in three phases: 1) it classifies the video frames into abnormal or normal using a weakly supervised convolutional neural network (WCNN) architecture; 2) detects salient points from deeper WCNN layers, using a deep saliency detection algorithm; and 3) localizes GI anomalies using an iterative cluster unification (ICU) algorithm. ICU is based on a pointwise cross-feature-map (PCFM) descriptor extracted locally from the detected salient points using information derived from the WCNN. Results, from extensive experimentation using publicly available collections of gastrointestinal endoscopy video frames, are presented. The data sets used include a variety of GI anomalies. Both anomaly detection and localization performance achieved, in terms of the area under receiver operating characteristic (AUC), were >80%. The highest AUC for anomaly detection was obtained on conventional gastroscopy images, reaching 96%, and the highest AUC for anomaly localization was obtained on wireless capsule endoscopy images, reaching 88%.

Novel experimental and software methods for image reconstruction and localization in capsule endoscopy.

BACKGROUND AND STUDY AIMS: Capsule endoscopy (CE) is invaluable for minimally invasive endoscopy of the gastrointestinal tract; however, several technological limitations remain including lack of reliable lesion localization. We present an approach to 3D reconstruction and localization using visual information from 2D CE images. PATIENTS AND METHODS: Colored thumbtacks were secured in rows to the internal wall of a LifeLike bowel model. A PillCam SB3 was calibrated and navigated linearly through the lumen by a high-precision robotic arm. The motion estimation algorithm used data (light falling on the object, fraction of reflected light and surface geometry) from 2D CE images in the video sequence to achieve 3D reconstruction of the bowel model at various frames. The ORB-SLAM technique was used for 3D reconstruction and CE localization within the reconstructed model. This algorithm compared pairs of points between images for reconstruction and localization. RESULTS: As the capsule moved through the model bowel 42 to 66 video frames were obtained per pass. Mean absolute error in the estimated distance travelled by the CE was 4.1 ±â€Š3.9 cm. Our algorithm was able to reconstruct the cylindrical shape of the model bowel with details of the attached thumbtacks. ORB-SLAM successfully reconstructed the bowel wall from simultaneous frames of the CE video. The "track" in the reconstruction corresponded well with the linear forwards-backwards movement of the capsule through the model lumen. CONCLUSION: The reconstruction methods, detailed above, were able to achieve good quality reconstruction of the bowel model and localization of the capsule trajectory using information from the CE video and images alone.

Variable background active contour model for computer-aided delineation of nodules in thyroid ultrasound images.

This paper presents a computer-aided approach for nodule delineation in thyroid ultrasound (US) images. The developed algorithm is based on a novel active contour model, named variable background active contour (VBAC), and incorporates the advantages of the level set region-based active contour without edges (ACWE) model, offering noise robustness and the ability to delineate multiple nodules. Unlike the classic active contour models that are sensitive in the presence of intensity inhomogeneities, the proposed VBAC model considers information of variable background regions. VBAC has been evaluated on synthetic images, as well as on real thyroid US images. From the quantification of the results, two major impacts have been derived: 1) higher average accuracy in the delineation of hypoechoic thyroid nodules, which exceeds 91%; and 2) faster convergence when compared with the ACWE model.