E pluribus unum interpretable convolutional neural networks.

The adoption of convolutional neural network (CNN) models in high-stake domains is hindered by their inability to meet society's demand for transparency in decision-making. So far, a growing number of methodologies have emerged for developing CNN models that are interpretable by design. However, such models are not capable of providing interpretations in accordance with human perception, while maintaining competent performance. In this paper, we tackle these challenges with a novel, general framework for instantiating inherently interpretable CNN models, named E pluribus unum interpretable CNN (EPU-CNN). An EPU-CNN model consists of CNN sub-networks, each of which receives a different representation of an input image expressing a perceptual feature, such as color or texture. The output of an EPU-CNN model consists of the classification prediction and its interpretation, in terms of relative contributions of perceptual features in different regions of the input image. EPU-CNN models have been extensively evaluated on various publicly available datasets, as well as a contributed benchmark dataset. Medical datasets are used to demonstrate the applicability of EPU-CNN for risk-sensitive decisions in medicine. The experimental results indicate that EPU-CNN models can achieve a comparable or better classification performance than other CNN architectures while providing humanly perceivable interpretations.

Interpretable EEG-Based Emotion Recognition Using Fuzzy Cognitive Maps.

The brain is one of the most complex parts of the human body, consisting of billions of neurons and it is involved in almost all vital functions. To study the brain functionality, Electroencephalography (EEG) is used to record the electrical activity generated by the brain through electrodes placed on the scalp surface. In this paper, an auto-constructed Fuzzy Cognitive Map (FCM) model is used for interpretable emotion recognition, based on EEG signals. The introduced model constitutes the first FCM that automatically detects the cause-and-effects relations existing among brain regions and emotions induced by movies watched by volunteers. In addition, it is simple to implement and earns the trust of the user, while providing interpretable results. The effectiveness of the model over other baseline and state-of-the-art methods is examined using a publicly available dataset.

Artificial Intelligence in Cardiology-A Narrative Review of Current Status.

Artificial intelligence (AI) is an integral part of clinical decision support systems (CDSS), offering methods to approximate human reasoning and computationally infer decisions. Such methods are generally based on medical knowledge, either directly encoded with rules or automatically extracted from medical data using machine learning (ML). ML techniques, such as Artificial Neural Networks (ANNs) and support vector machines (SVMs), are based on mathematical models with parameters that can be optimally tuned using appropriate algorithms. The ever-increasing computational capacity of today's computer systems enables more complex ML systems with millions of parameters, bringing AI closer to human intelligence. With this objective, the term deep learning (DL) has been introduced to characterize ML based on deep ANN (DNN) architectures with multiple layers of artificial neurons. Despite all of these promises, the impact of AI in current clinical practice is still limited. However, this could change shortly, as the significantly increased papers in AI, machine learning and deep learning in cardiology show. We highlight the significant achievements of recent years in nearly all areas of cardiology and underscore the mounting evidence suggesting how AI will take a central stage in the field.

Constructive Fuzzy Cognitive Map for Depression Severity Estimation.

Depression is a common and serious medical disorder that negatively affects the mood and the emotions of people, especially adolescents. In this paper, a novel framework for automatically creating Fuzzy Cognitive Maps (FCMs) is proposed. It is applied for the estimation of the severity of depression among adolescents, based on their electroencephalogram (EEG). The introduced Constructive FCM (CFCM) utilizes features extracted by a Constructive Fuzzy Representation Model (CFRM), which conduces to detect in a more intuitive way the cause-and-effect relationships between the brain activity and depression. CFCM contributes to limiting the participation of experts, and the manual interventions in the traditional construction of FCMs, it provides an embedded mechanism for dimensionality reduction, and it constitutes an inherently interpretable approach to decision making, while being uncertainty-aware and simple to implement. The results of the experiments, using a recent publicly available dataset, demonstrate the effectiveness of the proposed framework and highlight its advantages.

A Constructive Fuzzy Representation Model for Heart Data Classification.

The early detection of Heart Disease (HD) and the prediction of Heart Failure (HF) via telemonitoring and can contribute to the reduction of patients' mortality and morbidity as well as to the reduction of respective treatment costs. In this study we propose a novel classification model based on fuzzy logic applied in the context of HD detection and HF prediction. The proposed model considers that data can be represented by fuzzy phrases constructed from fuzzy words, which are fuzzy sets derived from data. Advantages of this approach include the robustness of data classification, as well as an intuitive way for feature selection. The accuracy of the proposed model is investigated on real home telemonitoring data and a publicly available dataset from UCI.

The future of capsule endoscopy in clinical practice: from diagnostic to therapeutic experimental prototype capsules.

Capsule endoscopy (CE) is indicated as a first-line clinical examination for the detection of small-bowel pathology, and there is an ever-growing drive for it to become a method for the screening of the entire gastrointestinal tract (GI). Although CE's main function is diagnosis, the research for therapeutic capabilities has intensified to make therapeutic capsule endoscopy (TCE) a target within reach. This manuscript presents the research evolution of CE and TCE through the last 5 years and describes notable problems, as well as clinical and technological challenges to overcome. This review also reports the state-of-the-art of capsule devices with a focus on CE research prototypes promising an enhanced diagnostic yield (DY) and treatment. Lastly, this article provides an overview of the research progress made in software for enhancing DY by increasing the accuracy of abnormality detection and lesion localisation.

Sensors, Signal and Image Processing in Biomedicine and Assisted Living.

Sensor technologies are crucial in biomedicine, as the biomedical systems and devices used for screening and diagnosis rely on their efficiency and effectiveness [...].

Uncertainty-Aware Visual Perception System for Outdoor Navigation of the Visually Challenged.

Every day, visually challenged people (VCP) face mobility restrictions and accessibility limitations. A short walk to a nearby destination, which for other individuals is taken for granted, becomes a challenge. To tackle this problem, we propose a novel visual perception system for outdoor navigation that can be evolved into an everyday visual aid for VCP. The proposed methodology is integrated in a wearable visual perception system (VPS). The proposed approach efficiently incorporates deep learning, object recognition models, along with an obstacle detection methodology based on human eye fixation prediction using Generative Adversarial Networks. An uncertainty-aware modeling of the obstacle risk assessment and spatial localization has been employed, following a fuzzy logic approach, for robust obstacle detection. The above combination can translate the position and the type of detected obstacles into descriptive linguistic expressions, allowing the users to easily understand their location in the environment and avoid them. The performance and capabilities of the proposed method are investigated in the context of safe navigation of VCP in outdoor environments of cultural interest through obstacle recognition and detection. Additionally, a comparison between the proposed system and relevant state-of-the-art systems for the safe navigation of VCP, focused on design and user-requirements satisfaction, is performed.

Follow-up on: optimizing lesion detection in small bowel capsule endoscopy and beyond: from present problems to future solutions.

INTRODUCTION: This review presents noteworthy advances in clinical and experimental Capsule Endoscopy (CE), focusing on the progress that has been reported over the last 5 years since our previous review on the subject. Areas covered: This study presents the commercially available CE platforms, as well as the advances made in optimizing the diagnostic capabilities of CE. The latter includes recent concept and prototype capsule endoscopes, medical approaches to improve diagnostic yield, and progress in software for enhancing visualization, abnormality detection, and lesion localization. Expert commentary: Currently, moving through the second decade of CE evolution, there are still several open issues and remarkable challenges to overcome.

Use of Hyperspectral/Multispectral Imaging in Gastroenterology. Shedding Some⁻Different⁻Light into the Dark.

Hyperspectral/Multispectral imaging (HSI/MSI) technologies are able to sample from tens to hundreds of spectral channels within the electromagnetic spectrum, exceeding the capabilities of human vision. These spectral techniques are based on the principle that every material has a different response (reflection and absorption) to different wavelengths. Thereby, this technology facilitates the discrimination between different materials. HSI has demonstrated good discrimination capabilities for materials in fields, for instance, remote sensing, pollution monitoring, field surveillance, food quality, agriculture, astronomy, geological mapping, and currently, also in medicine. HSI technology allows tissue observation beyond the limitations of the human eye. Moreover, many researchers are using HSI as a new diagnosis tool to analyze optical properties of tissue. Recently, HSI has shown good performance in identifying human diseases in a non-invasive manner. In this paper, we show the potential use of these technologies in the medical domain, with emphasis in the current advances in gastroenterology. The main aim of this review is to provide an overview of contemporary concepts regarding HSI technology together with state-of-art systems and applications in gastroenterology. Finally, we discuss the current limitations and upcoming trends of HSI in gastroenterology.

Deep Endoscopic Visual Measurements.

Robotic endoscopic systems offer a minimally invasive approach to the examination of internal body structures, and their application is rapidly extending to cover the increasing needs for accurate therapeutic interventions. In this context, it is essential for such systems to be able to perform measurements, such as measuring the distance traveled by a wireless capsule endoscope, so as to determine the location of a lesion in the gastrointestinal tract, or to measure the size of lesions for diagnostic purposes. In this paper, we investigate the feasibility of performing contactless measurements using a computer vision approach based on neural networks. The proposed system integrates a deep convolutional image registration approach and a multilayer feed-forward neural network into a novel architecture. The main advantage of this system, with respect to the state-of-the-art ones, is that it is more generic in the sense that it is 1) unconstrained by specific models, 2) more robust to nonrigid deformations, and 3) adaptable to most of the endoscopic systems and environment, while enabling measurements of enhanced accuracy. The performance of this system is evaluated under ex vivo conditions using a phantom experimental model and a robotically assisted test bench. The results obtained promise a wider applicability and impact in endoscopy in the era of big data.

DINOSARC: Color Features Based on Selective Aggregation of Chromatic Image Components for Wireless Capsule Endoscopy.

Wireless Capsule Endoscopy (WCE) is a noninvasive diagnostic technique enabling the inspection of the whole gastrointestinal (GI) tract by capturing and wirelessly transmitting thousands of color images. Proprietary software "stitches" the images into videos for examination by accredited readers. However, the videos produced are of large length and consequently the reading task becomes harder and more prone to human errors. Automating the WCE reading process could contribute in both the reduction of the examination time and the improvement of its diagnostic accuracy. In this paper, we present a novel feature extraction methodology for automated WCE image analysis. It aims at discriminating various kinds of abnormalities from the normal contents of WCE images, in a machine learning-based classification framework. The extraction of the proposed features involves an unsupervised color-based saliency detection scheme which, unlike current approaches, combines both point and region-level saliency information and the estimation of local and global image color descriptors. The salient point detection process involves estimation of DIstaNces On Selective Aggregation of chRomatic image Components (DINOSARC). The descriptors are extracted from superpixels by coevaluating both point and region-level information. The main conclusions of the experiments performed on a publicly available dataset of WCE images are (a) the proposed salient point detection scheme results in significantly less and more relevant salient points; (b) the proposed descriptors are more discriminative than relevant state-of-the-art descriptors, promising a wider adoption of the proposed approach for computer-aided diagnosis in WCE.

Detecting and Locating Gastrointestinal Anomalies Using Deep Learning and Iterative Cluster Unification.

This paper proposes a novel methodology for automatic detection and localization of gastrointestinal (GI) anomalies in endoscopic video frame sequences. Training is performed with weakly annotated images, using only image-level, semantic labels instead of detailed, and pixel-level annotations. This makes it a cost-effective approach for the analysis of large videoendoscopy repositories. Other advantages of the proposed methodology include its capability to suggest possible locations of GI anomalies within the video frames, and its generality, in the sense that abnormal frame detection is based on automatically derived image features. It is implemented in three phases: 1) it classifies the video frames into abnormal or normal using a weakly supervised convolutional neural network (WCNN) architecture; 2) detects salient points from deeper WCNN layers, using a deep saliency detection algorithm; and 3) localizes GI anomalies using an iterative cluster unification (ICU) algorithm. ICU is based on a pointwise cross-feature-map (PCFM) descriptor extracted locally from the detected salient points using information derived from the WCNN. Results, from extensive experimentation using publicly available collections of gastrointestinal endoscopy video frames, are presented. The data sets used include a variety of GI anomalies. Both anomaly detection and localization performance achieved, in terms of the area under receiver operating characteristic (AUC), were >80%. The highest AUC for anomaly detection was obtained on conventional gastroscopy images, reaching 96%, and the highest AUC for anomaly localization was obtained on wireless capsule endoscopy images, reaching 88%.

Novel experimental and software methods for image reconstruction and localization in capsule endoscopy.

BACKGROUND AND STUDY AIMS: Capsule endoscopy (CE) is invaluable for minimally invasive endoscopy of the gastrointestinal tract; however, several technological limitations remain including lack of reliable lesion localization. We present an approach to 3D reconstruction and localization using visual information from 2D CE images. PATIENTS AND METHODS: Colored thumbtacks were secured in rows to the internal wall of a LifeLike bowel model. A PillCam SB3 was calibrated and navigated linearly through the lumen by a high-precision robotic arm. The motion estimation algorithm used data (light falling on the object, fraction of reflected light and surface geometry) from 2D CE images in the video sequence to achieve 3D reconstruction of the bowel model at various frames. The ORB-SLAM technique was used for 3D reconstruction and CE localization within the reconstructed model. This algorithm compared pairs of points between images for reconstruction and localization. RESULTS: As the capsule moved through the model bowel 42 to 66 video frames were obtained per pass. Mean absolute error in the estimated distance travelled by the CE was 4.1 ±â€Š3.9 cm. Our algorithm was able to reconstruct the cylindrical shape of the model bowel with details of the attached thumbtacks. ORB-SLAM successfully reconstructed the bowel wall from simultaneous frames of the CE video. The "track" in the reconstruction corresponded well with the linear forwards-backwards movement of the capsule through the model lumen. CONCLUSION: The reconstruction methods, detailed above, were able to achieve good quality reconstruction of the bowel model and localization of the capsule trajectory using information from the CE video and images alone.

Intelligent visual localization of wireless capsule endoscopes enhanced by color information.

Wireless capsule endoscopy (WCE) is performed with a miniature swallowable endoscope enabling the visualization of the whole gastrointestinal (GI) tract. One of the most challenging problems in WCE is the localization of the capsule endoscope (CE) within the GI lumen. Contemporary, radiation-free localization approaches are mainly based on the use of external sensors and transit time estimation techniques, with practically low localization accuracy. Latest advances for the solution of this problem include localization approaches based solely on visual information from the CE camera. In this paper we present a novel visual localization approach based on an intelligent, artificial neural network, architecture which implements a generic visual odometry (VO) framework capable of estimating the motion of the CE in physical units. Unlike the conventional, geometric, VO approaches, the proposed one is adaptive to the geometric model of the CE used; therefore, it does not require any prior knowledge about and its intrinsic parameters. Furthermore, it exploits color as a cue to increase localization accuracy and robustness. Experiments were performed using a robotic-assisted setup providing ground truth information about the actual location of the CE. The lowest average localization error achieved is 2.70 ± 1.62 cm, which is significantly lower than the error obtained with the geometric approach. This result constitutes a promising step towards the in-vivo application of VO, which will open new horizons for accurate local treatment, including drug infusion and surgical interventions.

KID Project: an internet-based digital video atlas of capsule endoscopy for research purposes.

BACKGROUND AND AIMS: Capsule endoscopy (CE) has revolutionized small-bowel (SB) investigation. Computational methods can enhance diagnostic yield (DY); however, incorporating machine learning algorithms (MLAs) into CE reading is difficult as large amounts of image annotations are required for training. Current databases lack graphic annotations of pathologies and cannot be used. A novel database, KID, aims to provide a reference for research and development of medical decision support systems (MDSS) for CE. METHODS: Open-source software was used for the KID database. Clinicians contribute anonymized, annotated CE images and videos. Graphic annotations are supported by an open-access annotation tool (Ratsnake). We detail an experiment based on the KID database, examining differences in SB lesion measurement between human readers and a MLA. The Jaccard Index (JI) was used to evaluate similarity between annotations by the MLA and human readers. RESULTS: The MLA performed best in measuring lymphangiectasias with a JI of 81 ±â€Š6 %. The other lesion types were: angioectasias (JI 64 ±â€Š11 %), aphthae (JI 64 ±â€Š8 %), chylous cysts (JI 70 ±â€Š14 %), polypoid lesions (JI 75 ±â€Š21 %), and ulcers (JI 56 ±â€Š9 %). CONCLUSION: MLA can perform as well as human readers in the measurement of SB angioectasias in white light (WL). Automated lesion measurement is therefore feasible. KID is currently the only open-source CE database developed specifically to aid development of MDSS. Our experiment demonstrates this potential.

Wavelet-based signal analysis for heart failure hospitalization prediction.

Heart failure (HF) is commonly a chronic condition associated with frequent hospital admissions. Early knowledge about a possible deterioration of this condition would enable early treatment for the prevention of adverse events and related hospital admissions. In this paper we present a computational method for predictive information extraction from daily physiological signals, which can be obtained by a telemonitoring system with wearable sensors. It is based on wavelet analysis of temporal signal patterns. Experiments with data from patients enrolled in a telemonitoring protocol show that the proposed method is capable of predicting HF hospitalization events one day before they happen, even in the case of low compliance to the protocol. These results indicate a promising perspective towards a monitoring system that would provide improved life quality for HF patients.

Comparative assessment of feature extraction methods for visual odometry in wireless capsule endoscopy.

Wireless capsule endoscopy (WCE) enables the non-invasive examination of the gastrointestinal (GI) tract by a swallowable device equipped with a miniature camera. Accurate localization of the capsule in the GI tract enables accurate localization of abnormalities for medical interventions such as biopsy and polyp resection; therefore, the optimization of the localization outcome is important. Current approaches to endoscopic capsule localization are mainly based on external sensors and transit time estimations. Recently, we demonstrated the feasibility of capsule localization based-entirely-on visual features, without the use of external sensors. This technique relies on a motion estimation algorithm that enables measurements of the distance and the rotation of the capsule from the acquired video frames. Towards the determination of an optimal visual feature extraction technique for capsule motion estimation, an extensive comparative assessment of several state-of-the-art techniques, using a publicly available dataset, is presented. The results show that the minimization of the localization error is possible at the cost of computational efficiency. A localization error of approximately one order of magnitude higher than the minimal one can be considered as compromise for the use of current computationally efficient feature extraction techniques.

Wireless endoscopy in 2020: Will it still be a capsule?

Currently, the major problem of all existing commercial capsule devices is the lack of control of movement. In the future, with an interface application, the clinician will be able to stop and direct the device into points of interest for detailed inspection/diagnosis, and therapy delivery. This editorial presents current commercially-available new designs, European projects and delivery capsule and gives an overview of the progress required and progress that will be achieved -according to the opinion of the authors- in the next 5 year leading to 2020.