Integrating Egocentric and Robotic Vision for Object Identification Using Siamese Networks and Superquadric Estimations in Partial Occlusion Scenarios.

This paper introduces a novel method that enables robots to identify objects based on user gaze, tracked via eye-tracking glasses. This is achieved without prior knowledge of the objects' categories or their locations and without external markers. The method integrates a two-part system: a category-agnostic object shape and pose estimator using superquadrics and Siamese networks. The superquadrics-based component estimates the shapes and poses of all objects, while the Siamese network matches the object targeted by the user's gaze with the robot's viewpoint. Both components are effectively designed to function in scenarios with partial occlusions. A key feature of the system is the user's ability to move freely around the scenario, allowing dynamic object selection via gaze from any position. The system is capable of handling significant viewpoint differences between the user and the robot and adapts easily to new objects. In tests under partial occlusion conditions, the Siamese networks demonstrated an 85.2% accuracy in aligning the user-selected object with the robot's viewpoint. This gaze-based Human-Robot Interaction approach demonstrates its practicality and adaptability in real-world scenarios.

URI-CADS: A Fully Automated Computer-Aided Diagnosis System for Ultrasound Renal Imaging.

Ultrasound is a widespread imaging modality, with special application in medical fields such as nephrology. However, automated approaches for ultrasound renal interpretation still pose some challenges: (1) the need for manual supervision by experts at various stages of the system, which prevents its adoption in primary healthcare, and (2) their limited considered taxonomy (e.g., reduced number of pathologies), which makes them unsuitable for training practitioners and providing support to experts. This paper proposes a fully automated computer-aided diagnosis system for ultrasound renal imaging addressing both of these challenges. Our system is based in a multi-task architecture, which is implemented by a three-branched convolutional neural network and is capable of segmenting the kidney and detecting global and local pathologies with no need of human interaction during diagnosis. The integration of different image perspectives at distinct granularities enhanced the proposed diagnosis. We employ a large (1985 images) and demanding ultrasound renal imaging database, publicly released with the system and annotated on the basis of an exhaustive taxonomy of two global and nine local pathologies (including cysts, lithiasis, hydronephrosis, angiomyolipoma), establishing a benchmark for ultrasound renal interpretation. Experiments show that our proposed method outperforms several state-of-the-art methods in both segmentation and diagnosis tasks and leverages the combination of global and local image information to improve the diagnosis. Our results, with a 87.41% of AUC in healthy-pathological diagnosis and 81.90% in multi-pathological diagnosis, support the use of our system as a helpful tool in the healthcare system.

An interpretable CNN-based CAD system for skin lesion diagnosis.

Recently, convolutional neural networks have greatly outperformed previous systems based on handcrafted features once the size of public databases has increased. However, these algorithms learn feature representations that are difficult to interpret and analyse. On the other hand, experts require automatic systems to explain their decisions according to clinical criteria which, in the field of melanoma diagnosis, are related to the analysis of dermoscopic features found in the lesions. In recent years, the interpretability of deep networks has been explored using methods that obtain visual features highlighted by neurones or analyse activations to extract more useful information. Following the latter approach, this study proposes a system for melanoma diagnosis that explicitly incorporates dermoscopic feature segmentations into a diagnosis network through a channel modulation scheme. Modulation weights control the influence of the detected visual patterns based on the lesion content. As shown in the experimental section, our design not only improves the system performance on the ISIC 2016 (average AUC of 86.6% vs. 85.8%) and 2017 (average AUC of 94.0% vs. 93.8%) datasets, but also notably enhances the interpretability of the diagnosis, providing useful and intuitive cues to clinicians.

Behavioural immune landscapes of inflammation.

Transcriptional and proteomic profiling of individual cells have revolutionized interpretation of biological phenomena by providing cellular landscapes of healthy and diseased tissues1,2. These approaches, however, do not describe dynamic scenarios in which cells continuously change their biochemical properties and downstream 'behavioural' outputs3-5. Here we used 4D live imaging to record tens to hundreds of morpho-kinetic parameters describing the dynamics of individual leukocytes at sites of active inflammation. By analysing more than 100,000 reconstructions of cell shapes and tracks over time, we obtained behavioural descriptors of individual cells and used these high-dimensional datasets to build behavioural landscapes. These landscapes recognized leukocyte identities in the inflamed skin and trachea, and uncovered a continuum of neutrophil states inside blood vessels, including a large, sessile state that was embraced by the underlying endothelium and associated with pathogenic inflammation. Behavioural screening in 24 mouse mutants identified the kinase Fgr as a driver of this pathogenic state, and interference with Fgr protected mice from inflammatory injury. Thus, behavioural landscapes report distinct properties of dynamic environments at high cellular resolution.

ACME: Automatic feature extraction for cell migration examination through intravital microscopy imaging.

Cell detection and tracking applied to in vivo fluorescence microscopy has become an essential tool in biomedicine to characterize 4D (3D space plus time) biological processes at the cellular level. Traditional approaches to cell motion analysis by microscopy imaging, although based on automatic frameworks, still require manual supervision at some points of the system. Hence, when dealing with a large amount of data, the analysis becomes incredibly time-consuming and typically yields poor biological information. In this paper, we propose a fully-automated system for segmentation, tracking and feature extraction of migrating cells within blood vessels in 4D microscopy imaging. Our system consists of a robust 3D convolutional neural network (CNN) for joint blood vessel and cell segmentation, a 3D tracking module with collision handling, and a novel method for feature extraction, which takes into account the particular geometry in the cell-vessel arrangement. Experiments on a large 4D intravital microscopy dataset show that the proposed system achieves a significantly better performance than the state-of-the-art tools for cell segmentation and tracking. Furthermore, we have designed an analytical method of cell behaviors based on the automatically extracted features, which supports the hypotheses related to leukocyte migration posed by expert biologists. This is the first time that such a comprehensive automatic analysis of immune cell migration has been performed, where the total population under study reaches hundreds of neutrophils and thousands of time instances.

A new distance measure for model-based sequence clustering.

We review the existing alternatives for defining model-based distances for clustering sequences and propose a new one based on the Kullback-Leibler divergence. This distance is shown to be especially useful in combination with spectral clustering. For improved performance in real-world scenarios, a model selection scheme is also proposed.