Robust Shape Fitting for 3D Scene Abstraction.

Humans perceive and construct the world as an arrangement of simple parametric models. In particular, we can often describe man-made environments using volumetric primitives such as cuboids or cylinders. Inferring these primitives is important for attaining high-level, abstract scene descriptions. Previous approaches for primitive-based abstraction estimate shape parameters directly and are only able to reproduce simple objects. In contrast, we propose a robust estimator for primitive fitting, which meaningfully abstracts complex real-world environments using cuboids. A RANSAC estimator guided by a neural network fits these primitives to a depth map. We condition the network on previously detected parts of the scene, parsing it one-by-one. To obtain cuboids from single RGB images, we additionally optimise a depth estimation CNN end-to-end. Naively minimising point-to-primitive distances leads to large or spurious cuboids occluding parts of the scene. We thus propose an improved occlusion-aware distance metric correctly handling opaque scenes. Furthermore, we present a neural network based cuboid solver which provides more parsimonious scene abstractions while also reducing inference time. The proposed algorithm does not require labour-intensive labels, such as cuboid annotations, for training. Results on the NYU Depth v2 dataset demonstrate that the proposed algorithm successfully abstracts cluttered real-world 3D scene layouts.

RelTR: Relation Transformer for Scene Graph Generation.

Different objects in the same scene are more or less related to each other, but only a limited number of these relationships are noteworthy. Inspired by Detection Transformer, which excels in object detection, we view scene graph generation as a set prediction problem. In this article, we propose an end-to-end scene graph generation model Relation Transformer (RelTR), which has an encoder-decoder architecture. The encoder reasons about the visual feature context while the decoder infers a fixed-size set of triplets subject-predicate-object using different types of attention mechanisms with coupled subject and object queries. We design a set prediction loss performing the matching between the ground truth and predicted triplets for the end-to-end training. In contrast to most existing scene graph generation methods, RelTR is a one-stage method that predicts sparse scene graphs directly only using visual appearance without combining entities and labeling all possible predicates. Extensive experiments on the Visual Genome, Open Images V6, and VRD datasets demonstrate the superior performance and fast inference of our model.

Semantic segmentation for fully automated macrofouling analysis on coatings after field exposure.

Biofouling is a major challenge for sustainable shipping, filter membranes, heat exchangers, and medical devices. The development of fouling-resistant coatings requires the evaluation of their effectiveness. Such an evaluation is usually based on the assessment of fouling progression after different exposure times to the target medium (e.g. salt water). The manual assessment of macrofouling requires expert knowledge about local fouling communities due to high variances in phenotypical appearance, has single-image sampling inaccuracies for certain species, and lacks spatial information. Here an approach for automatic image-based macrofouling analysis was presented. A dataset with dense labels prepared from field panel images was made and a convolutional network (adapted U-Net) for the semantic segmentation of different macrofouling classes was proposed. The establishment of macrofouling localization allows for the generation of a successional model which enables the determination of direct surface attachment and in-depth epibiotic studies.

Wochenende - modular and flexible alignment-based shotgun metagenome analysis.

BACKGROUND: Shotgun metagenome analysis provides a robust and verifiable method for comprehensive microbiome analysis of fungal, viral, archaeal and bacterial taxonomy, particularly with regard to visualization of read mapping location, normalization options, growth dynamics and functional gene repertoires. Current read classification tools use non-standard output formats, or do not fully show information on mapping location. As reference datasets are not perfect, portrayal of mapping information is critical for judging results effectively. RESULTS: Our alignment-based pipeline, Wochenende, incorporates flexible quality control, trimming, mapping, various filters and normalization. Results are completely transparent and filters can be adjusted by the user. We observe stringent filtering of mismatches and use of mapping quality sharply reduces the number of false positives. Further modules allow genomic visualization and the calculation of growth rates, as well as integration and subsequent plotting of pipeline results as heatmaps or heat trees. Our novel normalization approach additionally allows calculation of absolute abundance profiles by comparison with reads assigned to the human host genome. CONCLUSION: Wochenende has the ability to find and filter alignments to all kingdoms of life using both short and long reads, and requires only good quality reference genomes. Wochenende automatically combines multiple available modules ranging from quality control and normalization to taxonomic visualization. Wochenende is available at https://github.com/MHH-RCUG/nf_wochenende .

Animal models and animal-free innovations for cardiovascular research: current status and routes to be explored. Consensus document of the ESC Working Group on Myocardial Function and the ESC Working Group on Cellular Biology of the Heart.

Cardiovascular diseases represent a major cause of morbidity and mortality, necessitating research to improve diagnostics, and to discover and test novel preventive and curative therapies, all of which warrant experimental models that recapitulate human disease. The translation of basic science results to clinical practice is a challenging task, in particular for complex conditions such as cardiovascular diseases, which often result from multiple risk factors and comorbidities. This difficulty might lead some individuals to question the value of animal research, citing the translational 'valley of death', which largely reflects the fact that studies in rodents are difficult to translate to humans. This is also influenced by the fact that new, human-derived in vitro models can recapitulate aspects of disease processes. However, it would be a mistake to think that animal models do not represent a vital step in the translational pathway as they do provide important pathophysiological insights into disease mechanisms particularly on an organ and systemic level. While stem cell-derived human models have the potential to become key in testing toxicity and effectiveness of new drugs, we need to be realistic, and carefully validate all new human-like disease models. In this position paper, we highlight recent advances in trying to reduce the number of animals for cardiovascular research ranging from stem cell-derived models to in situ modelling of heart properties, bioinformatic models based on large datasets, and state-of-the-art animal models, which show clinically relevant characteristics observed in patients with a cardiovascular disease. We aim to provide a guide to help researchers in their experimental design to translate bench findings to clinical routine taking the replacement, reduction, and refinement (3R) as a guiding concept.

Accurate Long-Term Multiple People Tracking using Video and Body-Worn IMUs.

Most modern approaches for video-based multiple people tracking rely on human appearance to exploit similarities between person detections. Consequently, tracking accuracy degrades if this kind of information is not discriminative or if people change apparel. In contrast, we present a method to fuse video information with additional motion signals from body-worn inertial measurement units (IMUs). In particular, we propose a neural network to relate person detections with IMU orientations, and formulate a graph labeling problem to obtain a tracking solution that is globally consistent with the video and inertial recordings. The fusion of visual and inertial cues provides several advantages. The association of detection boxes in the video and IMU devices is based on motion, which is independent of a person's outward appearance. Furthermore, inertial sensors provide motion information irrespective of visual occlusions. Hence, once detections in the video are associated with an IMU device, intermediate positions can be reconstructed from corresponding inertial sensor data, which would be unstable using video only. Since no dataset exists for this new setting, we release a dataset of challenging tracking sequences, containing video and IMU recordings together with ground-truth annotations. We evaluate our approach on our new dataset, achieving an average IDF1 score of 91.2%. The proposed method is applicable to any situation that allows one to equip people with inertial sensors.

Fully Convolutional Neural Network for Detection and Counting of Diatoms on Coatings after Short-Term Field Exposure.

While the use of deep learning is a valuable technology for automatic detection systems for medical data and images, the biofouling community is still lacking an analytical tool for the detection and counting of diatoms on samples after short-term field exposure. In this work, a fully convolutional neural network was implemented as a fast and simple approach to detect diatoms on two-channel (fluorescence and phase-contrast) microscopy images by predicting bounding boxes. The developed approach performs well with only a small number of trainable parameters and a F1 score of 0.82. Counting diatoms was evaluated on a data set of 600 microscopy images of three different surface chemistries (hydrophilic and hydrophobic) and is very similar to counting by humans while demanding only a fraction of the analysis time.

Occlusion-aware Method for Temporally Consistent Superpixels.

A wide variety of computer vision applications rely on superpixel or supervoxel algorithms as a preprocessing step. This underlines the overall importance that these approaches have gained in recent years. However, most methods show a lack of temporal consistency or fail in producing temporally stable superpixels. In this paper, we present an approach to generate temporally consistent superpixels for video content. Our method is formulated as a contour-evolving expectation-maximization framework, which utilizes an efficient label propagation scheme to encourage the preservation of superpixel shapes and their relative positioning over time. By explicitly detecting the occlusion of superpixels and the disocclusion of new image regions, our framework is able to terminate and create superpixels whose corresponding image region becomes hidden or newly appears. Additionally, the occluded parts of superpixels are incorporated in the further optimization. This increases the compliance of the superpixel flow with the optical flow present in the scene. Using established benchmark suites, we show the performance of our approach in comparison to state-of-the-art supervoxel and superpixel algorithms for video content.

Inner tegument proteins of Herpes Simplex Virus are sufficient for intracellular capsid motility in neurons but not for axonal targeting.

Upon reactivation from latency and during lytic infections in neurons, alphaherpesviruses assemble cytosolic capsids, capsids associated with enveloping membranes, and transport vesicles harboring fully enveloped capsids. It is debated whether capsid envelopment of herpes simplex virus (HSV) is completed in the soma prior to axonal targeting or later, and whether the mechanisms are the same in neurons derived from embryos or from adult hosts. We used HSV mutants impaired in capsid envelopment to test whether the inner tegument proteins pUL36 or pUL37 necessary for microtubule-mediated capsid transport were sufficient for axonal capsid targeting in neurons derived from the dorsal root ganglia of adult mice. Such neurons were infected with HSV1-ΔUL20 whose capsids recruited pUL36 and pUL37, with HSV1-ΔUL37 whose capsids associate only with pUL36, or with HSV1-ΔUL36 that assembles capsids lacking both proteins. While capsids of HSV1-ΔUL20 were actively transported along microtubules in epithelial cells and in the somata of neurons, those of HSV1-ΔUL36 and -ΔUL37 could only diffuse in the cytoplasm. Employing a novel image analysis algorithm to quantify capsid targeting to axons, we show that only a few capsids of HSV1-ΔUL20 entered axons, while vesicles transporting gD utilized axonal transport efficiently and independently of pUL36, pUL37, or pUL20. Our data indicate that capsid motility in the somata of neurons mediated by pUL36 and pUL37 does not suffice for targeting capsids to axons, and suggest that capsid envelopment needs to be completed in the soma prior to targeting of herpes simplex virus to the axons, and to spreading from neurons to neighboring cells.

Antipodally Invariant Metrics for Fast Regression-Based Super-Resolution.

Dictionary-based super-resolution (SR) algorithms usually select dictionary atoms based on the distance or similarity metrics. Although the optimal selection of the nearest neighbors is of central importance for such methods, the impact of using proper metrics for SR has been overlooked in literature, mainly due to the vast usage of Euclidean distance. In this paper, we present a very fast regression-based algorithm, which builds on the densely populated anchored neighborhoods and sublinear search structures. We perform a study of the nature of the features commonly used for SR, observing that those features usually lie in the unitary hypersphere, where every point has a diametrically opposite one, i.e., its antipode, with same module and angle, but the opposite direction. Even though, we validate the benefits of using antipodally invariant metrics, most of the binary splits use Euclidean distance, which does not handle antipodes optimally. In order to benefit from both the worlds, we propose a simple yet effective antipodally invariant transform that can be easily included in the Euclidean distance calculation. We modify the original spherical hashing algorithm with this metric in our antipodally invariant spherical hashing scheme, obtaining the same performance as a pure antipodally invariant metric. We round up our contributions with a novel feature transform that obtains a better coarse approximation of the input image thanks to iterative backprojection. The performance of our method, which we named antipodally invariant SR, improves quality (Peak Signal to Noise Ratio) and it is faster than any other state-of-the-art method.

3D Reconstruction of Human Motion from Monocular Image Sequences.

This article tackles the problem of estimating non-rigid human 3D shape and motion from image sequences taken by uncalibrated cameras. Similar to other state-of-the-art solutions we factorize 2D observations in camera parameters, base poses and mixing coefficients. Existing methods require sufficient camera motion during the sequence to achieve a correct 3D reconstruction. To obtain convincing 3D reconstructions from arbitrary camera motion, our method is based on a-priorly trained base poses. We show that strong periodic assumptions on the coefficients can be used to define an efficient and accurate algorithm for estimating periodic motion such as walking patterns. For the extension to non-periodic motion we propose a novel regularization term based on temporal bone length constancy. In contrast to other works, the proposed method does not use a predefined skeleton or anthropometric constraints and can handle arbitrary camera motion. We achieve convincing 3D reconstructions, even under the influence of noise and occlusions. Multiple experiments based on a 3D error metric demonstrate the stability of the proposed method. Compared to other state-of-the-art methods our algorithm shows a significant improvement.

Human Pose Estimation from Video and IMUs.

In this work, we present an approach to fuse video with sparse orientation data obtained from inertial sensors to improve and stabilize full-body human motion capture. Even though video data is a strong cue for motion analysis, tracking artifacts occur frequently due to ambiguities in the images, rapid motions, occlusions or noise. As a complementary data source, inertial sensors allow for accurate estimation of limb orientations even under fast motions. However, accurate position information cannot be obtained in continuous operation. Therefore, we propose a hybrid tracker that combines video with a small number of inertial units to compensate for the drawbacks of each sensor type: on the one hand, we obtain drift-free and accurate position information from video data and, on the other hand, we obtain accurate limb orientations and good performance under fast motions from inertial sensors. In several experiments we demonstrate the increased performance and stability of our human motion tracker.

The Kinect Recording System for objective three- and four-dimensional breast assessment with image overlays.

INTRODUCTION: We investigated the application of the validated portable Kinect camera for three- and four-dimensional breast assessment in female life models. METHOD: Breast images from six life models were captured using the Kinect camera. Capture was conducted with taking three different arm positions while standing upright: with the arms straight down, straight up to the side at 90° and straight all the way up. Images of the volunteers were superimposed on each other. Digital linear distances between sternal notch and nipple-areola complexes were obtained and compared. The views of plastic and breast surgeons to arm positions were questioned. An example for clinical application was provided. RESULTS: Successful capture of images of the female life breast models was achieved. Digital breast measurements at the three different arm positions revealed considerable variation in linear distances measured on the images obtained with the Kinect camera. The dynamic of breast movements could be demonstrated by image overlay and the first ever four-dimensional breast assessment was demonstrated. Fourteen plastic and breast surgeons were found to have nine different opinions regarding their favoured arm positions for breast capture. Even though precision of image sharpness still needs improvement, the images were satisfactory for clinical patient use. The Kinect data were shown to be applicable to surgery planning by designing a planar flap from the 3D mesh. CONCLUSION: The portable and low-cost Kinect camera proved to be easy to use for the first application in life models for three- and four-dimensional breast assessment.

Brain tumor classification and segmentation using sparse coding and dictionary learning.

This paper presents a novel fully automatic framework for multi-class brain tumor classification and segmentation using a sparse coding and dictionary learning method. The proposed framework consists of two steps: classification and segmentation. The classification of the brain tumors is based on brain topology and texture. The segmentation is based on voxel values of the image data. Using K-SVD, two types of dictionaries are learned from the training data and their associated ground truth segmentation: feature dictionary and voxel-wise coupled dictionaries. The feature dictionary consists of global image features (topological and texture features). The coupled dictionaries consist of coupled information: gray scale voxel values of the training image data and their associated label voxel values of the ground truth segmentation of the training data. For quantitative evaluation, the proposed framework is evaluated using different metrics. The segmentation results of the brain tumor segmentation (MICCAI-BraTS-2013) database are evaluated using five different metric scores, which are computed using the online evaluation tool provided by the BraTS-2013 challenge organizers. Experimental results demonstrate that the proposed approach achieves an accurate brain tumor classification and segmentation and outperforms the state-of-the-art methods.

Automatic tracking of vessel-like structures from a single starting point.

The identification of vascular networks is an important topic in the medical image analysis community. While most methods focus on single vessel tracking, the few solutions that exist for tracking complete vascular networks are usually computationally intensive and require a lot of user interaction. In this paper we present a method to track full vascular networks iteratively using a single starting point. Our approach is based on a cloud of sampling points distributed over concentric spherical layers. We also proposed a vessel model and a metric of how well a sample point fits this model. Then, we implement the network tracking as a min-cost flow problem, and propose a novel optimization scheme to iteratively track the vessel structure by inherently handling bifurcations and paths. The method was tested using both synthetic and real images. On the 9 different data-sets of synthetic blood vessels, we achieved maximum accuracies of more than 98%. We further use the synthetic data-set to analyze the sensibility of our method to parameter setting, showing the robustness of the proposed algorithm. For real images, we used coronary, carotid and pulmonary data to segment vascular structures and present the visual results. Still for real images, we present numerical and visual results for networks of nerve fibers in the olfactory system. Further visual results also show the potential of our approach for identifying vascular networks topologies. The presented method delivers good results for the several different datasets tested and have potential for segmenting vessel-like structures. Also, the topology information, inherently extracted, can be used for further analysis to computed aided diagnosis and surgical planning. Finally, the method's modular aspect holds potential for problem-oriented adjustments and improvements.

3D automatic liver segmentation using feature-constrained Mahalanobis distance in CT images.

Automatic 3D liver segmentation is a fundamental step in the liver disease diagnosis and surgery planning. This paper presents a novel fully automatic algorithm for 3D liver segmentation in clinical 3D computed tomography (CT) images. Based on image features, we propose a new Mahalanobis distance cost function using an active shape model (ASM). We call our method MD-ASM. Unlike the standard active shape model (ST-ASM), the proposed method introduces a new feature-constrained Mahalanobis distance cost function to measure the distance between the generated shape during the iterative step and the mean shape model. The proposed Mahalanobis distance function is learned from a public database of liver segmentation challenge (MICCAI-SLiver07). As a refinement step, we propose the use of a 3D graph-cut segmentation. Foreground and background labels are automatically selected using texture features of the learned Mahalanobis distance. Quantitatively, the proposed method is evaluated using two clinical 3D CT scan databases (MICCAI-SLiver07 and MIDAS). The evaluation of the MICCAI-SLiver07 database is obtained by the challenge organizers using five different metric scores. The experimental results demonstrate the availability of the proposed method by achieving an accurate liver segmentation compared to the state-of-the-art methods.

Alzheimer's disease detection via automatic 3D caudate nucleus segmentation using coupled dictionary learning with level set formulation.

BACKGROUND AND OBJECTIVE: This paper presents a novel method for Alzheimer's disease classification via an automatic 3D caudate nucleus segmentation. METHODS: The proposed method consists of segmentation and classification steps. In the segmentation step, we propose a novel level set cost function. The proposed cost function is constrained by a sparse representation of local image features using a dictionary learning method. We present coupled dictionaries: a feature dictionary of a grayscale brain image and a label dictionary of a caudate nucleus label image. Using online dictionary learning, the coupled dictionaries are learned from the training data. The learned coupled dictionaries are embedded into a level set function. In the classification step, a region-based feature dictionary is built. The region-based feature dictionary is learned from shape features of the caudate nucleus in the training data. The classification is based on the measure of the similarity between the sparse representation of region-based shape features of the segmented caudate in the test image and the region-based feature dictionary. RESULTS: The experimental results demonstrate the superiority of our method over the state-of-the-art methods by achieving a high segmentation (91.5%) and classification (92.5%) accuracy. CONCLUSIONS: In this paper, we find that the study of the caudate nucleus atrophy gives an advantage over the study of whole brain structure atrophy to detect Alzheimer's disease.

Method for 3D airway topology extraction.

In lungs the number of conducting airway generations as well as bifurcation patterns varies across species and shows specific characteristics relating to illnesses or gene variations. A method to characterize the topology of the mouse airway tree using scanning laser optical tomography (SLOT) tomograms is presented in this paper. It is used to test discrimination between two types of mice based on detected differences in their conducting airway pattern. Based on segmentations of the airways in these tomograms, the main spanning tree of the volume skeleton is computed. The resulting graph structure is used to distinguish between wild type and surfactant protein (SP-D) deficient knock-out mice.

Multi-region labeling and segmentation using a graph topology prior and atlas information in brain images.

Medical image segmentation and anatomical structure labeling according to the types of the tissues are important for accurate diagnosis and therapy. In this paper, we propose a novel approach for multi-region labeling and segmentation, which is based on a topological graph prior and the topological information of an atlas, using a modified multi-level set energy minimization method in brain images. We consider a topological graph prior and atlas information to evolve the contour based on a topological relationship presented via a graph relation. This novel method is capable of segmenting adjacent objects with very close gray level in low resolution brain image that would be difficult to segment correctly using standard methods. The topological information of an atlas are transformed to the topological graph of a low resolution (noisy) brain image to obtain region labeling. We explain our algorithm and show the topological graph prior and label transformation techniques to explain how it gives precise multi-region segmentation and labeling. The proposed algorithm is capable of segmenting and labeling different regions in noisy or low resolution MRI brain images of different modalities. We compare our approaches with other state-of-the-art approaches for multi-region labeling and segmentation.

Validation of the Kinect device as a new portable imaging system for three-dimensional breast assessment.

AIM: The aim of this study was the evaluation of a new, simple, touchless, low-cost and portable three-dimensional (3D) measurement system for objective breast assessment. METHOD: The Kinect Recording System by Microsoft was used. Coloured and depth images were captured of nine silicone breast implants of known volumes. The data were processed using Matlab(®) software. Volume measurements were obtained in a blinded calculation on the 3D images. For further comparison, implant volumes were assessed with the Arthur Morris device, a manual measurement tool. RESULTS: Four tests revealed that the true breast implant volumes were calculated within an error margin of 10%. Reproducibility of measurements was satisfactory. Overall, the accuracy and reproducibility of the measurements of the Kinect System were better than those of the Arthur Morris device. Accuracy of volume assessments with the Kinect System was satisfactory for clinical application. Our new portable 3D imaging system was successfully validated. DISCUSSION: The portable and easy-to-use system has several advantages against the currently available commercial systems. Despite a slight overestimation of the volume data, we felt that these results were very promising due to the repeatability of the measurements. After validating the measurement accuracy of the system in a simpler case, we aim to conduct further studies on 3D breast assessment. CONCLUSION: The results obtained with the Kinect System were sufficiently accurate and reproducible for application in 3D breast capture. We successfully validated the portable 3D imaging system for the first ever use in 3D breast assessment.