Image capturing, segmentation and data analysis of shredded refuse streams.

Refuse sorting is an important cornerstone of the recycling industry, but ever-changing refuse compositions and the desire to increase recycling rates still pose many unsolved challenges. The digitalisation of refuse sorting plants promises to overcome these challenges by optimising and automatically adapting the sorting process. This publication describes a system for image capturing, segmentation-based refuse recognition and data analysis of shredded refuse streams. The image capturing collects multispectral 2D and 3D images of the refuse streams on conveyor belts. The image recognition performs a semantic segmentation of the images to determine the refuse composition from the 2D images, whereas the 3D images approximate the volumes on the conveyor belts. The semantic segmentation is done by a combined convolutional neural network model, consisting of a foreground-background and a refuse class segmentation. Both models rely on synthetic training data to reduce the necessary amount of manually labelled training data, whereas the final segmentation performance reaches an Intersection over Union of up to 75%. The results of the semantic segmentation and volume estimation are combined with data of the shredding machinery by transforming it into a unified representation. This combined dataset is the basis for estimating the processed refuse masses from the semantic segmentation and volume estimation.

Automatic Detection of Periapical Osteolytic Lesions on Cone-beam Computed Tomography Using Deep Convolutional Neuronal Networks.

INTRODUCTION: Cone-beam computed tomography (CBCT) is an essential diagnostic tool in oral radiology. Radiolucent periapical lesions (PALs) represent the most frequent jaw lesions. However, the description, interpretation, and documentation of radiological findings, especially incidental findings, are time-consuming and resource-intensive, requiring a high degree of expertise. To improve quality, dentists may use artificial intelligence in the form of deep learning tools. This study was conducted to develop and validate a deep convolutional neuronal network for the automated detection of osteolytic PALs in CBCT data sets. METHODS: CBCT data sets from routine clinical operations (maxilla, mandible, or both) performed from January to October 2020 were retrospectively screened and selected. A 2-step approach was used for automatic PAL detection. First, tooth localization and identification were performed using the SpatialConfiguration-Net based on heatmap regression. Second, binary segmentation of lesions was performed using a modified U-Net architecture. A total of 144 CBCT images were used to train and test the networks. The method was evaluated using the 4-fold cross-validation technique. RESULTS: The success detection rate of the tooth localization network ranged between 72.6% and 97.3%, whereas the sensitivity and specificity values of lesion detection were 97.1% and 88.0%, respectively. CONCLUSIONS: Although PALs showed variations in appearance, size, and shape in the CBCT data set and a high imbalance existed between teeth with and without PALs, the proposed fully automated method provided excellent results compared with related literature.

Stereoscopic motion analysis in densely packed clusters: 3D analysis of the shimmering behaviour in Giant honey bees.

BACKGROUND: The detailed interpretation of mass phenomena such as human escape panic or swarm behaviour in birds, fish and insects requires detailed analysis of the 3D movements of individual participants. Here, we describe the adaptation of a 3D stereoscopic imaging method to measure the positional coordinates of individual agents in densely packed clusters. The method was applied to study behavioural aspects of shimmering in Giant honeybees, a collective defence behaviour that deters predatory wasps by visual cues, whereby individual bees flip their abdomen upwards in a split second, producing Mexican wave-like patterns. RESULTS: Stereoscopic imaging provided non-invasive, automated, simultaneous, in-situ 3D measurements of hundreds of bees on the nest surface regarding their thoracic position and orientation of the body length axis. Segmentation was the basis for the stereo matching, which defined correspondences of individual bees in pairs of stereo images. Stereo-matched "agent bees" were re-identified in subsequent frames by the tracking procedure and triangulated into real-world coordinates. These algorithms were required to calculate the three spatial motion components (dx: horizontal, dy: vertical and dz: towards and from the comb) of individual bees over time. CONCLUSIONS: The method enables the assessment of the 3D positions of individual Giant honeybees, which is not possible with single-view cameras. The method can be applied to distinguish at the individual bee level active movements of the thoraces produced by abdominal flipping from passive motions generated by the moving bee curtain. The data provide evidence that the z-deflections of thoraces are potential cues for colony-intrinsic communication. The method helps to understand the phenomenon of collective decision-making through mechanoceptive synchronization and to associate shimmering with the principles of wave propagation. With further, minor modifications, the method could be used to study aspects of other mass phenomena that involve active and passive movements of individual agents in densely packed clusters.

Deep Metric Learning with BIER: Boosting Independent Embeddings Robustly.

Learning similarity functions between image pairs with deep neural networks yields highly correlated activations of embeddings. In this work, we show how to improve the robustness of such embeddings by exploiting the independence within ensembles. To this end, we divide the last embedding layer of a deep network into an embedding ensemble and formulate the task of training this ensemble as an online gradient boosting problem. Each learner receives a reweighted training sample from the previous learners. Further, we propose two loss functions which increase the diversity in our ensemble. These loss functions can be applied either for weight initialization or during training. Together, our contributions leverage large embedding sizes more effectively by significantly reducing correlation of the embedding and consequently increase retrieval accuracy of the embedding. Our method works with any differentiable loss function and does not introduce any additional parameters during test time. We evaluate our metric learning method on image retrieval tasks and show that it improves over state-of-the-art methods on the CUB-200-2011, Cars-196, Stanford Online Products, In-Shop Clothes Retrieval and VehicleID datasets. Therefore, our findings suggest that by dividing deep networks at the end into several smaller and diverse networks, we can significantly reduce overfitting.

A practical procedure for real-time functional mapping of eloquent cortex using electrocorticographic signals in humans.

Functional mapping of eloquent cortex is often necessary prior to invasive brain surgery, but current techniques that derive this mapping have important limitations. In this article, we demonstrate the first comprehensive evaluation of a rapid, robust, and practical mapping system that uses passive recordings of electrocorticographic signals. This mapping procedure is based on the BCI2000 and SIGFRIED technologies that we have been developing over the past several years. In our study, we evaluated 10 patients with epilepsy from four different institutions and compared the results of our procedure with the results derived using electrical cortical stimulation (ECS) mapping. The results show that our procedure derives a functional motor cortical map in only a few minutes. They also show a substantial concurrence with the results derived using ECS mapping. Specifically, compared with ECS maps, a next-neighbor evaluation showed no false negatives, and only 0.46 and 1.10% false positives for hand and tongue maps, respectively. In summary, we demonstrate the first comprehensive evaluation of a practical and robust mapping procedure that could become a new tool for planning of invasive brain surgeries.

A phenomenological approach toward patient-specific computational modeling of articular cartilage including collagen fiber tracking.

To model the cartilage morphology and the material response, a phenomenological and patient-specific simulation approach incorporating the collagen fiber fabric is proposed. Cartilage tissue response is nearly isochoric and time-dependent under physiological pressure levels. Hence, a viscoelastic constitutive model capable of reproducing finite strains is employed, while the time-dependent deformation change is purely isochoric. The model incorporates seven material parameters, which all have a physical interpretation. To calibrate the model and facilitate further analysis, five human cartilage specimens underwent a number of tests. A series of magnetic resonance imaging (MRI) sequences is taken, next the cartilage surface is imaged, then mechanical indentation tests are completed at 2-7 different locations per sample, resulting in force/displacement data over time, and finally, the underlying bone surface is imaged. Imaging and mechanical testing are performed with a custom-built robotics-based testing device. Stereo reconstruction of the cartilage and subchondral bone surface is employed, which, together with the proposed constitutive model, led to specimen-specific finite element simulations of the mechanical indentation tests. The force-time response of 23 such indentation experiment simulations is optimized to estimate the mean material parameters and corresponding standard deviations. The model is capable of reproducing the deformation behavior of human articular cartilage in the physiological loading domain, as demonstrated by the good agreement between the experiment and numerical results (R(2)=0.95+/-0.03, mean+/-standard deviation of force-time response for 23 indentation tests). To address validation, a sevenfold cross-validation experiment is performed on the 21 experiments representing healthy cartilage. To quantify the predictive error, the mean of the absolute force differences and Pearson's correlation coefficient are both calculated. Deviations in the mean absolute difference, normalized by the peak force, range from 4% to 90%, with 40+/-25% (M+/-SD). The correlation coefficients across all predictions have a minimum of 0.939, and a maximum of 0.993 with 0.975+/-0.013 (M+/-SD), which demonstrates an excellent match of the decay characteristics. A novel feature of the proposed method is 3D sample-specific numerical tracking of the fiber fabric deformation under general loading. This feature is demonstrated by comparing the estimated fiber fabric deformation with recently published experimental data determined by diffusion tensor MRI. The proposed approach is efficient enough to enable large-scale 3D contact simulations of knee joint loading in simulations with accurate joint geometries.

Segmenting and tracking cell instances with cosine embeddings and recurrent hourglass networks.

Differently to semantic segmentation, instance segmentation assigns unique labels to each individual instance of the same object class. In this work, we propose a novel recurrent fully convolutional network architecture for tracking such instance segmentations over time, which is highly relevant, e.g., in biomedical applications involving cell growth and migration. Our network architecture incorporates convolutional gated recurrent units (ConvGRU) into a stacked hourglass network to utilize temporal information, e.g., from microscopy videos. Moreover, we train our network with a novel embedding loss based on cosine similarities, such that the network predicts unique embeddings for every instance throughout videos, even in the presence of dynamic structural changes due to mitosis of cells. To create the final tracked instance segmentations, the pixel-wise embeddings are clustered among subsequent video frames by using the mean shift algorithm. After showing the performance of the instance segmentation on a static in-house dataset of muscle fibers from H&E-stained microscopy images, we also evaluate our proposed recurrent stacked hourglass network regarding instance segmentation and tracking performance on six datasets from the ISBI celltracking challenge, where it delivers state-of-the-art results.

Integrating spatial configuration into heatmap regression based CNNs for landmark localization.

In many medical image analysis applications, only a limited amount of training data is available due to the costs of image acquisition and the large manual annotation effort required from experts. Training recent state-of-the-art machine learning methods like convolutional neural networks (CNNs) from small datasets is a challenging task. In this work on anatomical landmark localization, we propose a CNN architecture that learns to split the localization task into two simpler sub-problems, reducing the overall need for large training datasets. Our fully convolutional SpatialConfiguration-Net (SCN) learns this simplification due to multiplying the heatmap predictions of its two components and by training the network in an end-to-end manner. Thus, the SCN dedicates one component to locally accurate but ambiguous candidate predictions, while the other component improves robustness to ambiguities by incorporating the spatial configuration of landmarks. In our extensive experimental evaluation, we show that the proposed SCN outperforms related methods in terms of landmark localization error on a variety of size-limited 2D and 3D landmark localization datasets, i.e., hand radiographs, lateral cephalograms, hand MRIs, and spine CTs.

Integrated computer-aided forensic case analysis, presentation, and documentation based on multimodal 3D data.

Three-dimensional (3D) crime scene documentation using 3D scanners and medical imaging modalities like computed tomography (CT) and magnetic resonance imaging (MRI) are increasingly applied in forensic casework. Together with digital photography, these modalities enable comprehensive and non-invasive recording of forensically relevant information regarding injuries/pathologies inside the body and on its surface. Furthermore, it is possible to capture traces and items at crime scenes. Such digitally secured evidence has the potential to similarly increase case understanding by forensic experts and non-experts in court. Unlike photographs and 3D surface models, images from CT and MRI are not self-explanatory. Their interpretation and understanding requires radiological knowledge. Findings in tomography data must not only be revealed, but should also be jointly studied with all the 2D and 3D data available in order to clarify spatial interrelations and to optimally exploit the data at hand. This is technically challenging due to the heterogeneous data representations including volumetric data, polygonal 3D models, and images. This paper presents a novel computer-aided forensic toolbox providing tools to support the analysis, documentation, annotation, and illustration of forensic cases using heterogeneous digital data. Conjoint visualization of data from different modalities in their native form and efficient tools to visually extract and emphasize findings help experts to reveal unrecognized correlations and thereby enhance their case understanding. Moreover, the 3D case illustrations created for case analysis represent an efficient means to convey the insights gained from case analysis to forensic non-experts involved in court proceedings like jurists and laymen. The capability of the presented approach in the context of case analysis, its potential to speed up legal procedures and to ultimately enhance legal certainty is demonstrated by introducing a number of representative forensic cases.

An automatic model-based system for joint space measurements on hand radiographs: initial experience.

This ethics committee-approved pilot study was performed with informed consent. A Web-based service that was developed for automated measurement of joint space and automatic analysis of radiographs was tested prospectively. A total of 160 metacarpophalangeal joint spaces were measured in 20 patients (average age, 48 years; age range, 18-89 years; 16 women) suspected of having rheumatoid arthritis or osteoarthritis. The technical success rate was 93%. The smallest detectable difference in repeated automatic joint space width measurements varied from 0.08 to 0.31 mm, and the coefficient of variation was 2%-7%. Compared with the reference standard (interactive segmentation of the joint space widths) measurements, results were within a mean error of 0.19-0.40 mm. The proposed Web-based service enables reproducible joint space measurements to be obtained in metacarpophalangeal joints with moderate erosive and osteophytic disease.

Micro-colony array based high throughput platform for enzyme library screening.

Enzymes are becoming increasingly important tools for synthesizing and modifying fine and bulk chemicals. The availability of biocatalysts which fulfil the requirements of industrial processes is often limited. Recruiting suited enzymes from natural (e.g. metagenomes) and artificial (e.g. directed evolution) biodiversity is based on screening libraries of microbial clones expressing enzyme variants. However, exploring the complex diversity of such libraries needs efficient screening methods. Overcoming the "screening bottleneck" requires rapid high throughput technology allowing the analysis of a large diversity of different enzymes and applying different screening conditions. Facing these facts an efficient and cost effective method for high throughput screening of large enzyme libraries at the colony level was developed. Therefore, ordered high density micro-colony arrays were combined with optical sensor technology and automated image analysis. The system generally allows the simultaneous monitoring of enzyme activities reflected by up to 7000 micro-colonies spotted on a filter in the size of a micro-titer plate. A developed replica option also allows the analysis of clones under varying external conditions. The method was verified by a model screening using esterases and was proved to provide reliable enzyme activity measurements within single micro-colonies allowing the discrimination of activity differences in the range of 10-20%.

Brain-computer communication: motivation, aim, and impact of exploring a virtual apartment.

The step away from a synchronized or cue-based brain-computer interface (BCI) and from laboratory conditions towards real world applications is very important and crucial in BCI research. This work shows that ten naive subjects can be trained in a synchronous paradigm within three sessions to navigate freely through a virtual apartment, whereby at every junction the subjects could decide by their own, how they wanted to explore the virtual environment (VE). This virtual apartment was designed similar to a real world application, with a goal-oriented task, a high mental workload, and a variable decision period for the subject. All subjects were able to perform long and stable motor imagery over a minimum time of 2 s. Using only three electroencephalogram (EEG) channels to analyze these imaginations, we were able to convert them into navigation commands. Additionally, it could be demonstrated that motivation is a very crucial factor in BCI research; motivated subjects perform much better than unmotivated ones.

Model-based erosion spotting and visualization in rheumatoid arthritis.

RATIONALE AND OBJECTIVES: A method for the automatic detection and the visualization of erosions caused by rheumatoid arthritis is investigated. Erosion-enhanced viewing is a contribution to the computer-aided diagnosis of rheumatoid arthritis. It supports the clinician by providing the automatic marking of erosions and the visualization of any deviations from intact anatomy for a concise reviewing interface. MATERIALS AND METHODS: A generative appearance model is used to capture the variability of intact bone and erosions. The algorithm marks erosions on hand radiographs using this model, and visualizes these erosions with the help of the residual appearance error after fitting the model built from intact bone texture. The algorithm was evaluated on 17 hand radiographs. The standard of reference was an annotation of the erosions by a musculoskeletal radiologist. RESULTS: Detection results from the algorithm are reported for a set of 17 radiographs of moderately diseased hands. With a specificity of 84%, the detection of unequivocal erosions achieved a sensitivity of 85%. A receiver operating characteristic analysis yields an area under the curve of 0.92. The visualization provided a clear representation of the erosions as determined by two musculoskeletal radiologists. CONCLUSION: The automatic spotting of erosions provides promising results, and the visualization of the deviation from healthy anatomy aids clinicians in the evaluation of the erosions and in the reviewing of automatic detection results.

Algorithmic differentiation: application to variational problems in computer vision.

Many vision problems can be formulated as minimization of appropriate energy functionals. These energy functionals are usually minimized, based on the calculus of variations (Euler-Lagrange equation). Once the Euler-Lagrange equation has been determined, it needs to be discretized in order to implement it on a digital computer. This is not a trivial task and, is moreover, error-prone. In this paper, we propose a flexible alternative. We discretize the energy functional and, subsequently, apply the mathematical concept of algorithmic differentiation to directly derive algorithms that implement the energy functional's derivatives. This approach has several advantages: First, the computed derivatives are exact with respect to the implementation of the energy functional. Second, it is basically straightforward to compute second-order derivatives and, thus, the Hessian matrix of the energy functional. Third, algorithmic differentiation is a process which can be automated. We demonstrate this novel approach on three representative vision problems (namely, denoising, segmentation, and stereo) and show that state-of-the-art results are obtained with little effort.

Fast active appearance model search using canonical correlation analysis.

A fast AAM search algorithm based on canonical correlation analysis (CCA-AAM) is introduced. It efficiently models the dependency between texture residuals and model parameters during search. Experiments show that CCA-AAMs, while requiring similar implementation effort, consistently outperform standard search with regard to convergence speed by a factor of four.

Characterization of four-class motor imagery EEG data for the BCI-competition 2005.

To determine and compare the performance of different classifiers applied to four-class EEG data is the goal of this communication. The EEG data were recorded with 60 electrodes from five subjects performing four different motor-imagery tasks. The EEG signal was modeled by an adaptive autoregressive (AAR) process whose parameters were extracted by Kalman filtering. By these AAR parameters four classifiers were obtained, namely minimum distance analysis (MDA)--for single-channel analysis, and linear discriminant analysis (LDA), k-nearest-neighbor (kNN) classifiers as well as support vector machine (SVM) classifiers for multi-channel analysis. The performance of all four classifiers was quantified and evaluated by Cohen's kappa coefficient, an advantageous measure we introduced here to BCI research for the first time. The single-channel results gave rise to topographic maps that revealed the channels with the highest level of separability between classes for each subject. Our results of the multi-channel analysis indicate SVM as the most successful classifier, whereas kNN performed worst.

Robust active appearance models and their application to medical image analysis.

Active appearance models (AAMs) have been successfully used for a variety of segmentation tasks in medical image analysis. However, gross disturbances of objects can occur in routine clinical setting caused by pathological changes or medical interventions. This poses a problem for AAM-based segmentation, since the method is inherently not robust. In this paper, a novel robust AAM (RAAM) matching algorithm is presented. Compared to previous approaches, no assumptions are made regarding the kind of gray-value disturbance and/or the expected magnitude of residuals during matching. The method consists of two main stages. First, initial residuals are analyzed by means of a mean-shift-based mode detection step. Second, an objective function is utilized for the selection of a mode combination not representing the gross outliers. We demonstrate the robustness of the method in a variety of examples with different noise conditions. The RAAM performance is quantitatively demonstrated in two substantially different applications, diaphragm segmentation and rheumatoid arthritis assessment. In all cases, the robust method shows an excellent behavior, with the new method tolerating up to 50% object area covered by gross gray-level disturbances.

Bio-computational model of object-recognition: quantum Hebbian processing with neurally shaped Gabor wavelets.

Theoretical and simulational evidence, as well as experimental indications, are accumulating that quantum associative memory and imaging are possible. We compare these data with biological evidence, since we find them to a significant extent compatible. This paper presents a computationally implementable integrative model of appearance-based viewpoint-invariant recognition of objects. The neuro-quantum hybrid model incorporates neural processing up to V1 and quantum associative processing in V1, achieving together an object-recognition result in V2 and ITC. Results of our simulation of the central quantum-like parts of the bio-model, receiving neurally pre-processed inputs, are presented. This part contains our original simulated storage by multiple quantum interference of image-encoding Gabor wavelets done in a Hebbian way, especially using the Griniasty et al. pose-sequence learning rule.

Computer vision analysis of collagen fiber bundles in the adventitia of human blood vessels.

Numerical simulations, which are based on reliable biomechanical models of blood vessels, can help to get a better understanding of cardiovascular diseases such as atherosclerosis, and can be used to develop optimal medical treatment strategies.The adventitia is the outer most layer of blood vessels and its mechanical properties are essentially determined by the three-dimensional, structural arrangement of collagen fibre bundles embedded in the tissue. Global information such as the orientation statistics of the fibre bundles as well as detailed information as the crimp of the single fibres within the bundles is of particular interest in biomechanical modeling.In order to obtain a sufficiently large amount of data for biomechanical modeling, a fully automatic method for the structural analysis of the soft tissue is required. In this contribution we present methods based on computer vision to fulfill this task. We start by discussing proper tissue preparation and imaging techniques that have to be used to obtain data, which reliably represents the real three-dimensional tissue structure. The next step is concerned with algorithms that robustly segment the collagen fibre bundles and cope with various kinds of artifacts. Novel segmentation techniques for robust segmentation of individual fibril bundles and methods for estimation of their parameters, such as location, shape, mean fibril orientation, crimp of fibrils, etc, is discussed. The proposed algorithms are based on novel perceptual grouping methods operating on the extracted orientation data of fibrils.Finally, we demonstrate the results obtained by our fully automatic method on real data. In addition, for a more quantitative assessment, we introduce a generative structural model that enables the synthesis of three-dimensional fibre bundles with well-defined characteristics.