De novo prediction of RNA 3D structures with deep generative models.

We present a Deep Learning approach to predict 3D folding structures of RNAs from their nucleic acid sequence. Our approach combines an autoregressive Deep Generative Model, Monte Carlo Tree Search, and a score model to find and rank the most likely folding structures for a given RNA sequence. We show that RNA de novo structure prediction by deep learning is possible at atom resolution, despite the low number of experimentally measured structures that can be used for training. We confirm the predictive power of our approach by achieving competitive results in a retrospective evaluation of the RNA-Puzzles prediction challenges, without using structural contact information from multiple sequence alignments or additional data from chemical probing experiments. Blind predictions for recent RNA-Puzzle challenges under the name "Dfold" further support the competitive performance of our approach.

Comparison of cardiac volumetry using real-time MRI during free-breathing with standard cine MRI during breath-hold in children.

BACKGROUND: Cardiac real-time magnetic resonance imaging (RT-MRI) provides high-quality images even during free-breathing. Difficulties in post-processing impede its use in clinical routine. OBJECTIVE: To demonstrate the feasibility of quantitative analysis of cardiac free-breathing RT-MRI and to compare image quality and volumetry during free-breathing RT-MRI in pediatric patients to standard breath-hold cine MRI. MATERIALS AND METHODS: Pediatric patients (n = 22) received cardiac RT-MRI volumetry during free breathing (1.5 T; short axis; 30 frames per s) in addition to standard breath-hold cine imaging in end-expiration. Real-time images were binned retrospectively based on electrocardiography and respiratory bellows. Image quality and volumetry were compared using the European Cardiovascular Magnetic Resonance registry score, structure visibility rating, linear regression and Bland-Altman analyses. RESULTS: Additional time for binning of real-time images was 2 min. For both techniques, image quality was rated good to excellent. RT-MRI was significantly more robust against artifacts (P < 0.01). Linear regression revealed good correlations for the ventricular volumes. Bland-Altman plots showed a good limit of agreement (LoA) for end-diastolic volume (left ventricle [LV]: LoA -0.1 ± 2.7 ml/m2, right ventricle [RV]: LoA -1.9 ± 3.4 ml/m2), end-systolic volume (LV: LoA 0.4 ± 1.9 ml/m2, RV: LoA 0.6 ± 2.0 ml/m2), stroke volume (LV: LoA -0.5 ± 2.3 ml/m2, RV: LoA -2.6 ± 3.3 ml/m2) and ejection fraction (LV: LoA -0.5 ± 1.6%, RV: LoA -2.1 ± 2.8%). CONCLUSION: Compared to standard cine MRI with breath hold, RT-MRI during free breathing with retrospective respiratory binning offers good image quality, reduced image artifacts enabling fast quantitative evaluations of ventricular volumes in clinical practice under physiological conditions.

Contour proposal networks for biomedical instance segmentation.

We present a conceptually simple framework for object instance segmentation, called Contour Proposal Network (CPN), which detects possibly overlapping objects in an image while simultaneously fitting closed object contours using a fixed-size representation based on Fourier Descriptors. The CPN can incorporate state-of-the-art object detection architectures as backbone networks into a single-stage instance segmentation model that can be trained end-to-end. We construct CPN models with different backbone networks and apply them to instance segmentation of cells in datasets from different modalities. In our experiments, CPNs outperform U-Net, Mask R-CNN and StarDist in instance segmentation accuracy. We present variants with execution times suitable for real-time applications. The trained models generalize well across different domains of cell types. Since the main assumption of the framework is closed object contours, it is applicable to a wide range of detection problems also beyond the biomedical domain. An implementation of the model architecture in PyTorch is freely available.

Convolutional neural networks for cytoarchitectonic brain mapping at large scale.

Human brain atlases provide spatial reference systems for data characterizing brain organization at different levels, coming from different brains. Cytoarchitecture is a basic principle of the microstructural organization of the brain, as regional differences in the arrangement and composition of neuronal cells are indicators of changes in connectivity and function. Automated scanning procedures and observer-independent methods are prerequisites to reliably identify cytoarchitectonic areas, and to achieve reproducible models of brain segregation. Time becomes a key factor when moving from the analysis of single regions of interest towards high-throughput scanning of large series of whole-brain sections. Here we present a new workflow for mapping cytoarchitectonic areas in large series of cell-body stained histological sections of human postmortem brains. It is based on a Deep Convolutional Neural Network (CNN), which is trained on a pair of section images with annotations, with a large number of un-annotated sections in between. The model learns to create all missing annotations in between with high accuracy, and faster than our previous workflow based on observer-independent mapping. The new workflow does not require preceding 3D-reconstruction of sections, and is robust against histological artefacts. It processes large data sets with sizes in the order of multiple Terabytes efficiently. The workflow was integrated into a web interface, to allow access without expertise in deep learning and batch computing. Applying deep neural networks for cytoarchitectonic mapping opens new perspectives to enable high-resolution models of brain areas, introducing CNNs to identify borders of brain areas.

Spirometry-based reconstruction of real-time cardiac MRI: Motion control and quantification of heart-lung interactions.

PURPOSE: To test the feasibility of cardiac real-time MRI in combination with retrospective gating by MR-compatible spirometry, to improve motion control, and to allow quantification of respiratory-induced changes during free-breathing. METHODS: Cross-sectional real-time MRI (1.5T; 30 frames/s) using steady-state free precession contrast during free-breathing was combined with MR-compatible spirometry in healthy adult volunteers (n = 4). Retrospective binning assigned images to classes that were defined by electrocardiogram and spirometry. Left ventricular eccentricity index as an indicator of septal position and ventricular volumes in different respiratory phases were calculated to assess heart-lung interactions. RESULTS: Real-time MRI with MR-compatible spirometry is feasible and well tolerated. Spirometry-based binning improved motion control significantly. The end-diastolic epicardial eccentricity index increased significantly during inspiration (1.04 ± 0.04 to 1.19 ± 0.05; P < .05). During inspiration, right ventricular end-diastolic volume (79 ± 17 mL/m2 to 98 ± 18 mL/m2 ), stroke volume (41 ± 8 mL/m2 to 59 ± 11 mL/m2 ) and ejection fraction (53 ± 3% to 60 ± 1%) increased significantly, whereas the end-systolic volume remained almost unchanged. Left ventricular end-diastolic volume, left ventricular stroke volume, and left ventricular ejection fraction decreased during inspiration, whereas the left ventricular end-systolic volume increased. The relationship between stroke volume and end-diastolic volume (Frank-Starling relationship) based on changes induced by respiration allowed for a slope estimate of the Frank-Starling curve to be 0.9 to 1.1. CONCLUSION: Real-time MRI during free-breathing combined with MR-compatible spirometry and retrospective binning improves image stabilization, allows quantitative image analysis, and importantly, offers unique opportunities to judge heart-lung interactions.

Painfree and accurate Bayesian estimation of psychometric functions for (potentially) overdispersed data.

The psychometric function describes how an experimental variable, such as stimulus strength, influences the behaviour of an observer. Estimation of psychometric functions from experimental data plays a central role in fields such as psychophysics, experimental psychology and in the behavioural neurosciences. Experimental data may exhibit substantial overdispersion, which may result from non-stationarity in the behaviour of observers. Here we extend the standard binomial model which is typically used for psychometric function estimation to a beta-binomial model. We show that the use of the beta-binomial model makes it possible to determine accurate credible intervals even in data which exhibit substantial overdispersion. This goes beyond classical measures for overdispersion-goodness-of-fit-which can detect overdispersion but provide no method to do correct inference for overdispersed data. We use Bayesian inference methods for estimating the posterior distribution of the parameters of the psychometric function. Unlike previous Bayesian psychometric inference methods our software implementation-psignifit 4-performs numerical integration of the posterior within automatically determined bounds. This avoids the use of Markov chain Monte Carlo (MCMC) methods typically requiring expert knowledge. Extensive numerical tests show the validity of the approach and we discuss implications of overdispersion for experimental design. A comprehensive MATLAB toolbox implementing the method is freely available; a python implementation providing the basic capabilities is also available.

Learning to Deblur.

We describe a learning-based approach to blind image deconvolution. It uses a deep layered architecture, parts of which are borrowed from recent work on neural network learning, and parts of which incorporate computations that are specific to image deconvolution. The system is trained end-to-end on a set of artificially generated training examples, enabling competitive performance in blind deconvolution, both with respect to quality and runtime.

Attribute-based classification for zero-shot visual object categorization.

We study the problem of object recognition for categories for which we have no training examples, a task also called zero--data or zero-shot learning. This situation has hardly been studied in computer vision research, even though it occurs frequently; the world contains tens of thousands of different object classes, and image collections have been formed and suitably annotated for only a few of them. To tackle the problem, we introduce attribute-based classification: Objects are identified based on a high-level description that is phrased in terms of semantic attributes, such as the object's color or shape. Because the identification of each such property transcends the specific learning task at hand, the attribute classifiers can be prelearned independently, for example, from existing image data sets unrelated to the current task. Afterward, new classes can be detected based on their attribute representation, without the need for a new training phase. In this paper, we also introduce a new data set, Animals with Attributes, of over 30,000 images of 50 animal classes, annotated with 85 semantic attributes. Extensive experiments on this and two more data sets show that attribute-based classification indeed is able to categorize images without access to any training images of the target classes.

Greedy learning of binary latent trees.

Inferring latent structures from observations helps to model and possibly also understand underlying data generating processes. A rich class of latent structures is the latent trees, i.e., tree-structured distributions involving latent variables where the visible variables are leaves. These are also called hierarchical latent class (HLC) models. Zhang and Kocka proposed a search algorithm for learning such models in the spirit of Bayesian network structure learning. While such an approach can find good solutions, it can be computationally expensive. As an alternative, we investigate two greedy procedures: the BIN-G algorithm determines both the structure of the tree and the cardinality of the latent variables in a bottom-up fashion. The BIN-A algorithm first determines the tree structure using agglomerative hierarchical clustering, and then determines the cardinality of the latent variables as for BIN-G. We show that even with restricting ourselves to binary trees, we obtain HLC models of comparable quality to Zhang's solutions (in terms of cross-validated log-likelihood), while being generally faster to compute. This claim is validated by a comprehensive comparison on several data sets. Furthermore, we demonstrate that our methods are able to estimate interpretable latent structures on real-world data with a large number of variables. By applying our method to a restricted version of the 20 newsgroups data, these models turn out to be related to topic models, and on data from the PASCAL Visual Object Classes (VOC) 2007 challenge, we show how such treestructured models help us understand how objects co-occur in images. For reproducibility of all experiments in this paper, all code and data sets (or links to data) are available at http://people.kyb.tuebingen.mpg.de/harmeling/code/ltt-1.4.tar.