ilastik: interactive machine learning for (bio)image analysis.

We present ilastik, an easy-to-use interactive tool that brings machine-learning-based (bio)image analysis to end users without substantial computational expertise. It contains pre-defined workflows for image segmentation, object classification, counting and tracking. Users adapt the workflows to the problem at hand by interactively providing sparse training annotations for a nonlinear classifier. ilastik can process data in up to five dimensions (3D, time and number of channels). Its computational back end runs operations on-demand wherever possible, allowing for interactive prediction on data larger than RAM. Once the classifiers are trained, ilastik workflows can be applied to new data from the command line without further user interaction. We describe all ilastik workflows in detail, including three case studies and a discussion on the expected performance.

Graphical model for joint segmentation and tracking of multiple dividing cells.

MOTIVATION: To gain fundamental insight into the development of embryos, biologists seek to understand the fate of each and every embryonic cell. For the generation of cell tracks in embryogenesis, so-called tracking-by-assignment methods are flexible approaches. However, as every two-stage approach, they suffer from irrevocable errors propagated from the first stage to the second stage, here from segmentation to tracking. It is therefore desirable to model segmentation and tracking in a joint holistic assignment framework allowing the two stages to maximally benefit from each other. RESULTS: We propose a probabilistic graphical model, which both automatically selects the best segments from a time series of oversegmented images/volumes and links them across time. This is realized by introducing intra-frame and inter-frame constraints between conflicting segmentation and tracking hypotheses while at the same time allowing for cell division. We show the efficiency of our algorithm on a challenging 3D+t cell tracking dataset from Drosophila embryogenesis and on a 2D+t dataset of proliferating cells in a dense population with frequent overlaps. On the latter, we achieve results significantly better than state-of-the-art tracking methods. AVAILABILITY AND IMPLEMENTATION: Source code and the 3D+t Drosophila dataset along with our manual annotations will be freely available on http://hci.iwr.uni-heidelberg.de/MIP/Research/tracking/

Segmenting and Tracking Multiple Dividing Targets Using ilastik.

Tracking crowded cells or other targets in biology is often a challenging task due to poor signal-to-noise ratio, mutual occlusion, large displacements, little discernibility, and the ability of cells to divide. We here present an open source implementation of conservation tracking (Schiegg et al., IEEE international conference on computer vision (ICCV). IEEE, New York, pp 2928-2935, 2013) in the ilastik software framework. This robust tracking-by-assignment algorithm explicitly makes allowance for false positive detections, undersegmentation, and cell division. We give an overview over the underlying algorithm and parameters, and explain the use for a light sheet microscopy sequence of a Drosophila embryo. Equipped with this knowledge, users will be able to track targets of interest in their own data.

DALSA: Domain Adaptation for Supervised Learning From Sparsely Annotated MR Images.

We propose a new method that employs transfer learning techniques to effectively correct sampling selection errors introduced by sparse annotations during supervised learning for automated tumor segmentation. The practicality of current learning-based automated tissue classification approaches is severely impeded by their dependency on manually segmented training databases that need to be recreated for each scenario of application, site, or acquisition setup. The comprehensive annotation of reference datasets can be highly labor-intensive, complex, and error-prone. The proposed method derives high-quality classifiers for the different tissue classes from sparse and unambiguous annotations and employs domain adaptation techniques for effectively correcting sampling selection errors introduced by the sparse sampling. The new approach is validated on labeled, multi-modal MR images of 19 patients with malignant gliomas and by comparative analysis on the BraTS 2013 challenge data sets. Compared to training on fully labeled data, we reduced the time for labeling and training by a factor greater than 70 and 180 respectively without sacrificing accuracy. This dramatically eases the establishment and constant extension of large annotated databases in various scenarios and imaging setups and thus represents an important step towards practical applicability of learning-based approaches in tissue classification.

Automated detection of synapses in serial section transmission electron microscopy image stacks.

We describe a method for fully automated detection of chemical synapses in serial electron microscopy images with highly anisotropic axial and lateral resolution, such as images taken on transmission electron microscopes. Our pipeline starts from classification of the pixels based on 3D pixel features, which is followed by segmentation with an Ising model MRF and another classification step, based on object-level features. Classifiers are learned on sparse user labels; a fully annotated data subvolume is not required for training. The algorithm was validated on a set of 238 synapses in 20 serial 7197×7351 pixel images (4.5×4.5×45 nm resolution) of mouse visual cortex, manually labeled by three independent human annotators and additionally re-verified by an expert neuroscientist. The error rate of the algorithm (12% false negative, 7% false positive detections) is better than state-of-the-art, even though, unlike the state-of-the-art method, our algorithm does not require a prior segmentation of the image volume into cells. The software is based on the ilastik learning and segmentation toolkit and the vigra image processing library and is freely available on our website, along with the test data and gold standard annotations (http://www.ilastik.org/synapse-detection/sstem).

Learning to segment neurons with non-local quality measures.

Segmentation schemes such as hierarchical region merging or correllation clustering rely on edge weights between adjacent (super-)voxels. The quality of these edge weights directly affects the quality of the resulting segmentations. Unstructured learning methods seek to minimize the classification error on individual edges. This ignores that a few local mistakes (tiny boundary gaps) can cause catastrophic global segmentation errors. Boundary evidence learning should therefore optimize structured quality criteria such as Rand Error or Variation of Information. We present the first structured learning scheme using a structured loss function; and we introduce a new hierarchical scheme that allows to approximately solve the NP hard prediction problem even for huge volume images. The value of these contributions is demonstrated on two challenging neural circuit reconstruction problems in serial sectioning electron microscopic images with billions of voxels. Our contributions lead to a partitioning quality that improves over the current state of the art.

3D segmentation of SBFSEM images of neuropil by a graphical model over supervoxel boundaries.

The segmentation of large volume images of neuropil acquired by serial sectioning electron microscopy is an important step toward the 3D reconstruction of neural circuits. The only cue provided by the data at hand is boundaries between otherwise indistinguishable objects. This indistinguishability, combined with the boundaries becoming very thin or faint in places, makes the large body of work on region-based segmentation methods inapplicable. On the other hand, boundary-based methods that exploit purely local evidence do not reach the extremely high accuracy required by the application domain that cannot tolerate the global topological errors arising from false local decisions. As a consequence, we propose a supervoxel merging method that arrives at its decisions in a non-local fashion, by posing and approximately solving a joint combinatorial optimization problem over all faces between supervoxels. The use of supervoxels allows the extraction of expressive geometric features. These are used by the higher-order potentials in a graphical model that assimilate knowledge about the geometry of neural surfaces by automated training on a gold standard. The scope of this improvement is demonstrated on the benchmark dataset E1088 (Helmstaedter et al., 2011) of 7.5billionvoxels from the inner plexiform layer of rabbit retina. We provide C++ source code for annotation, geometry extraction, training and inference.

Automated detection and segmentation of synaptic contacts in nearly isotropic serial electron microscopy images.

We describe a protocol for fully automated detection and segmentation of asymmetric, presumed excitatory, synapses in serial electron microscopy images of the adult mammalian cerebral cortex, taken with the focused ion beam, scanning electron microscope (FIB/SEM). The procedure is based on interactive machine learning and only requires a few labeled synapses for training. The statistical learning is performed on geometrical features of 3D neighborhoods of each voxel and can fully exploit the high z-resolution of the data. On a quantitative validation dataset of 111 synapses in 409 images of 1948×1342 pixels with manual annotations by three independent experts the error rate of the algorithm was found to be comparable to that of the experts (0.92 recall at 0.89 precision). Our software offers a convenient interface for labeling the training data and the possibility to visualize and proofread the results in 3D. The source code, the test dataset and the ground truth annotation are freely available on the website http://www.ilastik.org/synapse-detection.