Transforms and Operators for Directional Bioimage Analysis: A Survey.

We give a methodology-oriented perspective on directional image analysis and rotation-invariant processing. We review the state of the art in the field and make connections with recent mathematical developments in functional analysis and wavelet theory. We unify our perspective within a common framework using operators. The intent is to provide image-processing methods that can be deployed in algorithms that analyze biomedical images with improved rotation invariance and high directional sensitivity. We start our survey with classical methods such as directional-gradient and the structure tensor. Then, we discuss how these methods can be improved with respect to robustness, invariance to geometric transformations (with a particular interest in scaling), and computation cost. To address robustness against noise, we move forward to higher degrees of directional selectivity and discuss Hessian-based detection schemes. To present multiscale approaches, we explain the differences between Fourier filters, directional wavelets, curvelets, and shearlets. To reduce the computational cost, we address the problem of matching directional patterns by proposing steerable filters, where one might perform arbitrary rotations and optimizations without discretizing the orientation. We define the property of steerability and give an introduction to the design of steerable filters. We cover the spectrum from simple steerable filters through pyramid schemes up to steerable wavelets. We also present illustrations on the design of steerable wavelets and their application to pattern recognition.

An Efficient Algorithm for the Piecewise Affine-Linear Mumford-Shah Model Based on a Taylor Jet Splitting.

We propose an algorithm to efficiently compute approximate solutions of the piecewise affine Mumford-Shah model. The algorithm is based on a novel reformulation of the underlying optimization problem in terms of Taylor jets. A splitting approach leads to linewise segmented jet estimation problems for which we propose an exact and efficient solver. The proposed method has the combined advantages of prior algorithms: it directly yields a partition, it does not need an initialization procedure, and it is highly parallelizable. The experiments show that the algorithm has lower computation times and that the solutions often have lower functional values than the state-of-the-art.

Bolus arrival time estimation in dynamic contrast-enhanced magnetic resonance imaging of small animals based on spline models.

Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is used to quantify perfusion and vascular permeability. In most cases a bolus arrival time (BAT) delay exists between the arterial input function (AIF) and the contrast agent arrival in the tissue of interest which needs to be estimated. Existing methods for BAT estimation are tailored to tissue concentration curves, which have a fast upslope to the peak as frequently observed in patient data. However, they may give poor results for curves that do not have this characteristic shape such as tissue concentration curves of small animals. In this paper, we propose a method for BAT estimation of signals that do not have a fast upslope to their peak. The model is based on splines which are able to adapt to a large variety of concentration curves. Furthermore, the method estimates BATs on a continuous time scale. All relevant model parameters are automatically determined by generalized cross validation. We use simulated concentration curves of small animal and patient settings to assess the accuracy and robustness of our approach. The proposed method outperforms a state-of-the-art method for small animal data and it gives competitive results for patient data. Finally, it is tested on in vivo acquired rat data where accuracy of BAT estimation was also improved upon the state-of-the-art method. The results indicate that the proposed method is suitable for accurate BAT estimation of DCE-MRI data, especially for small animals.

Total Variation Regularization of Pose Signals with an Application to 3D Freehand Ultrasound.

Three-dimensional freehand imaging techniques are gaining wider adoption due to their ?exibility and cost ef?ciency. Typical examples for such a combination of a tracking system with an imaging device are freehand SPECT or freehand 3D ultrasound. However, the quality of the resulting image data is heavily dependent on the skill of the human operator and on the level of noise of the tracking data. The latter aspect can introduce blur or strong artifacts, which can signi?cantly hamper the interpretation of image data. Unfortunately, the most commonly used tracking systems to date, i.e. optical and electromagnetic, present a trade-off between invading the surgeon's workspace (due to line-of-sight requirements) and higher levels of noise and sensitivity due to the interference of surrounding metallic objects. In this work, we propose a novel approach for total variation regularization of data from tracking systems (which we term pose signals) based on a variational formulation in the manifold of Euclidean transformations. The performance of the proposed approach was evaluated using synthetic data as well as real ultrasound sweeps executed on both a Lego phantom and human anatomy, showing signi?cant improvement in terms of tracking data quality and compounded ultrasound images. Source code can be found at https://github.com/IFL-CAMP/pose_regularization.

Fast Median Filtering for Phase or Orientation Data.

Median filtering is among the most utilized tools for smoothing real-valued data, as it is robust, edge-preserving, value-preserving, and yet can be computed efficiently. For data living on the unit circle, such as phase data or orientation data, a filter with similar properties is desirable. For these data, there is no unique means to define a median; so we discuss various possibilities. The arc distance median turns out to be the only variant which leads to robust, edge-preserving and value-preserving smoothing. However, there are no efficient algorithms for filtering based on the arc distance median. Here, we propose fast algorithms for filtering of signals and images with values on the unit circle based on the arc distance median. For non-quantized data, we develop an algorithm that scales linearly with the filter size. The runtime of our reference implementation is only moderately higher than the Matlab implementation of the classical median filter for real-valued data. For quantized data, we obtain an algorithm of constant complexity w.r.t. the filter size. We demonstrate the performance of our algorithms for real life data sets: phase images from interferometric synthetic aperture radar, planar flow fields from optical flow, and time series of wind directions.

Model-based learning of local image features for unsupervised texture segmentation.

Features that capture well the textural patterns of a certain class of images are crucial for the performance of texture segmentation methods. The manual selection of features or designing new ones can be a tedious task. Therefore, it is desirable to automatically adapt the features to a certain image or class of images. Typically, this requires a large set of training images with similar textures and ground truth segmentation. In this work, we propose a framework to learn features for texture segmentation when no such training data is available. The cost function for our learning process is constructed to match a commonly used segmentation model, the piecewise constant Mumford-Shah model. This means that the features are learned such that they provide an approximately piecewise constant feature image with a small jump set. Based on this idea, we develop a two-stage algorithm which first learns suitable convolutional features and then performs a segmentation. We note that the features can be learned from a small set of images, from a single image, or even from image patches. The proposed method achieves a competitive rank in the Prague texture segmentation benchmark, and it is effective for segmenting histological images.

Fast Piecewise-Affine Motion Estimation Without Segmentation.

Current algorithmic approaches for piecewise affine motion estimation are based on alternating motion segmentation and estimation. We propose a new method to estimate piecewise affine motion fields directly without intermediate segmentation. To this end, we reformulate the problem by imposing piecewise constancy of the parameter field, and derive a specific proximal splitting optimization scheme. A key component of our framework is an efficient one-dimensional piecewise-affine estimator for vector-valued signals. The first advantage of our approach over segmentation-based methods is its absence of initialization. The second advantage is its lower computational cost which is independent of the complexity of the motion field. In addition to these features, we demonstrate competitive accuracy with other piecewise-parametric methods on standard evaluation benchmarks. Our new regularization scheme also outperforms the more standard use of total variation and total generalized variation.

Fast Segmentation From Blurred Data in 3D Fluorescence Microscopy.

We develop a fast algorithm for segmenting 3D images from linear measurements based on the Potts model (or piecewise constant Mumford-Shah model). To that end, we first derive suitable space discretizations of the 3D Potts model, which are capable of dealing with 3D images defined on non-cubic grids. Our discretization allows us to utilize a specific splitting approach, which results in decoupled subproblems of moderate size. The crucial point in the 3D setup is that the number of independent subproblems is so large that we can reasonably exploit the parallel processing capabilities of the graphics processing units (GPUs). Our GPU implementation is up to 18 times faster than the sequential CPU version. This allows to process even large volumes in acceptable runtimes. As a further contribution, we extend the algorithm in order to deal with non-negativity constraints. We demonstrate the efficiency of our method for combined image deconvolution and segmentation on simulated data and on real 3D wide field fluorescence microscopy data.

Edge Preserving and Noise Reducing Reconstruction for Magnetic Particle Imaging.

Magnetic particle imaging (MPI) is an emerging medical imaging modality which is based on the non-linear response of magnetic nanoparticles to an applied magnetic field. It is an important feature of MPI that even fast dynamic processes can be captured for 3D volumes. The high temporal resolution in turn leads to large amounts of data which have to be handled efficiently. But as the system matrix of MPI is non-sparse, the image reconstruction gets computationally demanding. Therefore, currently only basic image reconstruction methods such as Tikhonov regularization are used. However, Tikhonov regularization is known to oversmooth edges in the reconstructed image and to have only a limited noise reducing effect. In this work, we develop an efficient edge preserving and noise reducing reconstruction method for MPI. As regularization model, we propose to use the nonnegative fused lasso model, and we devise a discretization that is adapted to the acquisition geometry of the preclinical MPI scanner considered in this work. We develop a customized solver based on a generalized forward-backward scheme which is particularly suitable for the dense and not well-structured system matrices in MPI. Already a non-optimized prototype implementation processes a 3D volume within a few seconds so that processing several frames per second seems amenable. We demonstrate the improvement in reconstruction quality over the state-of-the-art method in an experimental medical setup for an in-vitro angioplasty of a stenosis.

Joint Segmentation and Shape Regularization With a Generalized Forward-Backward Algorithm.

This paper presents a method for the simultaneous segmentation and regularization of a series of shapes from a corresponding sequence of images. Such series arise as time series of 2D images when considering video data, or as stacks of 2D images obtained by slicewise tomographic reconstruction. We first derive a model where the regularization of the shape signal is achieved by a total variation prior on the shape manifold. The method employs a modified Kendall shape space to facilitate explicit computations together with the concept of Sobolev gradients. For the proposed model, we derive an efficient and computationally accessible splitting scheme. Using a generalized forward-backward approach, our algorithm treats the total variation atoms of the splitting via proximal mappings, whereas the data terms are dealt with by gradient descent. The potential of the proposed method is demonstrated on various application examples dealing with 3D data. We explain how to extend the proposed combined approach to shape fields which, for instance, arise in the context of 3D+t imaging modalities, and show an application in this setup as well.

Combined Tensor Fitting and TV Regularization in Diffusion Tensor Imaging Based on a Riemannian Manifold Approach.

In this paper, we consider combined TV denoising and diffusion tensor fitting in DTI using the affine-invariant Riemannian metric on the space of diffusion tensors. Instead of first fitting the diffusion tensors, and then denoising them, we define a suitable TV type energy functional which incorporates the measured DWIs (using an inverse problem setup) and which measures the nearness of neighboring tensors in the manifold. To approach this functional, we propose generalized forward- backward splitting algorithms which combine an explicit and several implicit steps performed on a decomposition of the functional. We validate the performance of the derived algorithms on synthetic and real DTI data. In particular, we work on real 3D data. To our knowledge, the present paper describes the first approach to TV regularization in a combined manifold and inverse problem setup.

Iterative Potts and Blake-Zisserman minimization for the recovery of functions with discontinuities from indirect measurements.

Signals with discontinuities appear in many problems in the applied sciences ranging from mechanics, electrical engineering to biology and medicine. The concrete data acquired are typically discrete, indirect and noisy measurements of some quantities describing the signal under consideration. The task is to restore the signal and, in particular, the discontinuities. In this respect, classical methods perform rather poor, whereas non-convex non-smooth variational methods seem to be the correct choice. Examples are methods based on Mumford-Shah and piecewise constant Mumford-Shah functionals and discretized versions which are known as Blake-Zisserman and Potts functionals. Owing to their non-convexity, minimization of such functionals is challenging. In this paper, we propose a new iterative minimization strategy for Blake-Zisserman as well as Potts functionals and a related jump-sparsity problem dealing with indirect, noisy measurements. We provide a convergence analysis and underpin our findings with numerical experiments.

Steerable wavelet frames based on the Riesz transform.

We consider an extension of the 1-D concept of analytical wavelet to n-D which is by construction compatible with rotations. This extension, called a monogenic wavelet, yields a decomposition of the wavelet coefficients into amplitude, phase, and phase direction. The monogenic wavelet is based on the hypercomplex monogenic signal which is defined using Riesz transforms and perfectly isotropic wavelets frames. Employing the new concept of Clifford frames, we can show that the monogenic wavelet generates a wavelet frame. Furthermore, this approach yields wavelet frames that are steerable with respect to direction. Applications to descreening and contrast enhancement illustrate the versatility of this approach to image analysis and reconstruction.