DD-Net: spectral imaging from a monochromatic dispersed and diffused snapshot.

We propose a snapshot spectral imaging method for the visible spectral range using a single monochromatic camera equipped with a two-dimensional (2D) binary-encoded phase diffuser placed at the pupil of the imaging lens and by resorting to deep learning (DL) algorithms for signal reconstruction. While spectral imaging was shown to be feasible using two cameras equipped with a single, one-dimensional (1D) binary diffuser and compressed sensing (CS) algorithms [Appl. Opt.59, 7853 (2020).APOPAI0003-693510.1364/AO.395541], the suggested diffuser design expands the optical response and creates optical spatial and spectral encoding along both dimensions of the image sensor. To recover the spatial and spectral information from the dispersed and diffused (DD) monochromatic snapshot, we developed novel DL algorithms, dubbed DD-Nets, which are tailored to the unique response of the optical system, which includes either a 1D or a 2D diffuser. High-quality reconstructions of the spectral cube in simulation and lab experiments are presented for system configurations consisting of a single monochromatic camera with either a 1D or a 2D diffuser. We demonstrate that the suggested system configuration with the 2D diffuser outperforms system configurations with a 1D diffuser that utilize either DL-based or CS-based algorithms for the reconstruction of the spectral cube.

Dual-camera snapshot spectral imaging with a pupil-domain optical diffuser and compressed sensing algorithms.

We propose a snapshot spectral imaging method for the visible spectral range using two digital cameras placed side-by-side: a regular red-green-blue (RGB) camera and a monochromatic camera equipped with a dispersive diffractive diffuser placed at the pupil of the imaging lens. While spectral imaging was shown to be feasible using a single monochromatic camera with a pupil diffuser [Appl. Opt.55, 432 (2016)APOPAI0003-693510.1364/AO.55.000432], adding an RGB camera provides more spatial and spectral information for stable reconstruction of the spectral cube of a scene. Results of optical experiments confirm that the combined data from the two cameras relax the complexity of the underdetermined reconstruction problem and improve the reconstructed image quality obtained using compressed sensing-based algorithms.

Design of binary-phase diffusers for a compressed sensing snapshot spectral imaging system with two cameras.

We propose designs of pupil-domain optical diffusers for a snapshot spectral imaging system using binary-phase encoding. The suggested designs enable the creation of point-spread functions with defined optical response, having profiles that are dependent on incident wavefront wavelength. This efficient combination of dispersive and diffusive optical responses enables us to perform snapshot spectral imaging using compressed sensing algorithms while keeping a high optical throughput alongside a simple fabrication process. Experimental results are reported.

Application of MR morphologic, diffusion tensor, and perfusion imaging in the classification of brain tumors using machine learning scheme.

PURPOSE: While MRI is the modality of choice for the assessment of patients with brain tumors, differentiation between various tumors based on their imaging characteristics might be challenging due to overlapping imaging features. The purpose of this study was to apply a machine learning scheme using basic and advanced MR sequences for distinguishing different types of brain tumors. METHODS: The study cohort included 141 patients (41 glioblastoma, 38 metastasis, 50 meningioma, and 12 primary central nervous system lymphoma). A computer-assisted classification scheme, combining morphologic MRI, perfusion MRI, and DTI metrics, was developed and used for tumor classification. The proposed multistep scheme consists of pre-processing, ROI definition, features extraction, feature selection, and classification. Feature subset selection was performed using support vector machines (SVMs). Classification performance was assessed by leave-one-out cross-validation. Given an ROI, the entire classification process was done automatically via computer and without any human intervention. RESULTS: A binary hierarchical classification tree was chosen. In the first step, selected features were chosen for distinguishing glioblastoma from the remaining three classes, followed by separation of meningioma from metastasis and PCNSL, and then to discriminate PCNSL from metastasis. The binary SVM classification accuracy, sensitivity and specificity for glioblastoma, metastasis, meningiomas, and primary central nervous system lymphoma were 95.7, 81.6, and 91.2%; 92.7, 95.1, and 93.6%; 97, 90.8, and 58.3%; and 91.5, 90, and 96.9%, respectively. CONCLUSION: A machine learning scheme using data from anatomical and advanced MRI sequences resulted in high-performance automatic tumor classification algorithm. Such a scheme can be integrated into clinical decision support systems to optimize tumor classification.

Compressed sensing snapshot spectral imaging by a regular digital camera with an added optical diffuser.

We propose a spectral imaging method that allows a regular digital camera to be converted into a snapshot spectral imager by equipping the camera with a dispersive diffuser and with a compressed sensing-based algorithm for digital processing. Results of optical experiments are reported.

Gaussian bandwidth selection for manifold learning and classification.

Kernel methods play a critical role in many machine learning algorithms. They are useful in manifold learning, classification, clustering and other data analysis tasks. Setting the kernel's scale parameter, also referred to as the kernel's bandwidth, highly affects the performance of the task in hand. We propose to set a scale parameter that is tailored to one of two types of tasks: classification and manifold learning. For manifold learning, we seek a scale which is best at capturing the manifold's intrinsic dimension. For classification, we propose three methods for estimating the scale, which optimize the classification results in different senses. The proposed frameworks are simulated on artificial and on real datasets. The results show a high correlation between optimal classification rates and the estimated scales. Finally, we demonstrate the approach on a seismic event classification task.

Spectral multiplexing method for digital snapshot spectral imaging.

We propose a spectral imaging method for piecewise "macropixel" objects, which allows a regular digital camera to be converted into a digital snapshot spectral imager by equipping the camera with only a disperser and a demultiplexing algorithm. The method exploits a "multiplexed spectrum" intensity pattern, i.e., the superposition of spectra from adjacent different image points, formed on the image sensor of the digital camera. The spatial image resolution is restricted to a macropixel level in order to acquire both spectral and spatial data (i.e., an entire spectral cube) in a single snapshot. Results of laboratory experiments with a special macropixel object image, composed of small, spatially uniform squares, provide to our knowledge a first verification of the proposed spectral imaging method.

Unsupervised tumor detection in Dynamic PET/CT imaging of the prostate.

Early detection and localization of prostate tumors pose a challenge to the medical community. Several imaging techniques, including PET, have shown some success. But no robust and accurate solution has yet been reached. This work aims to detect prostate cancer foci in Dynamic PET images using an unsupervised learning approach. The proposed method extracts three feature classes from 4D imaging data that include statistical, kinetic biological and deep features that are learned by a deep stacked convolutional autoencoder. Anomalies, which are classified as tumors, are detected in feature space using density estimation. The proposed algorithm generates promising results for sufficiently large cancer foci in real PET scans imaging where the foci is not viewed by the tomographic devices used for detection.

Analysis of Human Brain Structure Reveals that the Brain "Types" Typical of Males Are Also Typical of Females, and Vice Versa.

Findings of average differences between females and males in the structure of specific brain regions are often interpreted as indicating that the typical male brain is different from the typical female brain. An alternative interpretation is that the brain types typical of females are also typical of males, and sex differences exist only in the frequency of rare brain types. Here we contrasted the two hypotheses by analyzing the structure of 2176 human brains using three analytical approaches. An anomaly detection analysis showed that brains from females are almost as likely to be classified as "normal male brains," as brains from males are, and vice versa. Unsupervised clustering algorithms revealed that common brain "types" are similarly common in females and in males and that a male and a female are almost as likely to have the same brain "type" as two females or two males are. Large sex differences were found only in the frequency of some rare brain "types." Last, supervised clustering algorithms revealed that the brain "type(s)" typical of one sex category in one sample could be typical of the other sex category in another sample. The present findings demonstrate that even when similarity and difference are defined mathematically, ignoring biological or functional relevance, sex category (i.e., whether one is female or male), is not a major predictor of the variability of human brain structure. Rather, the brain types typical of females are also typical of males, and vice versa, and large sex differences are found only in the prevalence of some rare brain types. We discuss the implications of these findings to studies of the structure and function of the human brain.

Image coding with geometric wavelets.

This paper describes a new and efficient method for low bit-rate image coding which is based on recent development in the theory of multivariate nonlinear piecewise polynomial approximation. It combines a binary space partition scheme with geometric wavelet (GW) tree approximation so as to efficiently capture curve singularities and provide a sparse representation of the image. The GW method successfully competes with state-of-the-art wavelet methods such as the EZW, SPIHT, and EBCOT algorithms. We report a gain of about 0.4 dB over the SPIHT and EBCOT algorithms at the bit-rate 0.0625 bits-per-pixels (bpp). It also outperforms other recent methods that are based on "sparse geometric representation." For example, we report a gain of 0.27 dB over the Bandelets algorithm at 0.1 bpp. Although the algorithm is computationally intensive, its time complexity can be significantely reduced by collecting a "global" GW n-term approximation to the image from a collection of GW trees, each constructed separately over tiles of the image.

Multisensor image registration via implicit similarity.

This paper presents an approach to the registration of significantly dissimilar images, acquired by sensors of different modalities. A robust matching criterion is derived by aligning the locations of gradient maxima. The alignment is achieved by iteratively maximizing the magnitudes of the intensity gradients of a set of pixels in one of the images, where the set is initialized by the gradient maxima locations of the second image. No explicit similarity measure that uses the intensities of both images is used. The computation utilizes the full spatial information of the first image and the accuracy and robustness of the registration depend only on it. False matchings are detected and adaptively weighted using a directional similarity measure. By embedding the scheme in a "coarse to fine" formulation, we were able to estimate affine and projective global motions, even when the images were characterized by complex space varying intensity transformations. The scheme is especially suitable when one of the images is of considerably better quality than the other (noise, blur, etc.). We demonstrate these properties via experiments on real multisensor image sets.

Edge-enhancement postprocessing using artificial dissipation.

We present a method to enhance, by postprocessing, the performance of gradient-based edge detectors. It improves the performance of the edge detector by adding terms which are similar to the artificial dissipation that appear in the numerical solution of hyperbolic PDEs. This term is added to the output of the edge detector. The edges that are missed or blurred by the edge detector are reconstructed through the addition of the artificial dissipation terms. Edges that are detected correctly by the edge detector are preserved. We present the theory of the artificial dissipation and its improvement of the quality of the detected edges. We demonstrate the performance of the algorithm on diverse images.

The angular difference function and its application to image registration.

The estimation of large motions without prior knowledge is an important problem in image registration. In this paper, we present the angular difference function (ADF) and demonstrate its applicability to rotation estimation. The ADF of two functions is defined as the integral of their spectral difference along the radial direction. It is efficiently computed using the pseudopolar Fourier transform, which computes the discrete Fourier transform of an image on a near spherical grid. Unlike other Fourier-based registration schemes, the suggested approach does not require any interpolation. Thus, it is more accurate and significantly faster.

Pseudopolar-based estimation of large translations, rotations, and scalings in images.

One of the major challenges related to image registration is the estimation of large motions without prior knowledge. This paper presents a Fourier-based approach that estimates large translations, scalings, and rotations. The algorithm uses the pseudopolar (PP) Fourier transform to achieve substantial improved approximations of the polar and log-polar Fourier transforms of an image. Thus, rotations and scalings are reduced to translations which are estimated using phase correlation. By utilizing the PP grid, we increase the performance (accuracy, speed, and robustness) of the registration algorithms. Scales up to 4 and arbitrary rotation angles can be robustly recovered, compared to a maximum scaling of 2 recovered by state-of-the-art algorithms. The algorithm only utilizes one-dimensional fast Fourier transform computations whose overall complexity is significantly lower than prior works. Experimental results demonstrate the applicability of the proposed algorithms.

Deblocking of block-transform compressed images using weighted sums of symmetrically aligned pixels.

A new class of related algorithms for deblocking block-transform compressed images and video sequences is proposed in this paper. The algorithms apply weighted sums on pixel quartets, which are symmetrically aligned with respect to block boundaries. The basic weights, which are aimed at very low bit-rate images, are obtained from a two-dimensional function which obeys predefined constraints. Using these weights on images compressed at higher bit rates produces a deblocked image which contains blurred "false" edges near real edges. We refer to this phenomenon as the ghosting effect. In order to prevent its occurrences, the weights of pixels, which belong to nonmonotone areas, are modified by dividing each pixel's weight by a predefined factor called a grade. This scheme is referred to as weight adaptation by grading (WABG). Better deblocking of monotone areas is achieved by applying three iterations of the WABG scheme on such areas followed by a fourth iteration which is applied on the rest of the image. We refer to this scheme as deblocking frames of variable size (DFOVS). DFOVS automatically adapts itself to the activity of each block. This new class of algorithms produces very good subjective results and PSNR results which are competitive relative to available state-of-the-art methods.

Accelerating Particle Filter Using Randomized Multiscale and Fast Multipole Type Methods.

Particle filter is a powerful tool for state tracking using non-linear observations. We present a multiscale based method that accelerates the tracking computation by particle filters. Unlike the conventional way, which calculates weights over all particles in each cycle of the algorithm, we sample a small subset from the source particles using matrix decomposition methods. Then, we apply a function extension algorithm that uses a particle subset to recover the density function for all the rest of the particles not included in the chosen subset. The computational effort is substantial especially when multiple objects are tracked concurrently. The proposed algorithm significantly reduces the computational load. By using the Fast Gaussian Transform, the complexity of the particle selection step is reduced to a linear time in n and k, where n is the number of particles and k is the number of particles in the selected subset. We demonstrate our method on both simulated and on real data such as object tracking in video sequences.

Fast motion estimation using bidirectional gradient methods.

Gradient-based motion estimation methods (GMs) are considered to be in the heart of state-of-the-art registration algorithms, being able to account for both pixel and subpixel registration and to handle various motion models (translation, rotation, affine, and projective). These methods estimate the motion between two images based on the local changes in the image intensities while assuming image smoothness. This paper offers two main contributions. The first is enhancement of the GM technique by introducing two new bidirectional formulations of the GM. These improve the convergence properties for large motions. The second is that we present an analytical convergence analysis of the GM and its properties. Experimental results demonstrate the applicability of these algorithms to real images.

A new family of spline-based biorthogonal wavelet transforms and their application to image compression.

In this paper. we design a new family of biorthogonal wavelet transforms and describe their applications to still image compression. The wavelet transforms are constructed from various types of interpolatory and quasiinterpolatory splines. The transforms use finite impulse response and infinite impulse response filters that are implemented in a fast lifting mode.

Multilayered image representation: application to image compression.

The main contribution of this work is a new paradigm for image representation and image compression. We describe a new multilayered representation technique for images. An image is parsed into a superposition of coherent layers: piecewise smooth regions layer, textures layer, etc. The multilayered decomposition algorithm consists in a cascade of compressions applied successively to the image itself and to the residuals that resulted from the previous compressions. During each iteration of the algorithm, we code the residual part in a lossy way: we only retain the most significant structures of the residual part, which results in a sparse representation. Each layer is encoded independently with a different transform, or basis, at a different bitrate, and the combination of the compressed layers can always be reconstructed in a meaningful way. The strength of the multilayer approach comes from the fact that different sets of basis functions complement each others: some of the basis functions will give reasonable account of the large trend of the data, while others will catch the local transients, or the oscillatory patterns. This multilayered representation has a lot of beautiful applications in image understanding, and image and video coding. We have implemented the algorithm and we have studied its capabilities.

CT reconstruction from parallel and fan-beam projections by a 2-D discrete Radon transform.

The discrete Radon transform (DRT) was defined by Abervuch as an analog of the continuous Radon transform for discrete data. Both the DRT and its inverse are computable in O(n(2) log n) operations for images of size n × n. In this paper, we demonstrate the applicability of the inverse DRT for the reconstruction of a 2-D object from its continuous projections. The DRT and its inverse are shown to model accurately the continuum as the number of samples increases. Numerical results for the reconstruction from parallel projections are presented. We also show that the inverse DRT can be used for reconstruction from fan-beam projections with equispaced detectors.