Transformed pupil-function misalignment calibration strategy for Fourier ptychographic microscopy.

Fourier ptychographic microscopy (FPM) is an enabling quantitative phase imaging technique with both high-resolution (HR) and wide field-of-view (FOV), which can surpass the diffraction limit of the objective lens by employing an LED array to provide angular-varying illumination. The precise illumination angles are critical to ensure exact reconstruction, while it's difficult to separate actual positional parameters in conventional algorithmic self-calibration approaches due to the mixing of multiple systematic error sources. In this paper, we report a pupil-function-based strategy for independently calibrating the position of LED array. We first deduce the relationship between positional deviation and pupil function in the Fourier domain through a common iterative route. Then, we propose a judgment criterion to determine the misalignment situations, which is based on the arrangement of LED array in the spatial domain. By combining the mapping of complex domains, we can accurately solve the spatial positional parameters concerning the LED array through a boundary-finding scheme. Relevant simulations and experiments demonstrate the proposed method is accessible to precisely correct the positional misalignment of LED array. The approach based on the pupil function is expected to provide valuable insights for precise position correction in the field of microscopy.

Polarimetry-inspired feature fusion spectroscopy (PIFFS) for ammonia sensing in water.

The sustainable use of water resources is inseparable from water pollution detection. The sensing of toxic ammonia nitrogen in water currently requires auxiliary reagents, which may cause secondary pollution. Benefiting from the ability of substances to change light characteristics, this work proposes polarimetry-inspired feature fusion spectroscopy (PIFFS) to detect ammonia. The PIFFS system mainly includes a light source, a quarter-wave plate (QWP), a linear polarizer (LP) and a fiber spectrometer. The target light containing substance information is polarization modulated by adjusting the QWP and LP angles. Then, the Stokes parameters of target light can be calculated by appropriate modulations. The feasibility of PIFFS method to detect ammonia nitrogen is verified by experiments on both standard water samples and environmental water samples. Experimental results show that inspired by the first Stokes parameter, the fused features provide superiority in classifying ammonia concentration. The results also demonstrate the effectiveness of support vector machine-based concentration classification and random forests-based spectral selection. The interaction between light and substances ensures that the proposed PIFFS method has the potential to detect other pollutants.

Research on Small Sample Dynamic Human Ear Recognition Based on Deep Learning.

Due to the problem of insufficient dynamic human ear data, the Changchun University dynamic human ear (CCU-DE) database, which is a small sample human ear database, was developed in this study. The database fully considers the various complex situations and posture changes of human ear images, such as translation angle, rotation angle, illumination change, occlusion and interference, etc., making the research of dynamic human ear recognition closer to complex real-life situations, and increasing the applicability of human ear dynamic recognition. In order to test the practicability and effectiveness of the developed CCU-DE small sample database, we designed a dynamic human ear recognition system block diagram based on a deep learning model, which was pre-trained by a migration learning method. Aiming at multi-posture changes under different contrasts, translation and rotation motions, and with or without occlusion, simulation studies were conducted using the CCU-DE small sample database and different deep learning models, such as YOLOv3, YOLOv4, YOLOv5, Faster R-CNN, and SSD. The experimental results showed that the CCU-DE database can be well used for dynamic ear recognition, and it can be tested by using different deep learning models with higher test accuracy.

Multiscale Joint Optimization Strategy for Retinal Vascular Segmentation.

The accurate segmentation of retinal vascular is of great significance for the diagnosis of diseases such as diabetes, hypertension, microaneurysms and arteriosclerosis. In order to segment more deep and small blood vessels and provide more information to doctors, a multi-scale joint optimization strategy for retinal vascular segmentation is presented in this paper. Firstly, the Multi-Scale Retinex (MSR) algorithm is used to improve the uneven illumination of fundus images. Then, the multi-scale Gaussian matched filtering method is used to enhance the contrast of the retinal images. Optimized by the Particle Swarm Optimization (PSO) algorithm, Otsu algorithm (OTSU) multi-threshold segmentation is utilized to segment the retinal image extracted by the multi-scale matched filtering method. Finally, the image is post-processed, including binarization, morphological operation and edge-contour removal. The test experiments are implemented on the DRIVE and STARE datasets to evaluate the effectiveness and practicability of the proposed method. Compared with other existing methods, it can be concluded that the proposed method can segment more small blood vessels while ensuring the integrity of vascular structure and has a higher performance. The proposed method has more obvious targets, a higher contrast, more plentiful detailed information, and local features. The qualitative and quantitative analysis results show that the presented method is superior to the other advanced methods.

An Experimental Study on Combustion and Cycle-by-Cycle Variations of an N-Butanol Engine with Hydrogen Direct Injection under Lean Burn Conditions.

This study experimentally investigated the effects of hydrogen direct injection on combustion and the cycle-by-cycle variations in a spark ignition n-butanol engine under lean burn conditions. For this purpose, a spark ignition engine installed with a hydrogen and n-butanol dual fuel injection system was specially developed. Experiments were conducted at four excess air ratios, four hydrogen fractions(φ(𝐻2)) and pure n-butanol. Engine speed and intake manifold absolute pressure (MAP) were kept at 1500 r/min and 43 kPa, respectively. The results indicate that the θ0-10 and θ10-90 decreased gradually with the increase in hydrogen fraction. Additionally, the indicated mean effective pressure (IMEP), the peak cylinder pressure (Pmax) and the maximum rate of pressure rise ((dP/dφ)max) increased gradually, while their cycle-by-cycle variations decreased with the increase in hydrogen fraction. In addition, the correlation between the (dP/dφ)max and its corresponding crank angle became weak with the increase in the excess air coefficient (λ), which tends to be strongly correlated with the increase in hydrogen fraction. The coefficient of variation of the Pmax and the IMEP increased with the increase in λ, while they decreased obviously after blending in the hydrogen under lean burn conditions. Furthermore, when λ was 1.0, a 5% hydrogen fraction improved the cycle-by-cycle variations most significantly. While a larger hydrogen fraction is needed to achieve the excellent combustion characteristics under lean burn conditions, hydrogen direct injection can promote combustion process and is beneficial for enhancing stable combustion and reducing the cycle-by-cycle variations.

Deep Multi-Feature Transfer Network for Fourier Ptychographic Microscopy Imaging Reconstruction.

Fourier ptychographic microscopy (FPM) is a potential imaging technique, which is used to achieve wide field-of-view (FOV), high-resolution and quantitative phase information. The LED array is used to irradiate the samples from different angles to obtain the corresponding low-resolution intensity images. However, the performance of reconstruction still suffers from noise and image data redundancy, which needs to be considered. In this paper, we present a novel Fourier ptychographic microscopy imaging reconstruction method based on a deep multi-feature transfer network, which can achieve good anti-noise performance and realize high-resolution reconstruction with reduced image data. First, in this paper, the image features are deeply extracted through transfer learning ResNet50, Xception and DenseNet121 networks, and utilize the complementarity of deep multiple features and adopt cascaded feature fusion strategy for channel merging to improve the quality of image reconstruction; then the pre-upsampling is used to reconstruct the network to improve the texture details of the high-resolution reconstructed image. We validate the performance of the reported method via both simulation and experiment. The model has good robustness to noise and blurred images. Better reconstruction results are obtained under the conditions of short time and low resolution. We hope that the end-to-end mapping method of neural network can provide a neural-network perspective to solve the FPM reconstruction.

Deep Learning in Medical Hyperspectral Images: A Review.

With the continuous progress of development, deep learning has made good progress in the analysis and recognition of images, which has also triggered some researchers to explore the area of combining deep learning with hyperspectral medical images and achieve some progress. This paper introduces the principles and techniques of hyperspectral imaging systems, summarizes the common medical hyperspectral imaging systems, and summarizes the progress of some emerging spectral imaging systems through analyzing the literature. In particular, this article introduces the more frequently used medical hyperspectral images and the pre-processing techniques of the spectra, and in other sections, it discusses the main developments of medical hyperspectral combined with deep learning for disease diagnosis. On the basis of the previous review, tne limited factors in the study on the application of deep learning to hyperspectral medical images are outlined, promising research directions are summarized, and the future research prospects are provided for subsequent scholars.

Combining Regional Energy and Intuitionistic Fuzzy Sets for Infrared and Visible Image Fusion.

To get more obvious target information and more texture features, a new fusion method for the infrared (IR) and visible (VIS) images combining regional energy (RE) and intuitionistic fuzzy sets (IFS) is proposed, and this method can be described by several steps as follows. Firstly, the IR and VIS images are decomposed into low- and high-frequency sub-bands by non-subsampled shearlet transform (NSST). Secondly, RE-based fusion rule is used to obtain the low-frequency pre-fusion image, which allows the important target information preserved in the resulting image. Based on the pre-fusion image, the IFS-based fusion rule is introduced to achieve the final low-frequency image, which enables more important texture information transferred to the resulting image. Thirdly, the 'max-absolute' fusion rule is adopted to fuse high-frequency sub-bands. Finally, the fused image is reconstructed by inverse NSST. The TNO and RoadScene datasets are used to evaluate the proposed method. The simulation results demonstrate that the fused images of the proposed method have more obvious targets, higher contrast, more plentiful detailed information, and local features. Qualitative and quantitative analysis results show that the presented method is superior to the other nine advanced fusion methods.

Scaling-Based Two-Step Reconstruction in Full Polarization-Compressed Hyperspectral Imaging.

Polarized hyperspectral images can reflect the rich physicochemical characteristics of targets. Meanwhile, the contained plentiful information also brings great challenges to signal processing. Although compressive sensing theory provides a good idea for image processing, the simplified compression imaging system has difficulty in reconstructing full polarization information. Focused on this problem, we propose a two-step reconstruction method to handle polarization characteristics of different scales progressively. This paper uses a quarter-wave plate and a liquid crystal tunable filter to achieve full polarization compression and hyperspectral imaging. According to their numerical features, the Stokes parameters and their modulation coefficients are simultaneously scaled. The first Stokes parameter is reconstructed in the first step based on compressive sensing. Then, the last three Stokes parameters with similar order of magnitude are reconstructed in the second step based on previous results. The simulation results show that the two-step reconstruction method improves the reconstruction accuracy by 7.6 dB for the parameters that failed to be reconstructed by the non-optimized method, and reduces the reconstruction time by 8.25 h without losing the high accuracy obtained by the current optimization method. This feature scaling method provides a reference for the fast and high-quality reconstruction of physical quantities with obvious numerical differences.

Compressive spectral imaging system for soil classification with three-dimensional convolutional neural network.

Compressive spectral imaging systems have promising applications in the field of object classification. However, for soil classification problem, conventional methods addressing this specific task often fail to produce satisfying results due to the tradeoff between the invariance and discrepancy of each soil. In this paper, we explore a liquid crystal tunable filters (LCTF)-based system and propose a three-dimensional convolutional neural network (3D-CNN) for soil classification. We first obtain a set of soil compressive measurements via a low spatial resolution detector, and soil hyperspectral images are reconstructed with improved resolution in spatial as well as spectral domains by a compressive sensing (CS) method. Furthermore, different from previous spectral-based object classification methods restricted to extract features from each type independently, on account of the potential of spectral property on individual solid, our method proposes to apply the principal component analysis(PCA) to achieve a dimensionality reduction in the spectral domain. Then, we explore a differential perception model for flexible feature extraction, and finally introduce a 3D-CNN framework to solve the multi-soil classification problem. Experimental results demonstrate that our algorithm not only is able to accelerate the ability of feature discriminability but also performs against conventional soil classification methods.

Fourier ptychographic microscopy reconstruction with multiscale deep residual network.

Fourier ptychographic microscopy (FPM) is a newly developed microscopic technique for large field of view, high-resolution and quantitative phase imaging by combining the techniques from ptychographic imaging, aperture synthesizing and phase retrieval. In FPM, an LED array is utilized to illuminate the specimen from different angles and the corresponding intensity images are synthesized to reconstruct a high-resolution complex field. As a flexible and low-cost approach to achieve high-resolution, wide-field and quantitative phase imaging, FPM is of enormous potential in biomedical applications such as hematology and pathology. Conventionally, the FPM reconstruction problem is solved by using a phase retrieval method, termed Alternate Projection. By iteratively updating the Fourier spectrum with low-resolution-intensity images, the result converges to a high-resolution complex field. Here we propose a new FPM reconstruction framework with deep learning methods and design a multiscale, deep residual neural network for FPM reconstruction. We employ the widely used open-source deep learning library PyTorch to train and test our model and carefully choose the hyperparameters of our model. To train and analyze our model, we build a large-scale simulation dataset with an FPM imaging model and an actual dataset captured with an FPM system. The simulation dataset and actual dataset are separated as training datasets and test datasets, respectively. Our model is trained with the simulation training dataset and fine tuned with the fine-tune dataset, which contains actual training data. Both our model and the conventional method are tested on the simulation test dataset and the actual test dataset to evaluate the performances. We also show the reconstruction result of another neural network-based method for comparison. The experiments demonstrate that our model achieves better reconstruction results and consumes much less time than conventional methods. The results also point out that our model is more robust under system aberrations such as noise and blurring (fewer intensity images) compared with conventional methods.

Automatic and Robust Infrared-Visible Image Sequence Registration via Spatio-Temporal Association.

To solve the problems of the large differences in gray value and inaccurate positioning of feature information during infrared-visible image registration, we propose an automatic and robust algorithm for registering planar infrared-visible image sequences through spatio-temporal association. In particular, we first create motion vector distribution descriptors which represent the temporal motion information of foreground contours in adjacent frames to complete coarse registration without feature extraction. Then, for precise registration, we extracted FAST corners of the foreground, which are described by the spatial location distribution of contour points based on connected blob detection, and match these corners using bidirectional optimal maximum strategy. Finally, a reservoir updated by Better-In, Worse-Out (BIWO) strategy is established to save matched point pairs and obtain the optimal global transformation matrix. Extensive evaluations on the LITIV dataset well demonstrate the effectiveness of the proposed algorithm. Particularly, our algorithm achieves lower registration overlapping errors than the other two state-of-the-arts.

Compressive spectral imaging system based on liquid crystal tunable filter.

Liquid crystal tunable filters (LCTF) are extensively used in hyperspectral imaging systems to successively acquire different spectral components of scenes by adjusting the center wavelength of the filter. However, the spectral and spatial resolutions of the imager are limited by the bandwidth of LCTF, and the pitch dimension of the detector, respectively. This paper applies compressive sensing principles to improve both of the spatial and spectral resolutions of the LCTF-based hyperspectral imaging system. An accurate transmission model of the LCTF is used to represent its bandpass filtering effects on the spectra. In addition, a random coded aperture placed behind the LCTF is used to modulate the spectral images in the spatial domain. Then, the three-dimensional encoded spectral images are projected onto a two-dimensional detector. Benefiting from the spectral-dependent transmission property of the LCTF, information of the entire spectrum is collected by a few snapshots using different center wavelengths of the LCTF. Super-resolution hyperspectral images can be reconstructed from a small set of compressive measurements by solving a convex optimization problem. Simulations and experiments show that the proposed method can effectively improve the spectral and spatial resolutions of traditional LCTF-based spectral imager without changing the structures of the LCTF and detector. Finally, a multi-channel spectral coding method is proposed to further increase the compression capacity of the system.

Adaptive Correlation Model for Visual Tracking Using Keypoints Matching and Deep Convolutional Feature.

Although correlation filter (CF)-based visual tracking algorithms have achieved appealing results, there are still some problems to be solved. When the target object goes through long-term occlusions or scale variation, the correlation model used in existing CF-based algorithms will inevitably learn some non-target information or partial-target information. In order to avoid model contamination and enhance the adaptability of model updating, we introduce the keypoints matching strategy and adjust the model learning rate dynamically according to the matching score. Moreover, the proposed approach extracts convolutional features from a deep convolutional neural network (DCNN) to accurately estimate the position and scale of the target. Experimental results demonstrate that the proposed tracker has achieved satisfactory performance in a wide range of challenging tracking scenarios.

Light field endoscopy and its parametric description.

In this Letter, we demonstrate the application of light field imaging to endoscopy. By introducing a microlens array into the image plane of a conventional endoscope, the 4D light field can be captured in one snapshot. This information can be used to obtain perspective images and to digitally refocus to different planes. These features allow for the recovery of 3D information in minimally invasive surgery. Important optical setup and performance parameters are derived to enable task specific engineering of the light field imaging system.

Consistently Sampled Correlation Filters with Space Anisotropic Regularization for Visual Tracking.

Most existing correlation filter-based tracking algorithms, which use fixed patches and cyclic shifts as training and detection measures, assume that the training samples are reliable and ignore the inconsistencies between training samples and detection samples. We propose to construct and study a consistently sampled correlation filter with space anisotropic regularization (CSSAR) to solve these two problems simultaneously. Our approach constructs a spatiotemporally consistent sample strategy to alleviate the redundancies in training samples caused by the cyclical shifts, eliminate the inconsistencies between training samples and detection samples, and introduce space anisotropic regularization to constrain the correlation filter for alleviating drift caused by occlusion. Moreover, an optimization strategy based on the Gauss-Seidel method was developed for obtaining robust and efficient online learning. Both qualitative and quantitative evaluations demonstrate that our tracker outperforms state-of-the-art trackers in object tracking benchmarks (OTBs).

A Hierarchical Framework Combining Motion and Feature Information for Infrared-Visible Video Registration.

In this paper, we propose a novel hierarchical framework that combines motion and feature information to implement infrared-visible video registration on nearly planar scenes. In contrast to previous approaches, which involve the direct use of feature matching to find the global homography, the framework adds coarse registration based on the motion vectors of targets to estimate scale and rotation prior to matching. In precise registration based on keypoint matching, the scale and rotation are used in re-location to eliminate their impact on targets and keypoints. To strictly match the keypoints, first, we improve the quality of keypoint matching by using normalized location descriptors and descriptors generated by the histogram of edge orientation. Second, we remove most mismatches by counting the matching directions of correspondences. We tested our framework on a public dataset, where our proposed framework outperformed two recently-proposed state-of-the-art global registration methods in almost all tested videos.

Multi-View Structural Local Subspace Tracking.

In this paper, we propose a multi-view structural local subspace tracking algorithm based on sparse representation. We approximate the optimal state from three views: (1) the template view; (2) the PCA (principal component analysis) basis view; and (3) the target candidate view. Then we propose a unified objective function to integrate these three view problems together. The proposed model not only exploits the intrinsic relationship among target candidates and their local patches, but also takes advantages of both sparse representation and incremental subspace learning. The optimization problem can be well solved by the customized APG (accelerated proximal gradient) methods together with an iteration manner. Then, we propose an alignment-weighting average method to obtain the optimal state of the target. Furthermore, an occlusion detection strategy is proposed to accurately update the model. Both qualitative and quantitative evaluations demonstrate that our tracker outperforms the state-of-the-art trackers in a wide range of tracking scenarios.

Alleviate Similar Object in Visual Tracking via Online Learning Interference-Target Spatial Structure.

Correlation Filter (CF) based trackers have demonstrated superior performance to many complex scenes in smart and autonomous systems, but similar object interference is still a challenge. When the target is occluded by a similar object, they not only have similar appearance feature but also are in same surrounding context. Existing CF tracking models only consider the target's appearance information and its surrounding context, and have insufficient discrimination to address the problem. We propose an approach that integrates interference-target spatial structure (ITSS) constraints into existing CF model to alleviate similar object interference. Our approach manages a dynamic graph of ITSS online, and jointly learns the target appearance model, similar object appearance model and the spatial structure between them to improve the discrimination between the target and a similar object. Experimental results on large benchmark datasets OTB-2013 and OTB-2015 show that the proposed approach achieves state-of-the-art performance.

An Automatic Multi-Target Independent Analysis Framework for Non-Planar Infrared-Visible Registration.

In this paper, we propose a novel automatic multi-target registration framework for non-planar infrared-visible videos. Previous approaches usually analyzed multiple targets together and then estimated a global homography for the whole scene, however, these cannot achieve precise multi-target registration when the scenes are non-planar. Our framework is devoted to solving the problem using feature matching and multi-target tracking. The key idea is to analyze and register each target independently. We present a fast and robust feature matching strategy, where only the features on the corresponding foreground pairs are matched. Besides, new reservoirs based on the Gaussian criterion are created for all targets, and a multi-target tracking method is adopted to determine the relationships between the reservoirs and foreground blobs. With the matches in the corresponding reservoir, the homography of each target is computed according to its moving state. We tested our framework on both public near-planar and non-planar datasets. The results demonstrate that the proposed framework outperforms the state-of-the-art global registration method and the manual global registration matrix in all tested datasets.