Polarized image super-resolution via a deep convolutional neural network.

Reduced resolution of polarized images makes it difficult to distinguish detailed polarization information and limits the ability to identify small targets and weak signals. A possible way to handle this problem is the polarization super-resolution (SR), which aims to obtain a high-resolution polarized image from a low-resolution one. However, compared with the traditional intensity-mode image SR, the polarization SR is more challenging because more channels and their nonlinear cross-links need to be considered as well as the polarization and intensity information need to be reconstructed simultaneously. This paper analyzes the polarized image degradation and proposes a deep convolutional neural network for polarization SR reconstruction based on two degradation models. The network structure and the well-designed loss function have been verified to effectively balance the restoration of intensity and polarization information, and can realize the SR with a maximum scaling factor of four. Experimental results show that the proposed method outperforms other SR methods in terms of both quantitative evaluation and visual effect evaluation for two degradation models with different scaling factors.

Pol2Pol: self-supervised polarimetric image denoising.

In this Letter, we present a self-supervised method, polarization to polarization (Pol2Pol), for polarimetric image denoising with only one-shot noisy images. First, a polarization generator is proposed to generate training image pairs, which are synthesized from one-shot noisy images by exploiting polarization relationships. Second, the Pol2Pol method is extensible and compatible, and any network that performs well in supervised image denoising tasks can be deployed to Pol2Pol after proper modifications. Experimental results show Pol2Pol outperforms other self-supervised methods and achieves comparable performance to supervised methods.

Physics-informed neural network for polarimetric underwater imaging.

Utilizing the polarization analysis in underwater imaging can effectively suppress the scattered light and help to restore target signals in turbid water. Neural network-based solutions can also boost the performance of polarimetric underwater imaging, while most of the existing networks are pure data driven which suffer from ignoring the physical mode. In this paper, we proposed an effective solution that informed the polarimetric physical model and constrains into the well-designed deep neural network. Especially compared with the conventional underwater imaging model, we mathematically transformed the two polarization-dependent parameters to a single parameter, making it easier for the network to converge to a better level. In addition, a polarization perceptual loss is designed and applied to the network to make full use of polarization information on the feature level rather than on the pixel level. Accordingly, the network was able to learn the polarization modulated parameter and to obtain clear de-scattered images. The experimental results verified that the combination of polarization model and neural network was beneficial to improve the image quality and outperformed other existing methods, even in a high turbidity condition.

Attention-based neural network for polarimetric image denoising.

In this Letter, we propose an attention-based neural network specially designed for the challenging task of polarimetric image denoising. In particular, the channel attention mechanism is used to effectively extract the features underlying the polarimetric images by rescaling the contributions of channels in the network. In addition, we also design the adaptive polarization loss to make the network focus on the polarization information. Experiments show that our method can well restore the details flooded by serious noise and outperforms previous methods. Moreover, the underlying mechanism of channel attention is revealed visually.

Underwater image restoration via Stokes decomposition.

In this Letter, we present a Stokes imaging-based method to restore objects and enhance image contrast in turbid water. In the system, a light source illuminates the objects with two orthometric polarization states; based on a new Stokes decomposition model, the recorded images are converted to Stokes maps and subsequently restored to a clear image, free of reflections and scattered lights. A mathematical model has been developed to explain the Stokes decomposition and how the undesired reflections and scattered lights are rejected. Imaging experiments have been devised and performed on different objects, e.g., metals and plastics, under different turbidities. The results demonstrate enhanced image quality and capability to distinguish polarization differences. This new, to the best of our knowledge, method can be readily applied to practical underwater object detection and potentially realize clear vision in other scattering media.

Unsupervised anomaly detection of MEMS in low illumination based on polarimetric Support Vector Data Description.

Low illuminated images make it challenging to conduct anomaly detection on material surface. Adding polarimetric information helps expand pixel range and recover background structure of network inputs. In this letter, an anomaly detection method in low illumination is proposed which utilizes polarization imaging and patch-wise Support Vector Data Description (SVDD) model. Polarimetric information of Micro Electromechanical System (MEMS) surface is captured by a division-of-focal- plane (DoFP) polarization camera and used to enhance low illuminated images. The enhanced images without defects serve as training sets of model to make it available for anomaly detection. The proposed method can generate heatmaps to locate defects correctly. It reaches 0.996 anomaly scores, which is 22.4% higher than that of low illuminated images and even higher than normal illuminated images.

Integration time optimization and starting angle autocalibration of full Stokes imagers based on a rotating retarder.

Full Stokes imaging can be performed with a continuously rotating retarder in front of a fixed polarizer and a standard camera (RRFP) or a division of a focal plane polarization camera (RRDOFP). We determine the optimal number and duration of intensity measurements through a cycle of the retarder for these two types of setups as a function of instrument and noise parameters. We show that this number mainly depends on the type of noise that corrupts the measurements. We also show that with these setups, the starting angle of the retarder need not be known precisely and can be autocalibrated, which facilitates synchronization of the rotating retarder with the camera. We investigate the precision and feasibility domain of this autocalibration and show the RRDOFP setup has more attractive properties compared with RRFP setup. These results are important to optimize and facilitate the operation of polarization imagers based on a rotating retarder.

Automatic underwater polarization imaging without background region or any prior.

Previous polarization underwater imaging methods based on the physical scattering model usually require background region included in the image and the prior knowledge, which hinders its practical application. In this paper, we analyze and optimize the physically feasible region and propose an improved method by degenerating intermediate variables, which can realize automatic underwater image recovery without background region or any prior. The proposed method does not need to estimate the intermediate variables in the traditional underwater imaging model and is adaptable to the underwater image with non-uniform illumination, which avoids the poor and unstable image recovery performance caused by inaccurate estimation of intermediate parameters due to the improper identification of the background region. Meanwhile, our method is effective for both images without background region and images in which the background region is hard to be identified. In addition, our method solves the significant variation in recovery results caused by the different selection of background regions and the inconsistency of parameter adjustment. The experimental results of different underwater scenes show that the proposed method can enhance image contrast while preserving image details without introducing considerable noise, and the proposed method is effective for the dense turbid medium.

Frequency interferometric localization microscopy.

We propose a new super-resolution microscopy, named frequency interferometric localization microscopy (FILM). FILM is implemented by incorporating a Michelson interferometer into wide-field fluorescence microscope, which introduces coherence time as a new auxiliary axis to obtain the spectral information of individual fluorophores. After the time-wavelength transformation, the homogeneous linewidth of individual fluorophores can be isolated from the inhomogeneous broadening distribution of the fluorophore ensemble. Thus, the nearby fluorophores with a distinguishable central wavelength can be separated in the frequency domain and localized with accuracy beyond the diffraction limit. The principle of the method, experimental schematics, and reconstruction algorithm are numerically demonstrated. With properly prepared fluorophores, FILM has the potential to reach, in principle, molecular-scale spatial resolution.

NaYF4:Yb/Tm@SiO2-Dox/Cur-CS/OSA nanoparticles with pH and photon responses.

Stimulus-triggered drug delivery systems (DDSs) based on lanthanide-doped upconversion nanoparticles (UCNPs) have attracted significant attention for treating cancers due to their merits of high drug availability, precisely controlled drug release, and low side-effects. However, such DDSs usually exhibit a single stimulus-response, which may limit the efficiency of cancer treatment. To extend response types in a single DDS, we construct NaYF4:Yb/Tm@SiO2-doxorubicin (Dox)/curcumin (Cur)-chitosan (CS)/2-Octen-1-ylsuccinic anhydride (OSA) nanoparticles with core-shell structures. Our method is based on the exploration of the synergistic effect of UCNPs and multiple drugs. In particular, the NaYF4:Yb/Tm is used to convert near-infrared light to visible light, activating Cur photosensitizers to produce singlet oxygen for photodynamic therapy, while CS/OSA responds to a low pH environment to release cancer drugs, including Dox and Cur for chemotherapy through breaking a free carboxyl group. The results show that the UCNPs with a 40 nm diameter, 23 nm thick mesoporous SiO2, and 19/1 mol% Yb3+/Tm3+concentrations could continuously release Dox and Cur at a pH value of 6.5 within 6 h after the excitation of a 980 nm-wavelength CW laser. Our study provides a promising approach for developing efficient DDSs for cancer treatment.

Graphene-based dual-mode modulators.

Mode-division multiplexing (MDM) has attracted broad attention as it could effectively boost up transmission capability by utilizing optical modes as a spatial dimension in optical interconnects. In such a technique, different data channels are usually modulated to the respective carriers over different spatial modes by using individual parallel electro-optic modulators. Each modulated channel is then multiplexed to a multi-mode waveguide. However, the method inevitably suffers from a high cost, large device footprint and high insertion loss. Here, we design intensity and phase dual-mode modulators, enabling simultaneous modulations over two channels via a graphene-on-silicon waveguide. Our method is based on the exploration of co-planar interactions between structured graphene nanoribbons (GNs) and spatial modes in a silicon waveguide. Specifically, the zeroth-order transverse electric (TE0) and first-order transverse electric (TE1) modes are modulated separately and simultaneously by applying independent driving electrodes to different GNs in an identical modulator. Our study is expected to open an avenue to develop high-density MDM photonics integrated circuits for tera-scale optical interconnects.

Theory of autocalibration feasibility and precision in full Stokes polarization imagers.

We propose a general theory of simultaneous estimation of Stokes vector and instrumental autocalibration of polarization imagers. This theory is applicable to any polarization imager defined by its measurement matrix. We illustrate it on the example of retardance autocalibration in a large class of polarization imagers based on rotating retarders and polarimeters. We show that although all these architectures can yield optimal estimation precision of the Stokes vector if they are properly configured, they do not have the same autocalibration capacity and have to be specifically optimized for that purpose. These results are important to determine the best compromise between autocalibration capacity and polarimetric precision in practical applications.

Temperature compensation of optical alternating magnetic field sensor via a novel method for on-line measuring.

The variation of environment temperature is a crucial problem for optical magnetic field sensors based on the magneto-optical crystal. In this paper, we propose a novel temperature compensation method for optical alternating magnetic field measuring by analyzing the demodulation principle and establishing the temperature compensation model, which can implement the functions of temperature compensation and on-line measuring simultaneously. Both the temperature and the alternating magnetic field flux density can be obtained only by adding two magnet rings on the magnetic field sensor. The experimental phenomenon agrees well with the temperature characteristics of the magneto-optical crystal and the theoretical compensation model. The experimental results demonstrate that this sensor has excellent stability whose max relative fluctuation is only 0.7402% in the range of 0-4 mT under a constant temperature. In the temperature compensation experiment of 0 °C, 20 °C and 40 °C, the sensor shows strong temperature robustness that the max absolute and relative errors are 0.07 mT and 3.50%, respectively. Meanwhile, compensation efficiency reaches 83.968%, which can effectively avoid temperature crosstalk to a large extent. Additionally, it has a better compensation performance whose max absolute and relative errors are 0.15 mT and 1.66% in the broader range of 0-16 mT when the actual temperature is accurately known.

Learning-based denoising for polarimetric images.

Based on measuring the polarimetric parameters which contain specific physical information, polarimetric imaging has been widely applied to various fields. However, in practice, the noise during image acquisition could lead to the output of noisy polarimetric images. In this paper, we propose, for the first time to our knowledge, a learning-based method for polarimetric image denoising. This method is based on the residual dense network and can significantly suppress the noise in polarimetric images. The experimental results show that the proposed method has an evident performance on the noise suppression and outperforms other existing methods. Especially for the images of the degree of polarization and the angle of polarization, which are quite sensitive to the noise, the proposed learning-based method can well reconstruct the details flooded in strong noise.

IPLNet: a neural network for intensity-polarization imaging in low light.

Imaging in low light is significant but challenging in many applications. Adding the polarization information into the imaging system compromises the drawbacks of the conventional intensity imaging to some extent. However, generally speaking, the qualities of intensity images and polarization images cannot be compatible due to the characteristic differences in polarimetric operators. In this Letter, we collected, to the best of our knowledge, the first polarimetric imaging dataset in low light and present a specially designed neural network to enhance the image qualities of intensity and polarization simultaneously. Both indoor and outdoor experiments demonstrate the effectiveness and superiority of this neural network-based solution, which may find important applications for object detection and vision in photon-starved environments.

When is retardance autocalibration of microgrid-based full Stokes imagers possible and useful?

Full Stokes polarimetric images can be obtained from two acquisitions with a microgrid polarization camera equipped with a retarder. When the retardance is imperfectly known, it can be calibrated from the measurements, but this requires three image acquisitions and may cause divergence of estimation variance at a low signal-to-noise ratio. We determine closed-form equations allowing one to decide in which experimental conditions autocalibration is possible and useful, and to quantify the performance gain obtained in practice. These results are validated by real-world experiments.

Fundamental precision limits of full Stokes polarimeters based on DoFP polarization cameras for an arbitrary number of acquisitions.

As an emerging technology, division-of-focal-plane (DoFP) polarization cameras have raised attention due to their integrated structure. In this paper, we address the fundamental precision limits of full Stokes polarimeters based on a linear DoFP polarization camera and a controllable retarder in the presence of additive and Poisson shot noise. We demonstrate that if the number of image acquisitions is greater than or equal to three, there exists retarder configurations that reach the theoretical lower bound on estimation variance. Examples of such configurations are one rotatable retarder with fixed retardance of 125.26° or two rotatable quarter-waveplates (QWPs) in pair. In contrast, the lower bound cannot be reached with a single QWP or a single variable retarder with fixed orientation. These results are important to get the most out of DoFP polarization imagers in real applications.

Waveguide-integrated graphene spatial mode filters for on-chip mode-division multiplexing.

On-chip mode-division multiplexing (MDM), as a promising method of scaling communication bandwidth, has attracted tremendous attention due to potential applications in optical interconnects ranging from intra-chip to board-to-board communications. However, the MDM technique usually suffers from signal degradation due to crosstalk between spatial modes in multimode waveguide devices. Here we design waveguide-integrated graphene spatial mode filters for on-chip MDM to overcome this limitation. Specifically, TE1-mode-pass and TE2-mode-pass filters are designed based on different waveguide architectures. For the TE1-mode-pass filters, we have obtained a maximum TE1-to-TE0 modal extinction ratio (ER) of 9.19 dB in a 200-µm-long waveguide. While, for the TE2-mode-pass filters, we have achieved a maximum TE2-to-TE1 modal ER of 5.37 dB and TE2-to-TE0 modal ER of 6.44 dB in a 200-µm-long waveguide. Our study could help improve the signal-to-noise ratio for on-chip MDM optical interconnects.

Contrast optimization in broadband passive polarimetric imaging based on color camera.

Broadband polarimetric imaging consists of forming an image under spectrally wide illumination after having optimized the polarization state analyzer (PSA) to maximize the target/background discriminability. In previous works, the image sensor was monochrome, and only the intensity contrast was optimized. However, due to its spectrally varying response, the PSA not only changes the light's intensity, but also its color. This color information can serve as a further parameter to improve discrimination. In this paper, we employ a color camera in a broadband Stokes (passive) polarimetric imaging system and take into color difference's contribution to discrimination ability in optimizing the PSA setting. We show through experiments that a significant improvement of discrimination ability over monochrome imaging is obtained, especially when there are multiple objects in the scene.

Optimal tradeoff between precision and sampling rate in DoFP imaging polarimeters.

A linear division-of-focal-plane camera combined with a controllable polarization modulator constitutes a versatile full-Stokes imager with four possible sampling rate modes, depending on the number of acquisitions. Considering several polarization modulator architectures, we determine the parameter settings that minimize estimation variance in each sampling rate mode, so that precision, sampling rate, and acquisition time can be optimally and dynamically balanced to implement the imaging solution best adapted to a given application.