Wavelength encoding spectral imaging based on the combination of deeply learned filters and an RGB camera.

Hyperspectral imaging is a critical tool for gathering spatial-spectral information in various scientific research fields. As a result of improvements in spectral reconstruction algorithms, significant progress has been made in reconstructing hyperspectral images from commonly acquired RGB images. However, due to the limited input, reconstructing spectral information from RGB images is ill-posed. Furthermore, conventional camera color filter arrays (CFA) are designed for human perception and are not optimal for spectral reconstruction. To increase the diversity of wavelength encoding, we propose to place broadband encoding filters in front of the RGB camera. In this condition, the spectral sensitivity of the imaging system is determined by the filters and the camera itself. To achieve an optimal encoding scheme, we use an end-to-end optimization framework to automatically design the filters' transmittance functions and optimize the weights of the spectral reconstruction network. Simulation experiments show that our proposed spectral reconstruction network has excellent spectral mapping capabilities. Additionally, our novel joint wavelength encoding imaging framework is superior to traditional RGB imaging systems. We develop the deeply learned filter and conduct actual shooting experiments. The spectral reconstruction results have an attractive spatial resolution and spectral accuracy.

Reliable image dehazing by NeRF.

Image dehazing is a typical low-level visual task. With the continuous improvement of network performance and the introduction of various prior knowledge, the ability of image dehazing is becoming stronger. However, the existing dehazing methods have problems such as the inability to obtain real shooting datasets, unreliable dehazing processes, and the difficulty to deal with complex lighting scenes. To solve these problems, we propose a new haze model combining the optical scattering model and the computer graphics rendering. Based on the new haze model, we propose a high-quality and widely applicable dehazing dataset generation pipeline that does not require paired-data training and prior knowledge. We reconstruct the three-dimensional fog space with array camera and remove haze by thresholding voxel deletion. We use the Unreal Engine 5 to generate simulation datasets and the real shooting in laboratory to verify the effectiveness and the reliability of our generation pipeline. Through our pipeline, we can obtain wonderful dehaze results and dehaze datasets under various complex outdoors lighting conditions. We also propose a dehaze dataset enhancement method based on voxel control. Our pipeline and data enhancement are suitable for the latest algorithm model, these solutions can obtain better visual effects and objective indicators.

Design of an optimized Alvarez lens based on the fifth-order polynomial combination.

This paper proposes an optimized design of the Alvarez lens by utilizing a combination of three fifth-order X-Y polynomials. It can effectively minimize the curvature of the lens surface to meet the manufacturing requirements. The phase modulation function and aberration of the proposed lens are evaluated by using first-order optical analysis. Simulations compare the proposed lens with the traditional Alvarez lens in terms of surface curvature, zoom capability, and imaging quality. The results demonstrate the exceptional performance of the proposed lens, achieving a remarkable 26.36% reduction in the maximum curvature of the Alvarez lens (with a coefficient A value of 4×10-4 and a diameter of 26 mm) while preserving its original zoom capability and imaging quality.

Image restoration for optical zooming system based on Alvarez lenses.

Alvarez lenses are known for their ability to achieve a broad range of optical power adjustment by utilizing complementary freeform surfaces. However, these lenses suffer from optical aberrations, which restrict their potential applications. To address this issue, we propose a field of view (FOV) attention image restoration model for continuous zooming. In order to simulate the degradation of optical zooming systems based on Alvarez lenses (OZA), a baseline OZA is designed where the polynomial for the Alvarez lenses consists of only three coefficients. By computing spatially varying point spread functions (PSFs), we simulate the degraded images of multiple zoom configurations and conduct restoration experiments. The results demonstrate that our approach surpasses the compared methods in the restoration of degraded images across various zoom configurations while also exhibiting strong generalization capabilities under untrained configurations.

ASF-Transformer: neutralizing the impact of atmospheric turbulence on optical imaging through alternating learning in the spatial and frequency domains.

Atmospheric turbulence, a pervasive and complex physical phenomenon, challenges optical imaging across various applications. This paper presents the Alternating Spatial-Frequency (ASF)-Transformer, a learning-based method for neutralizing the impact of atmospheric turbulence on optical imaging. Drawing inspiration from split-step propagation and correlated imaging principles, we propose the Alternating Learning in Spatial and Frequency domains (LASF) mechanism. This mechanism utilizes two specially designed transformer blocks that alternate between the spatial and Fourier domains. Assisted by the proposed patch FFT loss, our model can enhance the recovery of intricate textures without the need for generative adversarial networks (GANs). Evaluated across diverse test mediums, our model demonstrated state-of-the-art performance in comparison to recent methods. The ASF-Transformer diverges from mainstream GAN-based solutions, offering a new strategy to combat image degradation introduced by atmospheric turbulence. Additionally, this work provides insights into neural network architecture by integrating principles from optical theory, paving the way for innovative neural network designs in the future.

Direct distortion prediction method for AR-HUD dynamic distortion correction.

Dynamic distortion is one of the most critical factors affecting the experience of automotive augmented reality head-up displays (AR-HUDs). A wide range of views and the extensive display area result in extraordinarily complex distortions. Existing methods based on the neural network first obtain distorted images and then get the predistorted data for training mostly. This paper proposes a distortion prediction framework based on the neural network. It directly trains the network with the distorted data, realizing dynamic adaptation for AR-HUD distortion correction and avoiding errors in coordinate interpolation. Additionally, we predict the distortion offsets instead of the distortion coordinates and present a field of view (FOV)-weighted loss function based on the spatial-variance characteristic to further improve the prediction accuracy of distortion. Experiments show that our methods improve the prediction accuracy of AR-HUD dynamic distortion without increasing the network complexity or data processing overhead.

Nighttime Image Dehazing by Render.

Nighttime image dehazing presents unique challenges due to the unevenly distributed haze caused by the color change of artificial light sources. This results in multiple interferences, including atmospheric light, glow, and direct light, which make the complex scattering haze interference difficult to accurately distinguish and remove. Additionally, obtaining pairs of high-definition data for fog removal at night is a difficult task. These challenges make nighttime image dehazing a particularly challenging problem to solve. To address these challenges, we introduced the haze scattering formula to more accurately express the haze in three-dimensional space. We also proposed a novel data synthesis method using the latest CG textures and lumen lighting technology to build scenes where various hazes can be seen clearly through ray tracing. We converted the complex 3D scattering relationship transformation into a 2D image dataset to better learn the mapping from 3D haze to 2D haze. Additionally, we improved the existing neural network and established a night haze intensity evaluation label based on the idea of optical PSF. This allowed us to adjust the haze intensity of the rendered dataset according to the intensity of the real haze image and improve the accuracy of dehazing. Our experiments showed that our data construction and network improvement achieved better visual effects, objective indicators, and calculation speed.

Snapshot hyperspectral imaging based on equalization designed DOE.

Hyperspectral imaging attempts to determine distinctive information in spatial and spectral domain of a target. Over the past few years, hyperspectral imaging systems have developed towards lighter and faster. In phase-coded hyperspectral imaging systems, a better coding aperture design can improve the spectral accuracy relatively. Using wave optics, we post an equalization designed phase-coded aperture to achieve desired equalization point spread functions (PSFs) which provides richer features for subsequent image reconstruction. During the reconstruction of images, our raised hyperspectral reconstruction network, CAFormer, achieves better results than the state-of-the-art networks with less computation by substituting self-attention with channel-attention. Our work revolves around the equalization design of the phase-coded aperture and optimizes the imaging process from three aspects: hardware design, reconstruction algorithm, and PSF calibration. Our work is putting snapshot compact hyperspectral technology closer to a practical application.

Elimination of varying chromatic aberrations based on diffractive optics.

In telescopic systems consisting of Alvarez lenses, chromatic aberrations vary with the magnifications and the fields of view. Computational imaging has developed rapidly in recent years, therefore we propose a method of optimizing the DOE and the post-processing neural network in 2 stages for achromatic aberrations. We apply the iterative algorithm and the gradient descent method to optimize the DOE, respectively, and then adopt U-Net to further optimize the results. The results show that the optimized DOEs improve the results, the gradient descent optimized DOE with U-Net performs the best and has a very robust and good performance in the case of simulated chromatic aberrations. The results also verify the validity of our algorithm.

Computational Optics for Mobile Terminals in Mass Production.

Correcting the optical aberrations and the manufacturing deviations of cameras is a challenging task. Due to the limitation on volume and the demand for mass production, existing mobile terminals cannot rectify optical degradation. In this work, we systematically construct the perturbed lens system model to illustrate the relationship between the deviated system parameters and the spatial frequency response (SFR) measured from photographs. To further address this issue, an optimization framework is proposed based on this model to build proxy cameras from the machining samples' SFRs. Engaging with the proxy cameras, we synthetic data pairs, which encode the optical aberrations and the random manufacturing biases, for training the learning-based algorithms. In correcting aberration, although promising results have been shown recently with convolutional neural networks, they are hard to generalize to stochastic machining biases. Therefore, we propose a dilated Omni-dimensional dynamic convolution (DOConv) and implement it in post-processing to account for the manufacturing degradation. Extensive experiments which evaluate multiple samples of two representative devices demonstrate that the proposed optimization framework accurately constructs the proxy camera. And the dynamic processing model is well-adapted to manufacturing deviations of different cameras, realizing perfect computational photography. The evaluation shows that the proposed method bridges the gap between optical design, system machining, and post-processing pipeline, shedding light on the joint of image signal reception (lens and sensor) and image signal processing (ISP).

Dark2Light: multi-stage progressive learning model for low-light image enhancement.

Due to severe noise and extremely low illuminance, restoring from low-light images to normal-light images remains challenging. Unpredictable noise can tangle the weak signals, making it difficult for models to learn signals from low-light images, while simply restoring the illumination can lead to noise amplification. To address this dilemma, we propose a multi-stage model that can progressively restore normal-light images from low-light images, namely Dark2Light. Within each stage, We divide the low-light image enhancement (LLIE) into two main problems: (1) illumination enhancement and (2) noise removal. Firstly, we convert the image space from sRGB to linear RGB to ensure that illumination enhancement is approximately linear, and design a contextual transformer block to conduct illumination enhancement in a coarse-to-fine manner. Secondly, a U-Net shaped denoising block is adopted for noise removal. Lastly, we design a dual-supervised attention block to facilitate progressive restoration and feature transfer. Extensive experimental results demonstrate that the proposed Dark2Light outperforms the state-of-the-art LLIE methods both quantitatively and qualitatively.

Deep learning-based image reconstruction for photonic integrated interferometric imaging.

Photonic integrated interferometric imaging (PIII) is an emerging technique that uses far-field spatial coherence measurements to extract intensity information from a source to form an image. At present, low sampling rate and noise disturbance are the main factors hindering the development of this technology. This paper implements a deep learning-based method to improve image quality. Firstly, we propose a frequency-domain dataset generation method based on imaging principles. Secondly, spatial-frequency dual-domain fusion networks (SFDF-Nets) are presented for image reconstruction. We utilize normalized amplitude and phase to train networks, which reduces the difficulty of network training using complex data. SFDF-Nets can fuse multi-frame data captured by rotation sampling to increase the sampling rate and generate high-quality spatial images through dual-domain supervised learning and frequency domain fusion. Furthermore, we propose an inverse fast Fourier transform loss (IFFT loss) for network training in the frequency domain. Extensive experiments show that our method improves PSNR and SSIM by 5.64 dB and 0.20, respectively. Our method effectively improves the reconstructed image quality and opens a new dimension in interferometric imaging.

Non-blind optical degradation correction via frequency self-adaptive and finetune tactics.

In mobile photography applications, limited volume constraints the diversity of optical design. In addition to the narrow space, the deviations introduced in mass production cause random bias to the real camera. In consequence, these factors introduce spatially varying aberration and stochastic degradation into the physical formation of an image. Many existing methods obtain excellent performance on one specific device but are not able to quickly adapt to mass production. To address this issue, we propose a frequency self-adaptive model to restore realistic features of the latent image. The restoration is mainly performed in the Fourier domain and two attention mechanisms are introduced to match the feature between Fourier and spatial domain. Our method applies a lightweight network, without requiring modification when the fields of view (FoV) changes. Considering the manufacturing deviations of a specific camera, we first pre-train a simulation-based model, then finetune it with additional manufacturing error, which greatly decreases the time and computational overhead consumption in implementation. Extensive results verify the promising applications of our technique for being integrated with the existing post-processing systems.

Fast method for designing broadband achromatic diffractive optical elements.

Diffractive optical elements play a crucial role in the miniaturization of the optical systems, especially in correcting achromatic aberration. Considering the rapidity and validity of the design method, we propose a fast method for designing broadband achromatic diffractive optical elements. Based on the direct binary search algorithm, some improvements have been made including the selection of the initial height map to mitigate the uncertainty, the reduction of the variations to accelerate the optimization and the increase of sampling rate to deal with the large operation bandwidth. The initial height map is calculated instead of random initial value. Due to different regions of the height map contributing to point spread functions differently, the variations are reduced to speed up the optimization. The large operation bandwidth is solved by increasing the sampling rate at unfitted wavelengths instead of setting weighting coefficients. We demonstrate via simulations that our method is effective through several examples. The design of broadband achromatic diffractive optical elements can be quickly achieved by our method.

Omnidirectional, thin metasurface exhibiting selective absorption for un-polarized broadband incidence.

Thin devices with large areas have strong and omnidirectional absorption over a wide bandwidth and are in demand for applications such as energy harvesting, structural color, and vehicle LiDAR (laser detection and ranging). Despite persistent efforts in the design and fabrication of such devices, the simultaneous realization of all these desired properties remains a challenge. In this study, a 190-nm-thick metasurface with an area of 3 cm2, incorporating dielectric cylinder arrays, a chromium layer, a silicon nitride (SiNx) layer, and an aluminum layer is theoretically and experimentally demonstrated. The developed device achieves an average absorptivity of ∼99% (97% in the experiment) in the entire visible spectrum 400-700 nm. Moreover, it exhibits strong absorption over a wide range of incident angles (∼91% and 90% at 60° in the calculation and experiment, respectively). Importantly, the feasibility of applying the developed metasurface absorber to solar thermophotovoltaics and vehicle LiDAR (laser detection and ranging) has been explored. Moreover, the photoresist can be replaced by other glues and easily scaled up to a large area using the roll-to-roll nanoimprinting process. With the excellent spectral properties and performance, this device is promising for large-area applications.

Image restoration for real-world under-display imaging.

Under-display imaging technique was recently proposed to enlarge the screen-to-body ratio for full-screen devices. However, existing image restoration algorithms have difficulty generalizing to real-world under-display (UD) images, especially to images containing strong light sources. To address this issue, we propose a novel method for building a synthetic dataset (CalibPSF dataset) and introduce a two-stage neural network to solve the under-display imaging degradation problem. The CalibPSF dataset is generated using the calibrated high dynamic range point spread function (PSF) of the under-display optical system and contains various simulated light sources. The two-stage network solves the color distortion and diffraction degradation in order. We evaluate the performance of our algorithm on our captured real-world test set. Comprehensive experiments demonstrate the superiority of our method in different dynamic range scenes.

Deep denoiser prior based deep analytic network for lensless image restoration.

Mask based lensless imagers have huge application prospects due to their ultra-thin body. However, the visual perception of the restored images is poor due to the ill conditioned nature of the system. In this work, we proposed a deep analytic network by imitating the traditional optimization process as an end-to-end network. Our network combines analytic updates with a deep denoiser prior to progressively improve lensless image quality over a few iterations. The convergence is proven mathematically and verified in the results. In addition, our method is universal in non-blind restoration. We detailed the solution for the general inverse problem and conducted five groups of deblurring experiments as examples. Both experimental results demonstrate that our method achieves superior performance against the existing state-of-the-art methods.

Research on Diagnostic Information of Smart Medical Care Based on Big Data.

The medical field has gradually become intelligent, and information and the research of intelligent medical diagnosis information have received extensive attention in the medical field. In response to this situation, this paper proposes a Hadoop-based medical big data processing system. The system first realized the ETL module based on Sqoop and the transmission function of unstructured data and then realized the distributed storage management function based on HDFS. Finally, a MapReduce algorithm with variable key values is proposed in the data statistical analysis module. Through simulation experiments on the system modules and each algorithm, the results show that because the traditional nondistributed big data acquisition module transmits the same scale of data, it consumes more than 3200 s and the transmission time exceeds 3000 s. The time consumption of smart medical care under the 6G protocol is 150 s, the transmission time is 146 s, and the time is reduced to 1/10 of the original. The research of intelligent medical diagnosis information based on big data has good rationality and feasibility.

Optimization of Process Control Parameters for Fully Mechanized Mining Face Based on ANN and GA.

In the traditional optimization mathod, the process control parameters for fully mechanized mining face are determined by experts or technicians based on their own experience, which is lack of scientific basis, and need long production adjustment cycle. It is cause large loss, and not conducive to improving mine production efficiency. In order to solve this problem, the study proposes a process control parameter optimization method based on a mixed strategy of artificial neural network and genetic algorithm and uses a cross-entropy cost function to optimize an artificial neural network, which improves the learning speed and fitting accuracy of the neural network. Using the historical production data of a fully mechanized coal mining face, taking the pulling speed of the shearer, hydraulic support moving speed, chain speed of scraper conveyor, chain speed of stage loader, emulsion pump outlet pressure, and spray pump outlet pressure as the optimization objects and taking the value range of each process control parameter as a constraint condition to establish a mixed strategy optimization model of process control parameters for a fully mechanized mining face, each process control parameter is optimized with the output of coal per minute as the optimization goal. The results show that the method has high accuracy and short optimization process time and can effectively improve the production efficiency of the working face.

Rotated rectangular aperture imaging through multi-frame blind deconvolution with Hyper-Laplacian priors.

Rotated rectangular aperture imaging has many advantages in large aperture telephoto systems due to its lower cost and lower complexity. This technology makes it possible to build super large aperture telescopes. In this paper, we combine the ideas of deblurring with rotated rectangular aperture imaging and propose an image synthesis algorithm based on multi-frame deconvolution. In the specific reconstruction process, Hyper-Laplacian priors and sparse priors are used, and an effective solution is developed. The simulation and real shooting experiments show that our algorithm has excellent performance in visual effect and objective evaluation. The synthetic images are significantly sharper than the results of the existing methods.