Deep learning network for fusing optical and infrared images in a complex imaging environment by using the modified U-Net.

ABSTRACT

The fusion of optical and infrared images is a critical task in the field of image processing. However, it is challenging to achieve optimal results when fusing images from complex environments. In this paper, we propose a deep learning network model comprising an encoding network and a decoding network based on the modified U-Net network to fuse low-quality images from complex imaging environments. As both encoding and decoding networks use similar convolutional modules, they can share similar layer structures to improve the overall fusion performance. Furthermore, an attention mechanism module is integrated into the decoding network to identify and capture the crucial features of the fused images. It can assist the deep learning network to extract more relevant image features and thus get more accurate fusion. The proposed model has been compared with some existing methods to prove its performance in view of subjective and objective evaluations.