Colorful image reconstruction from neuromorphic event cameras with biologically inspired deep color fusion neural networks.
Bioinspir Biomim
; 19(3)2024 Mar 04.
Article
en En
| MEDLINE
| ID: mdl-38373337
ABSTRACT
Neuromorphic event-based cameras communicate transients in luminance instead of frames, providing visual information with a fine temporal resolution, high dynamic range and high signal-to-noise ratio. Enriching event data with color information allows for the reconstruction of colorful frame-like intensity maps, supporting improved performance and visually appealing results in various computer vision tasks. In this work, we simulated a biologically inspired color fusion system featuring a three-stage convolutional neural network for reconstructing color intensity maps from event data and sparse color cues. While current approaches for color fusion use full RGB frames in high resolution, our design uses event data and low-spatial and tonal-resolution quantized color cues, providing a high-performing small model for efficient colorful image reconstruction. The proposed model outperforms existing coloring schemes in terms of SSIM, LPIPS, PSNR, and CIEDE2000 metrics. We demonstrate that auxiliary limited color information can be used in conjunction with event data to successfully reconstruct both color and intensity frames, paving the way for more efficient hardware designs.
Palabras clave
Texto completo:
1
Bases de datos:
MEDLINE
Asunto principal:
Redes Neurales de la Computación
/
Benchmarking
Idioma:
En
Revista:
Bioinspir Biomim
Asunto de la revista:
BIOLOGIA
/
ENGENHARIA BIOMEDICA
Año:
2024
Tipo del documento:
Article
País de afiliación:
Israel