RESUMEN
Deep learning methods have achieved impressive performance in compressed video quality enhancement tasks. However, these methods rely excessively on practical experience by manually designing the network structure and do not fully exploit the potential of the feature information contained in the video sequences, i.e., not taking full advantage of the multiscale similarity of the compressed artifact information and not seriously considering the impact of the partition boundaries in the compressed video on the overall video quality. In this article, we propose a novel Mixed Difference Equation inspired Transformer (MDEformer) for compressed video quality enhancement, which provides a relatively reliable principle to guide the network design and yields a new insight into the interpretable transformer. Specifically, drawing on the graphical concept of the mixed difference equation (MDE), we utilize multiple cross-layer cross-attention aggregation (CCA) modules to establish long-range dependencies between encoders and decoders of the transformer, where partition boundary smoothing (PBS) modules are inserted as feedforward networks. The CCA module can make full use of the multiscale similarity of compression artifacts to effectively remove compression artifacts, and recover the texture and detail information of the frame. The PBS module leverages the sensitivity of smoothing convolution to partition boundaries to eliminate the impact of partition boundaries on the quality of compressed video and improve its overall quality, while not having too much impacts on non-boundary pixels. Extensive experiments on the MFQE 2.0 dataset demonstrate that the proposed MDEformer can eliminate compression artifacts for improving the quality of the compressed video, and surpasses the state-of-the-arts (SOTAs) in terms of both objective metrics and visual quality.
RESUMEN
Reconstructing a high-resolution hyperspectral image (HSI) from a low-resolution HSI is significant for many applications, such as remote sensing and aerospace. Most deep learning-based HSI super-resolution methods pay more attention to developing novel network structures but rarely study the HSI super-resolution problem from the perspective of image dynamic evolution. In this article, we propose that the HSI pixel motion during the super-resolution reconstruction process can be analogized to the particle movement in the smoothed particle hydrodynamics (SPH) field. To this end, we design an SPH network (SPH-Net) for HSI super-resolution in light of the SPH theory. Specifically, we construct a smooth function based on SPH and design a smooth convolution in multiscales to exploit spectral correlation and preserve the spectral information in the super-resolved image. In addition, we apply the SPH approximation method to discretize the Navier-Stokes motion equation into SPH equation form, which can guide the HSI pixel motion in the desired direction during super-resolution reconstruction, thereby producing clear edges in the spatial domain. Experiments on three public hyperspectral datasets demonstrate that the proposed SPH-Net outperforms the state-of-the-art methods in terms of objective metrics and visual quality.
