Melhoria de Qualidade de Vídeo Comprimido: soluções com Redes Neurais Profundas para Múltiplos Codecs

Abstract

The compression process necessary to enable video transmission and storage generates artifacts that reduce image quality, thereby resulting in a poorer user experience. To mitigate these artifacts, filtering strategies are typically applied to the decompressed frames, which are classified as in-loop and post-processing strategies. The later functions as a second layer of filtering, as some artifacts persist after in-loop filtering. Since they are not tied to any specific codec, these filters can be used as post-processing for any codec. The most recent post-processing filtering techniques are based on Deep Neural Networks, specifically Convolutional Neural Networks (CNNs), such as Spatio-Temporal Deformable Fusion (STDF), which is the state-of-the-art technique for video quality enhancement. However, as the results obtained in this research show, the STDF architecture does not yield good results when tested with other coding standards and quantization levels, causing quality losses in certain scenarios. Therefore, this work proposes the exploration of training techniques for the STDF architecture with the aim of improving its generalization capability across various video codecs. The first and second solutions are similar in methodology, proposing training based on a mixed dataset consisting of videos encoded by AOMedia Video 1 (AV1) and Versatile Video Coding (VVC). Unlike the first solution, which is trained from scratch without using a pre-trained model, the second solution employs fine-tuning strategy, starting from the original STDF model. The third solution is based on the multi-domain training paradigm, where each domain corresponds to a video codec. Experimental results show that the third solution achieved objective quality improvements of up to 1.234 dB. On average, the quality improvement achieved was 0.544 dB, and the solution proved capable of enhancing visual quality for all tested encoders and videos, being generic enough to be used as a single post-processing filter for multiple encoding standards/formats.