Using Conditional Video Compressors for Image Restoration

To address the ill-posed nature of image restoration tasks, recent research efforts have been focused on integrating conditional generative models, such as conditional variational autoencoders (CVAE). However, how to condition the autoencoder to maximize the conditional evidence lower bound remains an open issue, particularly for the restoration tasks. Inspired by the rapid advancements in CVAE-based video compression, we make the first attempt to adapt a conditional video compressor for image restoration. In doing so, we have the low-quality image to be enhanced, which plays the same role as the reference frame for conditional video coding. Our scheme applies scalar quantization in training the autoencoder, circumventing the difficulties of training a large-size codebook as with prior works that adopt vector-quantized VAE (VQ-VAE). Moreover, it trains end-to-end a fully conditioned autoencoder, including a conditional encoder, a conditional decoder, and a conditional prior network, to maximize the conditional evidence lower bound. Extensive experiments confirm the superiority of our scheme on denoising and deblurring tasks.