Deep Learning for Visual Data Compression

Abstract

With the tremendous success of neural networks, a few learning-based image codecs were proposed and outperformed those traditional image codecs. However, the field of learning-based compression research for other categories of visual data has remained much less explored. This thesis will investigate the effectiveness of deep learning for visual data compression and propose three end-to-end learning-based compression methods for respectively compressing standard videos, 3D volumetric images and stereo videos. First, we improve the existing learning-based video codecs by using a newly proposed adaptive coding method called Resolution-adaptive Motion Coding (RaMC) to effectively compress the introduced motion information for reducing the bit-rate cost. Then, we investigate the effectiveness of deep learning for lossless 3D volumetric image compression and propose the first end-to-end optimized learning framework for losslessly compressing 3D volumetric images. We introduce an Intra-slice and Inter-slice Conditional Entropy Coding (ICEC) module to fuse multi-scale intra-slice and inter-slice features as the context information for better entropy coding. Besides the aforementioned single-view visual data, we further attempt to employ the neural networks for compressing the multi-view visual data and propose the first end-to-end Learning-based Stereo Video Compression (LSVC) framework. It compresses both left and right views of the stereo video by using deep motion and disparity compensation strategy with fully-differentiable modules and can be optimized in an end-to-end manner. We conduct extensive experiments on multiple publicly available datasets to demonstrate the effectiveness of our proposed RaMC, ICEC, and LSVC methods. The results indicate that these three methods achieve state-of-the-art compression performance in the corresponding visual data compression tasks and outperform traditional visual data compression frameworks.