Lossy Compression for Lossless Prediction

Most data is automatically collected and only ever "seen" by algorithms. Yet, data compressors preserve perceptual fidelity rather than just the information needed by algorithms performing downstream tasks. In this paper, we characterize the bit-rate required to ensure high performance on all predictive tasks that are invariant under a set of transformations, such as data augmentations. Based on our theory, we design unsupervised objectives for training neural compressors. Using these objectives, we train a generic image compressor that achieves substantial rate savings (more than $1000\times$ on ImageNet) compared to JPEG on 8 datasets, without decreasing downstream classification performance.

PDF Abstract NeurIPS 2021 PDF NeurIPS 2021 Abstract

Results from the Paper


 Ranked #1 on Image Compression on ImageNet (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Benchmark
Image Compression Caltech101 Lossyless Compressor Bit rate 1340 # 1
Image Compression Cars-196 Lossyless Compressor Bit rate 1470 # 1
Image Compression CIFAR-10 Lossyless Compressor Bit rate 1410 # 1
Image Compression Food-101 Lossyless Compressor Bit rate 1270 # 1
Image Compression ImageNet Lossyless Compressor Bit rate 1350 # 1
Image Compression Oxford-IIIT Pets Lossyless Compressor Bit rate 1210 # 1
Image Compression PCam Lossyless Compressor Bit rate 1490 # 1
Image Compression STL-10 Lossyless Compressor Bit rate 1340 # 1

Methods


No methods listed for this paper. Add relevant methods here