Learning JPEG Compression Artifacts for Image Manipulation Detection and Localization

30 Aug 2021  ·  Myung-Joon Kwon, Seung-Hun Nam, In-Jae Yu, Heung-Kyu Lee, Changick Kim ·

Detecting and localizing image manipulation are necessary to counter malicious use of image editing techniques. Accordingly, it is essential to distinguish between authentic and tampered regions by analyzing intrinsic statistics in an image. We focus on JPEG compression artifacts left during image acquisition and editing. We propose a convolutional neural network (CNN) that uses discrete cosine transform (DCT) coefficients, where compression artifacts remain, to localize image manipulation. Standard CNNs cannot learn the distribution of DCT coefficients because the convolution throws away the spatial coordinates, which are essential for DCT coefficients. We illustrate how to design and train a neural network that can learn the distribution of DCT coefficients. Furthermore, we introduce Compression Artifact Tracing Network (CAT-Net) that jointly uses image acquisition artifacts and compression artifacts. It significantly outperforms traditional and deep neural network-based methods in detecting and localizing tampered regions.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Image Manipulation Localization Casia V1+ CAT-Net v2 Average Pixel F1(Fixed threshold) .752 # 3
Image Manipulation Detection Casia V1+ CAT-Net v2 AUC .942 # 1
Balanced Accuracy .838 # 3
Image Manipulation Localization CocoGlide CAT-Net v2 Average Pixel F1(Fixed threshold) .434 # 7
Image Manipulation Detection CocoGlide CAT-Net v2 AUC .667 # 5
Balanced Accuracy .580 # 4
Image Manipulation Localization Columbia CAT-Net v2 Average Pixel F1(Fixed threshold) .859 # 3
Image Manipulation Detection Columbia CAT-Net v2 AUC .977 # 4
Balanced Accuracy .803 # 4
Image Manipulation Detection COVERAGE CAT-Net v2 AUC .680 # 6
Balanced Accuracy .635 # 4
Image Manipulation Localization COVERAGE CAT-Net v2 Average Pixel F1(Fixed threshold) .381 # 6
Image Manipulation Detection DSO-1 CAT-Net v2 AUC .747 # 5
Balanced Accuracy .525 # 4
Image Manipulation Localization DSO-1 CAT-Net v2 Average Pixel F1(Fixed threshold) .584 # 4

Methods