What Information Does a ResNet Compress?

ICLR 2019 Luke Nicholas DarlowAmos Storkey

The information bottleneck principle (Shwartz-Ziv & Tishby, 2017) suggests that SGD-based training of deep neural networks results in optimally compressed hidden layers, from an information theoretic perspective. However, this claim was established on toy data... (read more)

PDF Abstract ICLR 2019 PDF ICLR 2019 Abstract

Code


No code implementations yet. Submit your code now

Tasks


Results from the Paper


  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.

Methods used in the Paper