Unpacking Information Bottlenecks: Unifying Information-Theoretic Objectives in Deep Learning

27 Mar 2020Andreas KirschClare LyleYarin Gal

The information bottleneck (IB) principle offers both a mechanism to explain how deep neural networks train and generalize, as well as a regularized objective with which to train models. However, multiple competing objectives have been proposed based on this principle, and the information-theoretic quantities in these objectives are difficult to compute for large deep neural networks... (read more)

PDF Abstract


No code implementations yet. Submit your code now


Results from the Paper

  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.

Methods used in the Paper

🤖 No Methods Found Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet