Information Plane

7 papers with code • 0 benchmarks • 0 datasets

To obtain the Information Plane (IP) of deep neural networks, which shows the trajectories of the hidden layers during training in a 2D plane using as coordinate axes the mutual information between the input and the hidden layer, and the mutual information between the output and the hidden layer.

Datasets


Greatest papers with code

Opening the Black Box of Deep Neural Networks via Information

ravidziv/IDNNs 2 Mar 2017

Previous work proposed to analyze DNNs in the \textit{Information Plane}; i. e., the plane of the Mutual Information values that each layer preserves on the input and output variables.

Information Plane

On the Information Bottleneck Theory of Deep Learning

artemyk/ibsgd ICLR 2018

The practical successes of deep neural networks have not been matched by theoretical progress that satisfyingly explains their behavior.

Information Plane

Scalable Mutual Information Estimation using Dependence Graphs

mrtnoshad/EDGE 27 Jan 2018

To the best of our knowledge EDGE is the first non-parametric MI estimator that can achieve parametric MSE rates with linear time complexity.

Information Plane Mutual Information Estimation +1

Compression phase is not necessary for generalization in representation learning

Sungyeop/IPRL 15 Feb 2021

Thus, we conclude that the compression phase is not necessary for generalization in representation learning.

Information Plane Representation Learning

The Dual Information Bottleneck

ravidziv/dual_IB 8 Jun 2020

The Information Bottleneck (IB) framework is a general characterization of optimal representations obtained using a principled approach for balancing accuracy and complexity.

Information Plane

Malicious Network Traffic Detection via Deep Learning: An Information Theoretic View

erickgalinkin/jhu_masters 16 Sep 2020

Applying our results can serve to guide analysis methods for machine learning engineers and suggests that neural networks that can exploit the convolution theorem are equally accurate as standard convolutional neural networks, and can be more computationally efficient.

Feature Engineering Information Plane +1

On the Information Plane of Autoencoders

nicolasigor/entropy 15 May 2020

Recently, the Information Plane (IP) was proposed to analyze them, which is based on the information-theoretic concept of mutual information (MI).

Information Plane Mutual Information Estimation