Mutual Information Estimation

13 papers with code • 0 benchmarks • 0 datasets

To estimate mutual information from samples, specially for high-dimensional variables.

Datasets


Greatest papers with code

Practical and Consistent Estimation of f-Divergences

google-research/google-research NeurIPS 2019

The estimation of an f-divergence between two probability distributions based on samples is a fundamental problem in statistics and machine learning.

Machine Learning Mutual Information Estimation +1

Learning deep representations by mutual information estimation and maximization

rdevon/DIM ICLR 2019

In this work, we perform unsupervised learning of representations by maximizing mutual information between an input and the output of a deep neural network encoder.

General Classification Mutual Information Estimation +1

Empowerment-driven Exploration using Mutual Information Estimation

navneet-nmk/pytorch-rl 11 Oct 2018

However, many of the state of the art deep reinforcement learning algorithms, that rely on epsilon-greedy, fail on these environments.

Montezuma's Revenge Mutual Information Estimation

Graph Representation Learning via Graphical Mutual Information Maximization

zpeng27/GMI 4 Feb 2020

The richness in the content of various information networks such as social networks and communication networks provides the unprecedented potential for learning high-quality expressive representations without external supervision.

Graph Representation Learning Link Prediction +2

Scalable Mutual Information Estimation using Dependence Graphs

mrtnoshad/EDGE 27 Jan 2018

To the best of our knowledge EDGE is the first non-parametric MI estimator that can achieve parametric MSE rates with linear time complexity.

Information Plane Mutual Information Estimation +1

Neural Entropic Estimation: A faster path to mutual information estimation

ccha23/MI-NEE 30 May 2019

In particular, we show that MI-NEE reduces to MINE in the special case when the reference distribution is the product of marginal distributions, but faster convergence is possible by choosing the uniform distribution as the reference distribution instead.

Mutual Information Estimation

copent: Estimating Copula Entropy and Transfer Entropy in R

majianthu/copent 27 May 2020

Copula Entropy is a mathematical concept defined by Ma and Sun for multivariate statistical independence measuring and testing, and also proved to be closely related to conditional independence (or transfer entropy).

Causal Discovery Machine Learning +3

Better Long-Range Dependency By Bootstrapping A Mutual Information Regularizer

BorealisAI/BMI 28 May 2019

In this work, we develop a novel regularizer to improve the learning of long-range dependency of sequence data.

General Classification Language Modelling +1

Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Model

lijuntaopku/UFD 23 Nov 2020

Experimental results show that our proposed method achieves significant performance improvements over the state-of-the-art pretrained cross-lingual language model in the CLCD setting.

Language Modelling Mutual Information Estimation +1