no code implementations • 27 Sep 2019 • O Duranthon, M Marsili, R. Xie
We show that the mutual information between the representation of a learning machine and the hidden features that it extracts from data is bounded from below by the relevance, which is the entropy of the model's energy distribution.
