Search Results for author:
Found 8 papers, 1 papers with code
A Probabilistic Representation of DNNs: Bridging Mutual Information and Generalization
However, it is intractable to accurately estimate the MI in DNNs, thus most previous works have to relax the MI bound, which in turn weakens the information theoretic explanation for generalization.
A Probabilistic Representation for Deep Learning: Delving into The Information Bottleneck Principle
The Information Bottleneck (IB) principle has recently attracted great attention to explaining Deep Neural Networks (DNNs), and the key is to accurately estimate the mutual information between a hidden layer and dataset.
A Probabilistic Representation of Deep Learning for Improving The Information Theoretic Interpretability
Based on the probabilistic explanations for MLPs, we improve the information-theoretic interpretability of MLPs in three aspects: (i) the random variable of f is discrete and the corresponding entropy is finite; (ii) the information bottleneck theory cannot correctly explain the information flow in MLPs if we take into account the back-propagation; and (iii) we propose novel information-theoretic explanations for the generalization of MLPs.
PAC-Bayesian Generalization Bounds for MultiLayer Perceptrons
We study PAC-Bayesian generalization bounds for Multilayer Perceptrons (MLPs) with the cross entropy loss.
Probabilistic modeling the hidden layers of deep neural networks
Based on the probabilistic representation, we demonstrate that the entire architecture of DNNs can be explained as a Bayesian hierarchical model.
A Probabilistic Representation of Deep Learning
In this work, we introduce a novel probabilistic representation of deep learning, which provides an explicit explanation for the Deep Neural Networks (DNNs) in three aspects: (i) neurons define the energy of a Gibbs distribution; (ii) the hidden layers of DNNs formulate Gibbs distributions; and (iii) the whole architecture of DNNs can be interpreted as a Bayesian neural network.
A synthetic dataset for deep learning
In this paper, we propose a novel method for generating a synthetic dataset obeying Gaussian distribution.
