Search Results for author:
Found 28 papers, 6 papers with code
Discrete Restricted Boltzmann Machines
We describe discrete restricted Boltzmann machines: probabilistic graphical models with bipartite interactions between visible and hidden discrete variables.
On the number of response regions of deep feed forward networks with piece-wise linear activations
For a $k$ layer model with $n$ hidden units on each layer it is $\Omega(\left\lfloor {n}/{n_0}\right\rfloor^{k-1}n^{n_0})$.
Geometry and Expressive Power of Conditional Restricted Boltzmann Machines
Conditional restricted Boltzmann machines are undirected stochastic neural networks with a layer of input and output units connected bipartitely to a layer of hidden units.
Expressive Power and Approximation Errors of Restricted Boltzmann Machines
We present explicit classes of probability distributions that can be learned by Restricted Boltzmann Machines (RBMs) depending on the number of units that they contain, and which are representative for the expressive power of the model.
Deep Narrow Boltzmann Machines are Universal Approximators
We show that deep narrow Boltzmann machines are universal approximators of probability distributions on the activities of their visible units, provided they have sufficiently many hidden layers, each containing the same number of units as the visible layer.
Universal Approximation of Markov Kernels by Shallow Stochastic Feedforward Networks
We establish upper bounds for the minimal number of hidden units for which a binary stochastic feedforward network with sigmoid activation probabilities and a single hidden layer is a universal approximator of Markov kernels.
Geometry and Determinism of Optimal Stationary Control in Partially Observable Markov Decision Processes
For partially observable Markov decision processes (POMDPs), optimal memoryless policies are generally stochastic.
Hierarchical Models as Marginals of Hierarchical Models
We investigate the representation of hierarchical models in terms of marginals of other hierarchical models with smaller interactions.
Dimension of Marginals of Kronecker Product Models
The limit is described by the tropical morphism; a piecewise linear map with pieces corresponding to slicings of the visible matrix by the normal fan of the hidden matrix.
Evaluating Morphological Computation in Muscle and DC-motor Driven Models of Human Hopping
An important aspect of morphological computation is that it cannot be assigned to an embodied system per se, but that it is, as we show, behavior- and state-dependent.
Information Theoretically Aided Reinforcement Learning for Embodied Agents
Reinforcement learning for embodied agents is a challenging problem.
Geometry of Policy Improvement
We investigate the geometry of optimal memoryless time independent decision making in relation to the amount of information that the acting agent has about the state of the system.
Mixtures and products in two graphical models
We compare two statistical models of three binary random variables.
Restricted Boltzmann Machines: Introduction and Review
The restricted Boltzmann machine is a network of stochastic units with undirected interactions between pairs of visible and hidden units.
Decentralized Multi-Agents by Imitation of a Centralized Controller
In order to obtain multi-agents that act in a decentralized manner, we introduce a novel algorithm under the popular framework of centralized training, but decentralized execution.
Wasserstein Diffusion Tikhonov Regularization
We propose regularization strategies for learning discriminative models that are robust to in-class variations of the input data.
Haar Graph Pooling
Deep Graph Neural Networks (GNNs) are useful models for graph classification and graph-based regression tasks.
On the Dynamics and Convergence of Weight Normalization for Training Neural Networks
We present a proof of convergence for ReLU networks trained with weight normalization.
HaarPooling: Graph Pooling with Compressive Haar Basis
The input of each pooling layer is transformed by the compressive Haar basis of the corresponding clustering.
Kernelized Wasserstein Natural Gradient
Many machine learning problems can be expressed as the optimization of some cost functional over a parametric family of probability distributions.
Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks
In this paper, we provide tight bounds on the smallest eigenvalue of NTK matrices for deep ReLU nets, both in the limiting case of infinite widths and for finite widths.
How Framelets Enhance Graph Neural Networks
The graph neural networks with the proposed framelet convolution and pooling achieve state-of-the-art performance in many node and graph prediction tasks.
Wasserstein Proximal of GANs
We introduce a new method for training generative adversarial networks by applying the Wasserstein-2 metric proximal on the generators.
PAC-Bayes and Information Complexity
We point out that a number of well-known PAC-Bayesian-style and information-theoretic generalization bounds for randomized learning algorithms can be derived under a common framework starting from a fundamental information exponential inequality.
Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks
We study the dynamics of a neural network in function space when optimizing the mean squared error via gradient flow.
Spectral Bias Outside the Training Set for Deep Networks in the Kernel Regime
This bias depends on the model architecture and input distribution alone and thus does not depend on the target function which does not need to be in the RKHS of the kernel.
Characterizing the Spectrum of the NTK via a Power Series Expansion
We provide expressions for the coefficients of this power series which depend on both the Hermite coefficients of the activation function as well as the depth of the network.
Supermodular Rank: Set Function Decomposition and Optimization
We define the supermodular rank of a function on a lattice.
