Search Results for author: Guido Montufar

Found 28 papers, 6 papers with code

Characterizing the Spectrum of the NTK via a Power Series Expansion

1 code implementation15 Nov 2022 Michael Murray, Hui Jin, Benjamin Bowman, Guido Montufar

We provide expressions for the coefficients of this power series which depend on both the Hermite coefficients of the activation function as well as the depth of the network.

Spectral Bias Outside the Training Set for Deep Networks in the Kernel Regime

no code implementations6 Jun 2022 Benjamin Bowman, Guido Montufar

This bias depends on the model architecture and input distribution alone and thus does not depend on the target function which does not need to be in the RKHS of the kernel.

Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks

no code implementations ICLR 2022 Benjamin Bowman, Guido Montufar

We study the dynamics of a neural network in function space when optimizing the mean squared error via gradient flow.

PAC-Bayes and Information Complexity

no code implementations ICLR Workshop Neural_Compression 2021 Pradeep Kr. Banerjee, Guido Montufar

We point out that a number of well-known PAC-Bayesian-style and information-theoretic generalization bounds for randomized learning algorithms can be derived under a common framework starting from a fundamental information exponential inequality.

Generalization Bounds

Wasserstein Proximal of GANs

no code implementations ICLR 2019 Alex Tong Lin, Wuchen Li, Stanley Osher, Guido Montufar

We introduce a new method for training generative adversarial networks by applying the Wasserstein-2 metric proximal on the generators.

How Framelets Enhance Graph Neural Networks

1 code implementation13 Feb 2021 Xuebin Zheng, Bingxin Zhou, Junbin Gao, Yu Guang Wang, Pietro Lio, Ming Li, Guido Montufar

The graph neural networks with the proposed framelet convolution and pooling achieve state-of-the-art performance in many node and graph prediction tasks.


Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks

no code implementations21 Dec 2020 Quynh Nguyen, Marco Mondelli, Guido Montufar

In this paper, we provide tight bounds on the smallest eigenvalue of NTK matrices for deep ReLU nets, both in the limiting case of infinite widths and for finite widths.


Kernelized Wasserstein Natural Gradient

1 code implementation ICLR 2020 Michael Arbel, Arthur Gretton, Wuchen Li, Guido Montufar

Many machine learning problems can be expressed as the optimization of some cost functional over a parametric family of probability distributions.

Haar Graph Pooling

1 code implementation ICML 2020 Yu Guang Wang, Ming Li, Zheng Ma, Guido Montufar, Xiaosheng Zhuang, Yanan Fan

Deep Graph Neural Networks (GNNs) are useful models for graph classification and graph-based regression tasks.

General Classification Graph Classification +1

HaarPooling: Graph Pooling with Compressive Haar Basis

no code implementations25 Sep 2019 Yu Guang Wang, Ming Li, Zheng Ma, Guido Montufar, Xiaosheng Zhuang, Yanan Fan

The input of each pooling layer is transformed by the compressive Haar basis of the corresponding clustering.

Graph Classification

On the Dynamics and Convergence of Weight Normalization for Training Neural Networks

no code implementations25 Sep 2019 Yonatan Dukler, Quanquan Gu, Guido Montufar

We present a proof of convergence for ReLU networks trained with weight normalization.

Wasserstein Diffusion Tikhonov Regularization

no code implementations15 Sep 2019 Alex Tong Lin, Yonatan Dukler, Wuchen Li, Guido Montufar

We propose regularization strategies for learning discriminative models that are robust to in-class variations of the input data.

Data Augmentation

Decentralized Multi-Agents by Imitation of a Centralized Controller

no code implementations6 Feb 2019 Alex Tong Lin, Mark J. Debord, Katia Estabridis, Gary Hewer, Guido Montufar, Stanley Osher

In order to obtain multi-agents that act in a decentralized manner, we introduce a novel algorithm under the popular framework of centralized training, but decentralized execution.

Imitation Learning Multi-agent Reinforcement Learning +2

Restricted Boltzmann Machines: Introduction and Review

1 code implementation19 Jun 2018 Guido Montufar

The restricted Boltzmann machine is a network of stochastic units with undirected interactions between pairs of visible and hidden units.

BIG-bench Machine Learning

Geometry of Policy Improvement

no code implementations6 Apr 2017 Guido Montufar, Johannes Rauh

We investigate the geometry of optimal memoryless time independent decision making in relation to the amount of information that the acting agent has about the state of the system.

Decision Making LEMMA

Evaluating Morphological Computation in Muscle and DC-motor Driven Models of Human Hopping

2 code implementations1 Dec 2015 Keyan Ghazi-Zahedi, Daniel F. B. Haeufle, Guido Montufar, Syn Schmitt, Nihat Ay

An important aspect of morphological computation is that it cannot be assigned to an embodied system per se, but that it is, as we show, behavior- and state-dependent.

Dimension of Marginals of Kronecker Product Models

no code implementations10 Nov 2015 Guido Montufar, Jason Morton

The limit is described by the tropical morphism; a piecewise linear map with pieces corresponding to slicings of the visible matrix by the normal fan of the hidden matrix.

Hierarchical Models as Marginals of Hierarchical Models

no code implementations14 Aug 2015 Guido Montufar, Johannes Rauh

We investigate the representation of hierarchical models in terms of marginals of other hierarchical models with smaller interactions.

Universal Approximation of Markov Kernels by Shallow Stochastic Feedforward Networks

no code implementations24 Mar 2015 Guido Montufar

We establish upper bounds for the minimal number of hidden units for which a binary stochastic feedforward network with sigmoid activation probabilities and a single hidden layer is a universal approximator of Markov kernels.

Geometry and Determinism of Optimal Stationary Control in Partially Observable Markov Decision Processes

no code implementations24 Mar 2015 Guido Montufar, Keyan Ghazi-Zahedi, Nihat Ay

For partially observable Markov decision processes (POMDPs), optimal memoryless policies are generally stochastic.

Deep Narrow Boltzmann Machines are Universal Approximators

no code implementations14 Nov 2014 Guido Montufar

We show that deep narrow Boltzmann machines are universal approximators of probability distributions on the activities of their visible units, provided they have sufficiently many hidden layers, each containing the same number of units as the visible layer.

Expressive Power and Approximation Errors of Restricted Boltzmann Machines

no code implementations NeurIPS 2011 Guido Montufar, Johannes Rauh, Nihat Ay

We present explicit classes of probability distributions that can be learned by Restricted Boltzmann Machines (RBMs) depending on the number of units that they contain, and which are representative for the expressive power of the model.

Geometry and Expressive Power of Conditional Restricted Boltzmann Machines

no code implementations14 Feb 2014 Guido Montufar, Nihat Ay, Keyan Ghazi-Zahedi

Conditional restricted Boltzmann machines are undirected stochastic neural networks with a layer of input and output units connected bipartitely to a layer of hidden units.

On the number of response regions of deep feed forward networks with piece-wise linear activations

no code implementations20 Dec 2013 Razvan Pascanu, Guido Montufar, Yoshua Bengio

For a $k$ layer model with $n$ hidden units on each layer it is $\Omega(\left\lfloor {n}/{n_0}\right\rfloor^{k-1}n^{n_0})$.

Discrete Restricted Boltzmann Machines

no code implementations15 Jan 2013 Guido Montufar, Jason Morton

We describe discrete restricted Boltzmann machines: probabilistic graphical models with bipartite interactions between visible and hidden discrete variables.

Cannot find the paper you are looking for? You can Submit a new open access paper.