Search Results for author:
Found 30 papers, 11 papers with code
Fisher-Rao Gradient Flows of Linear Programs and State-Action Natural Policy Gradients
Kakade's natural policy gradient method has been studied extensively in the last years showing linear convergence with and without regularization.
The Real Tropical Geometry of Neural Networks
The parameter space of ReLU neural networks is contained as a semialgebraic set inside the parameter space of tropical rational functions.
Benign overfitting in leaky ReLU networks with moderate input dimension
The problem of benign overfitting asks whether it is possible for a model to perfectly fit noisy training data and still generalize well.
Mildly Overparameterized ReLU Networks Have a Favorable Loss Landscape
We study the loss landscape of both shallow and deep, mildly overparameterized ReLU neural networks on a generic finite input dataset for the squared error loss.
Function Space and Critical Points of Linear Convolutional Networks
We study the geometry of linear networks with one-dimensional convolutional layers.
Critical Points and Convergence Analysis of Generative Deep Linear Networks Trained with Bures-Wasserstein Loss
We consider a deep matrix factorization model of covariance matrices trained with the Bures-Wasserstein distance.
Expected Gradients of Maxout Networks and Consequences to Parameter Initialization
We study the gradients of a maxout network with respect to inputs and parameters and obtain bounds for the moments depending on the architecture and the parameter distribution.
Geometry and convergence of natural policy gradient methods
We study the convergence of several natural policy gradient (NPG) methods in infinite-horizon discounted Markov decision processes with regular policy parametrizations.
FoSR: First-order spectral rewiring for addressing oversquashing in GNNs
On the other hand, adding edges to the message-passing graph can lead to increasingly similar node representations and a problem known as oversmoothing.
Enumeration of max-pooling responses with generalized permutohedra
We investigate the combinatorics of max-pooling layers, which are functions that downsample input arrays by taking the maximum over shifted windows of input coordinates, and which are commonly used in convolutional neural networks.
Oversquashing in GNNs through the lens of information contraction and graph expansion
We compare the spectral expansion properties of our algorithm with that of an existing curvature-based non-local rewiring strategy.
On the effectiveness of persistent homology
The goal of this work is to identify some types of problems where PH performs well or even better than other methods in data analysis.
Solving infinite-horizon POMDPs with memoryless stochastic policies in state-action space
Reward optimization in fully observable Markov decision processes is equivalent to a linear program over the polytope of state-action frequencies.
Training Wasserstein GANs without gradient penalties
We propose a stable method to train Wasserstein generative adversarial networks.
Learning curves for Gaussian process regression with power-law priors and targets
We characterize the power-law asymptotics of learning curves for Gaussian process regression (GPR) under the assumption that the eigenspectrum of the prior and the eigenexpansion coefficients of the target function follow a power law.
The Geometry of Memoryless Stochastic Policy Optimization in Infinite-Horizon POMDPs
We then describe the optimization problem as a linear optimization problem in the space of feasible state-action frequencies subject to polynomial constraints that we characterize explicitly.
Geometry of Linear Convolutional Networks
We study the family of functions that are represented by a linear convolutional neural network (LCN).
On the Expected Complexity of Maxout Networks
Learning with neural networks relies on the complexity of the representable functions, but more importantly, the particular assignment of typical parameters to functions of different complexity.
Weisfeiler and Lehman Go Cellular: CW Networks
Nevertheless, these models can be severely constrained by the rigid combinatorial structure of Simplicial Complexes (SCs).
Ranked #1 on
Graph Regression
on ZINC 100k
Information Complexity and Generalization Bounds
We present a unifying picture of PAC-Bayesian and mutual information-based upper bounds on the generalization error of randomized learning algorithms.
Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sums
We present results on the number of linear regions of the functions that can be represented by artificial feedforward neural networks with maxout units.
Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks
The pairwise interaction paradigm of graph machine learning has predominantly governed the modelling of relational systems.
Can neural networks learn persistent homology features?
Topological data analysis uses tools from topology -- the mathematical area that studies shapes -- to create representations of data.
Distributed Learning via Filtered Hyperinterpolation on Manifolds
Learning mappings of data on manifolds is an important topic in contemporary machine learning, with applications in astrophysics, geophysics, statistical physics, medical diagnosis, biochemistry, 3D object analysis.
Implicit Bias of Gradient Descent for Mean Squared Error Regression with Two-Layer Wide Neural Networks
\hj{For stochastic gradient descent we obtain the same implicit bias result.}
Optimization Theory for ReLU Neural Networks Trained with Normalization Layers
The success of deep neural networks is in part due to the use of normalization layers.
Stochastic Feedforward Neural Networks: Universal Approximation
We investigate different types of shallow and deep architectures, and the minimal number of layers and units per layer that are sufficient and necessary in order for the network to be able to approximate any given stochastic mapping from the set of inputs to the set of outputs arbitrarily well.
How Well Do WGANs Estimate the Wasserstein Metric?
Generative modelling is often cast as minimizing a similarity measure between a data distribution and a model distribution.
The Variational Deficiency Bottleneck
We introduce a bottleneck method for learning data representations based on information deficiency, rather than the more traditional information sufficiency.
On the Number of Linear Regions of Deep Neural Networks
We study the complexity of functions computable by deep feedforward neural networks with piecewise linear activations in terms of the symmetries and the number of linear regions that they have.
