Search Results for author:
Found 17 papers, 2 papers with code
Group Equivariant Fourier Neural Operators for Partial Differential Equations
We consider solving partial differential equations (PDEs) with Fourier neural operators (FNOs), which operate in the frequency domain.
SineNet: Learning Temporal Dynamics in Time-Dependent Partial Differential Equations
While the U-Net architecture with skip connections is commonly used by prior studies to enable multi-scale processing, our analysis shows that the need for features to evolve across layers results in temporally misaligned features in skip connections, which limits the model's performance.
Can Shallow Neural Networks Beat the Curse of Dimensionality? A mean field training perspective
Thus gradient descent training for fitting reasonably smooth, but truly high-dimensional data may be subject to the curse of dimensionality.
Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernels
We establish a scale separation of Kolmogorov width type between subspaces of a given Banach space under the condition that a sequence of linear maps converges much faster on one of the subspaces.
On the Convergence of Gradient Descent Training for Two-layer ReLU-networks in the Mean Field Regime
The condition does not depend on the initalization of parameters and concerns only the weak convergence of the realization of the neural network, not its parameter distribution.
Representation formulas and pointwise properties for Barron functions
We study the natural function space for infinitely wide two-layer neural networks with ReLU activation (Barron space) and establish different representation formulae.
On the Banach spaces associated with multi-layer ReLU networks: Function representation, approximation theory and gradient descent dynamics
The key to this work is a new way of representing functions in some form of expectations, motivated by multi-layer neural networks.
Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't
The purpose of this article is to review the achievements made in the last few years towards the understanding of the reasons behind the success and subtleties of neural network-based machine learning.
A priori estimates for classification problems using neural networks
We consider binary and multi-class classification problems using hypothesis classes of neural networks.
Some observations on high-dimensional partial differential equations with Barron data
We use explicit representation formulas to show that solutions to certain partial differential equations lie in Barron spaces or multilayer spaces if the PDE data lie in such function spaces.
On the emergence of simplex symmetry in the final and penultimate layers of neural network classifiers
A recent numerical study observed that neural network classifiers enjoy a large degree of symmetry in the penultimate layer.
Stochastic gradient descent with noise of machine learning type. Part I: Discrete time analysis
Stochastic gradient descent (SGD) is one of the most popular algorithms in modern machine learning.
Stochastic gradient descent with noise of machine learning type. Part II: Continuous time analysis
The representation of functions by artificial neural networks depends on a large number of parameters in a non-linear fashion.
Qualitative neural network approximation over R and C: Elementary proofs for analytic and polynomial activation
We prove for both real and complex networks with non-polynomial activation that the closure of the class of neural networks coincides with the closure of the space of polynomials.
Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionality
In this note, we study how neural networks with a single hidden layer and ReLU activation interpolate data drawn from a radially symmetric distribution with target labels 1 at the origin and 0 outside the unit ball, if no labels are known inside the unit ball.
Achieving acceleration despite very noisy gradients
We present a generalization of Nesterov's accelerated gradient descent algorithm.
A qualitative difference between gradient flows of convex functions in finite- and infinite-dimensional Hilbert spaces
In the case of stochastic gradient descent, the summability of $\mathbb E[f(x_n) - \inf f]$ is used to prove that $f(x_n)\to \inf f$ almost surely - an improvement on the convergence almost surely up to a subsequence which follows from the $O(1/n)$ decay estimate.
