Search Results for author: Stephan Wojtowytsch

Found 13 papers, 0 papers with code

Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionality

no code implementations2 Sep 2022 Stephan Wojtowytsch

In this note, we study how neural networks with a single hidden layer and ReLU activation interpolate data drawn from a radially symmetric distribution with target labels 1 at the origin and 0 outside the unit ball, if no labels are known inside the unit ball.

Qualitative neural network approximation over R and C: Elementary proofs for analytic and polynomial activation

no code implementations25 Mar 2022 Josiah Park, Stephan Wojtowytsch

We prove for both real and complex networks with non-polynomial activation that the closure of the class of neural networks coincides with the closure of the space of polynomials.

Stochastic gradient descent with noise of machine learning type. Part II: Continuous time analysis

no code implementations4 Jun 2021 Stephan Wojtowytsch

The representation of functions by artificial neural networks depends on a large number of parameters in a non-linear fashion.

BIG-bench Machine Learning

Stochastic gradient descent with noise of machine learning type. Part I: Discrete time analysis

no code implementations4 May 2021 Stephan Wojtowytsch

Stochastic gradient descent (SGD) is one of the most popular algorithms in modern machine learning.

BIG-bench Machine Learning

On the emergence of simplex symmetry in the final and penultimate layers of neural network classifiers

no code implementations10 Dec 2020 Weinan E, Stephan Wojtowytsch

A recent numerical study observed that neural network classifiers enjoy a large degree of symmetry in the penultimate layer.

Some observations on high-dimensional partial differential equations with Barron data

no code implementations2 Dec 2020 Weinan E, Stephan Wojtowytsch

We use explicit representation formulas to show that solutions to certain partial differential equations lie in Barron spaces or multilayer spaces if the PDE data lie in such function spaces.

A priori estimates for classification problems using neural networks

no code implementations28 Sep 2020 Weinan E, Stephan Wojtowytsch

We consider binary and multi-class classification problems using hypothesis classes of neural networks.

Classification General Classification +1

Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't

no code implementations22 Sep 2020 Weinan E, Chao Ma, Stephan Wojtowytsch, Lei Wu

The purpose of this article is to review the achievements made in the last few years towards the understanding of the reasons behind the success and subtleties of neural network-based machine learning.

On the Banach spaces associated with multi-layer ReLU networks: Function representation, approximation theory and gradient descent dynamics

no code implementations30 Jul 2020 Weinan E, Stephan Wojtowytsch

The key to this work is a new way of representing functions in some form of expectations, motivated by multi-layer neural networks.

Representation formulas and pointwise properties for Barron functions

no code implementations10 Jun 2020 Weinan E, Stephan Wojtowytsch

We study the natural function space for infinitely wide two-layer neural networks with ReLU activation (Barron space) and establish different representation formulae.

On the Convergence of Gradient Descent Training for Two-layer ReLU-networks in the Mean Field Regime

no code implementations27 May 2020 Stephan Wojtowytsch

The condition does not depend on the initalization of parameters and concerns only the weak convergence of the realization of the neural network, not its parameter distribution.

Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernels

no code implementations21 May 2020 Weinan E, Stephan Wojtowytsch

We establish a scale separation of Kolmogorov width type between subspaces of a given Banach space under the condition that a sequence of linear maps converges much faster on one of the subspaces.

Can Shallow Neural Networks Beat the Curse of Dimensionality? A mean field training perspective

no code implementations21 May 2020 Stephan Wojtowytsch, Weinan E

Thus gradient descent training for fitting reasonably smooth, but truly high-dimensional data may be subject to the curse of dimensionality.

Cannot find the paper you are looking for? You can Submit a new open access paper.