Search Results for author:
Found 14 papers, 4 papers with code
Kernel Limit of Recurrent Neural Networks Trained on Ergodic Data Sequences
Mathematical methods are developed to characterize the asymptotics of recurrent neural networks (RNN) as the number of hidden units, data samples in the sequence, hidden state updates, and training steps simultaneously grow to infinity.
Transport map unadjusted Langevin algorithms: learning and discretizing perturbed samplers
We show that in continuous time, when a transport map is applied to Langevin dynamics, the result is a Riemannian manifold Langevin dynamics (RMLD) with metric defined by the transport map.
Normalization effects on deep neural networks
A given layer $i$ with $N_{i}$ hidden units is allowed to be normalized by $1/N_{i}^{\gamma_{i}}$ with $\gamma_{i}\in[1/2, 1]$ and we study the effect of the choice of the $\gamma_{i}$ on the statistical behavior of the neural network's output (such as variance) as well as on the test accuracy on the MNIST data set.
Geometry-informed irreversible perturbations for accelerated convergence of Langevin dynamics
We introduce a novel geometry-informed irreversible perturbation that accelerates convergence of the Langevin algorithm for Bayesian computation.
PDE-constrained Models with Neural Network Terms: Optimization and Global Convergence
Recent research has used deep learning to develop partial differential equation (PDE) models in science and engineering.
Normalization effects on shallow neural networks and related asymptotic expansions
In addition, we show that to leading order in $N$, the variance of the neural network's statistical output decays as the implied normalization by the scaling parameter approaches the mean field normalization.
Asymptotics of Reinforcement Learning with Neural Networks
In addition, we study the convergence of the limit differential equation to the stationary solution.
Scaling Limit of Neural Networks with the Xavier Initialization and Convergence to a Global Minimum
We analyze single-layer neural networks with the Xavier initialization in the asymptotic regime of large numbers of hidden units and large numbers of stochastic gradient descent training steps.
Mean Field Analysis of Deep Neural Networks
The limit procedure is valid for any number of hidden layers and it naturally also describes the limiting behavior of the training loss.
Information geometry for approximate Bayesian computation
We use relative entropy ideas to analyze the behavior of the algorithm as a function of the threshold parameter and of the size of the data.
Mean Field Analysis of Neural Networks: A Central Limit Theorem
We rigorously prove a central limit theorem for neural network models with a single hidden layer.
Stochastic Gradient Descent in Continuous Time: A Central Limit Theorem
Stochastic gradient descent in continuous time (SGDCT) provides a computationally efficient method for the statistical learning of continuous-time models, which are widely used in science, engineering, and finance.
DGM: A deep learning algorithm for solving partial differential equations
The algorithm is tested on a class of high-dimensional free boundary PDEs, which we are able to accurately solve in up to $200$ dimensions.
Stochastic Gradient Descent in Continuous Time
Stochastic gradient descent in continuous time (SGDCT) provides a computationally efficient method for the statistical learning of continuous-time models, which are widely used in science, engineering, and finance.
