Search Results for author:
Found 27 papers, 6 papers with code
Symbolic Equation Solving via Reinforcement Learning
Machine-learning methods are gradually being adopted in a great variety of social, economic, and scientific contexts, yet they are notorious for struggling with exact mathematics.
Law of Balance and Stationary Distribution of Stochastic Gradient Descent
The stochastic gradient descent (SGD) algorithm is the algorithm we use to train neural networks.
The Probabilistic Stability of Stochastic Gradient Descent
Characterizing and understanding the stability of Stochastic Gradient Descent (SGD) remains an open problem in deep learning.
What shapes the loss landscape of self-supervised learning?
Prevention of complete and dimensional collapse of representations has recently become a design principle for self-supervised learning (SSL).
Three Learning Stages and Accuracy-Efficiency Tradeoff of Restricted Boltzmann Machines
Restricted Boltzmann Machines (RBMs) offer a versatile architecture for unsupervised machine learning that can in principle approximate any target probability distribution with arbitrary accuracy.
Exact Phase Transitions in Deep Learning
This work reports deep-learning-unique first-order and second-order phase transitions, whose phenomenology closely follows that in statistical physics.
Stochastic Neural Networks with Infinite Width are Deterministic
This work theoretically studies stochastic neural networks, a main type of neural network in use.
Interplay between depth of neural networks and locality of target functions
It has been recognized that heavily overparameterized deep neural networks (DNNs) exhibit surprisingly good generalization performance in various machine-learning tasks.
SGD Can Converge to Local Maxima
Stochastic gradient descent (SGD) is widely used for the nonlinear, nonconvex problem of training deep neural networks, but its behavior remains poorly understood.
Logarithmic landscape and power-law escape rate of SGD
Stochastic gradient descent (SGD) undergoes complicated multiplicative noise for the mean-square loss.
Convergent and Efficient Deep Q Learning Algorithm
Despite the empirical success of the deep Q network (DQN) reinforcement learning algorithm and its variants, DQN is still not well understood and it does not guarantee convergence.
SGD with a Constant Large Learning Rate Can Converge to Local Maxima
Previous works on stochastic gradient descent (SGD) often focus on its success.
Convergent and Efficient Deep Q Network Algorithm
Despite the empirical success of the deep Q network (DQN) reinforcement learning algorithm and its variants, DQN is still not well understood and it does not guarantee convergence.
Power-law escape rate of SGD
Stochastic gradient descent (SGD) undergoes complicated multiplicative noise for the mean-square loss.
Strength of Minibatch Noise in SGD
The noise in stochastic gradient descent (SGD), caused by minibatch sampling, is poorly understood despite its practical importance in deep learning.
Embedding the Yang-Lee Quantum Criticality in Open Quantum Systems
The Yang-Lee edge singularity is a quintessential nonunitary critical phenomenon accompanied by anomalous scaling laws.
Statistical Mechanics Quantum Physics
Noise and Fluctuation of Finite Learning Rate Stochastic Gradient Descent
In the vanishing learning rate regime, stochastic gradient descent (SGD) is now relatively well understood.
Improved generalization by noise enhancement
Recent studies have demonstrated that noise in stochastic gradient descent (SGD) is closely related to generalization: A larger SGD noise, if not too large, results in better generalization.
Intercomponent entanglement entropy and spectrum in binary Bose-Einstein condensates
We study the entanglement entropy and spectrum between components in binary Bose-Einstein condensates in $d$ spatial dimensions.
Quantum Gases Statistical Mechanics Quantum Physics
Neural Networks Fail to Learn Periodic Functions and How to Fix It
Previous literature offers limited clues on how to learn a periodic function using modern neural networks.
Is deeper better? It depends on locality of relevant features
It is shown that the NTK does not correctly capture the depth dependence of the generalization performance, which indicates the importance of the feature learning rather than the lazy learning.
Volumization as a Natural Generalization of Weight Decay
We propose a novel regularization method, called \textit{volumization}, for neural networks.
Learning Not to Learn in the Presence of Noisy Labels
Learning in the presence of label noise is a challenging yet important task: it is crucial to design models that are robust in the presence of mislabeled datasets.
LaProp: Separating Momentum and Adaptivity in Adam
We also bound the regret of Laprop on a convex problem and show that our bound differs from that of Adam by a key factor, which demonstrates its advantage.
Deep Reinforcement Learning Control of Quantum Cartpoles
We generalize a standard benchmark of reinforcement learning, the classical cartpole balancing problem, to the quantum regime by stabilizing a particle in an unstable potential through measurement and feedback.
A Simple Approach to the Noisy Label Problem Through the Gambler's Loss
Learning in the presence of label noise is a challenging yet important task.
Deep Gamblers: Learning to Abstain with Portfolio Theory
We deal with the \textit{selective classification} problem (supervised-learning problem with a rejection option), where we want to achieve the best performance at a certain level of coverage of the data.
