Search Results for author:
Found 5 papers, 0 papers with code
Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks
In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent.
(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability
In this paper, we investigate the impact of stochasticity and large stepsizes on the implicit regularisation of gradient descent (GD) and stochastic gradient descent (SGD) over diagonal linear networks.
Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity
Understanding the implicit bias of training algorithms is of crucial importance in order to explain the success of overparametrised neural networks.
On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent
Constant step-size Stochastic Gradient Descent exhibits two phases: a transient phase during which iterates make fast progress towards the optimum, followed by a stationary phase during which iterates oscillate around the optimal point.
Online Robust Regression via SGD on the l1 loss
We consider the robust linear regression problem in the online setting where we have access to the data in a streaming manner, one data point after the other.
