Search Results for author:
Found 10 papers, 4 papers with code
Non-Uniform Smoothness for Gradient Descent
The analysis of gradient descent-type methods typically relies on the Lipschitz continuity of the objective gradient.
An adaptively inexact first-order method for bilevel optimization with application to hyperparameter learning
In this work, we propose an algorithm with backtracking line search that only relies on inexact function evaluations and hypergradients and show convergence to a stationary point.
Analyzing Inexact Hypergradients for Bilevel Learning
Estimating hyperparameters has been a long-standing problem in machine learning.
Optimizing illumination patterns for classical ghost imaging
Classical ghost imaging is a new paradigm in imaging where the image of an object is not measured directly with a pixelated detector.
A simplified convergence theory for Byzantine resilient stochastic gradient descent
In distributed learning, a central server trains a model according to updates provided by nodes holding local data samples.
Efficient Hyperparameter Tuning with Dynamic Accuracy Derivative-Free Optimization
Here, we apply a recent dynamic accuracy derivative-free optimization method to hyperparameter tuning, which allows inexact evaluations of the learning problem while retaining convergence guarantees.
Scalable Derivative-Free Optimization for Nonlinear Least-Squares Problems
Derivative-free - or zeroth-order - optimization (DFO) has gained recent attention for its ability to solve problems in a variety of application areas, including machine learning, particularly involving objectives which are stochastic and/or expensive to compute.
Inexact Derivative-Free Optimization for Bilevel Learning
A drawback of these techniques is that they are dependent on a number of parameters which have to be set by the user.
Escaping local minima with derivative-free methods: a numerical investigation
We apply a state-of-the-art, local derivative-free solver, Py-BOBYQA, to global optimization problems, and propose an algorithmic improvement that is beneficial in this context.
Optimization and Control
Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers
Numerical results show DFO-LS can gain reasonable progress on some medium-scale problems with fewer objective evaluations than is needed for one gradient evaluation.
Optimization and Control
