## Optimal rates for zero-order convex optimization: the power of two function evaluations

7 Dec 2013  ·  John C. Duchi, Michael I. Jordan, Martin J. Wainwright, Andre Wibisono ·

We consider derivative-free algorithms for stochastic and non-stochastic convex optimization problems that use only function values rather than gradients. Focusing on non-asymptotic bounds on convergence rates, we show that if pairs of function values are available, algorithms for $d$-dimensional optimization that use gradient estimates based on random perturbations suffer a factor of at most $\sqrt{d}$ in convergence rate over traditional stochastic gradient methods... We establish such results for both smooth and non-smooth cases, sharpening previous analyses that suggested a worse dimension dependence, and extend our results to the case of multiple ($m \ge 2$) evaluations. We complement our algorithmic development with information-theoretic lower bounds on the minimax convergence rate of such problems, establishing the sharpness of our achievable results up to constant (sometimes logarithmic) factors. read more

PDF Abstract

# Code Add Remove Mark official

No code implementations yet. Submit your code now

# Datasets

Add Datasets introduced or used in this paper

# Results from the Paper Add Remove

Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.