no code implementations • 19 May 2023 • Chuying Han, Yasong Feng, Tianyu Wang
We show that, when the environment is noise-free, the output of random search converges to the optimal value in probability at rate $ \widetilde{\mathcal{O}} \left( \left( \frac{1}{T} \right)^{ \frac{1}{d_s} } \right) $, where $ d_s \ge 0 $ is the scattering dimension of the underlying function.
no code implementations • 3 Feb 2023 • Yasong Feng, Weijian Luo, Yimin Huang, Tianyu Wang
We also apply BLiE to search for noise schedule of diffusion models.
no code implementations • 31 Oct 2022 • Tianyu Wang, Yasong Feng
We prove convergence rates of Stochastic Zeroth-order Gradient Descent (SZGD) algorithms for Lojasiewicz functions.
1 code implementation • 29 May 2022 • Yasong Feng, Tianyu Wang
In particular, we design estimators for smooth functions such that, if one uses $ \Theta \left( k \right) $ random directions sampled from the Stiefel's manifold $ \text{St} (n, k) $ and finite-difference granularity $\delta$, the variance of the gradient estimator is bounded by $ \mathcal{O} \left( \left( \frac{n}{k} - 1 \right) + \left( \frac{n^2}{k} - n \right) \delta^2 + \frac{ n^2 \delta^4 }{ k } \right) $, and the variance of the Hessian estimator is bounded by $\mathcal{O} \left( \left( \frac{n^2}{k^2} - 1 \right) + \left( \frac{n^4}{k^2} - n^2 \right) \delta^2 + \frac{n^4 \delta^4 }{k^2} \right) $.
no code implementations • 9 Dec 2021 • Yasong Feng, Weiguo Gao
Experiments of matrix completion and handwriting digit recognition demonstrate that the stability of our new method is better.
no code implementations • 19 Oct 2021 • Yasong Feng, Zengfeng Huang, Tianyu Wang
Specifically, we show that for a $T$-step problem with Lipschitz reward of zooming dimension $d_z$, our algorithm achieves theoretically optimal (up to logarithmic factors) regret rate $\widetilde{\mathcal{O}}\left(T^{\frac{d_z+1}{d_z+2}}\right)$ using only $ \mathcal{O} \left( \log\log T\right) $ batches.
no code implementations • 1 Jan 2021 • Yasong Feng, Weiguo Gao
To give more insight about the acceleration phenomenon, an ordinary differential equation was obtained from Nesterov's accelerated method by taking step sizes approaching zero, and the relationship between Nesterov's method and the differential equation is still of research interest.