Stochastic Optimization

Polyak Averaging

Polyak Averaging is an optimization technique that sets final parameters to an average of (recent) parameters visited in the optimization trajectory. Specifically if in $t$ iterations we have parameters $\theta_{1}, \theta_{2}, \dots, \theta_{t}$, then Polyak Averaging suggests setting

$$ \theta_t =\frac{1}{t}\sum_{i}\theta_{i} $$

Image Credit: Shubhendu Trivedi & Risi Kondor


Paper Code Results Date Stars


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign