Stochastic Optimization
PowerSGD
Introduced by Vogels et al. in PowerSGD: Practical Low-Rank Gradient Compression for Distributed OptimizationPowerSGD is a distributed optimization technique that computes a low-rank approximation of the gradient using a generalized power iteration (known as subspace iteration). The approximation is computationally light-weight, avoiding any prohibitively expensive Singular Value Decomposition. To improve the quality of the efficient approximation, the authors warm-start the power iteration by reusing the approximation from the previous optimization step.
Source: PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization
Papers
| Paper | Code | Results | Date | Stars |
|---|
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
| 🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |
