Search Results for author:
Found 6 papers, 3 papers with code
Is Heuristic Sampling Necessary in Training Deep Object Detectors?
In this paper, we challenge the necessity of such hard/soft sampling methods for training accurate deep object detectors.
DropIT: Dropping Intermediate Tensors for Memory-Efficient DNN Training
A standard hardware bottleneck when training deep neural networks is GPU memory.
Variance Reduced Local SGD with Lower Communication Complexity
To accelerate the training of machine learning models, distributed stochastic gradient descent (SGD) and its variants have been widely adopted, which apply multiple workers in parallel to speed up training.
Faster Distributed Deep Net Training: Computation and Communication Decoupled Stochastic Gradient Descent
Nevertheless, although distributed stochastic gradient descent (SGD) algorithms can achieve a linear iteration speedup, they are limited significantly in practice by the communication cost, making it difficult to achieve a linear time speedup.
STL-SGD: Speeding Up Local SGD with Stagewise Communication Period
Distributed parallel stochastic gradient descent algorithms are workhorses for large scale machine learning tasks.
Communication-Efficient Distributed Learning with Local Immediate Error Compensation
However, existing compression methods either perform only unidirectional compression in one iteration with higher communication cost, or bidirectional compression with slower convergence rate.
