Search Results for author:
Found 7 papers, 1 papers with code
Improved Quantization Strategies for Managing Heavy-tailed Gradients in Distributed Learning
We consider a general family of heavy-tail gradients that follow a power-law distribution, we aim to minimize the error resulting from quantization, thereby determining optimal values for two critical parameters: the truncation threshold and the quantization density.
Truncated Non-Uniform Quantization for Distributed SGD
To address the communication bottleneck challenge in distributed learning, our work introduces a novel two-stage quantization strategy designed to enhance the communication efficiency of distributed Stochastic Gradient Descent (SGD).
Killing Two Birds with One Stone: Quantization Achieves Privacy in Distributed Learning
Communication efficiency and privacy protection are two critical issues in distributed machine learning.
Adaptive Top-K in SGD for Communication-Efficient Distributed Learning
This paper proposes a novel adaptive Top-K in SGD framework that enables an adaptive degree of sparsification for each gradient descent step to optimize the convergence performance by balancing the trade-off between communication cost and convergence error.
DQ-SGD: Dynamic Quantization in SGD for Communication-Efficient Distributed Learning
Gradient quantization is an emerging technique in reducing communication costs in distributed learning.
DQSGD: DYNAMIC QUANTIZED STOCHASTIC GRADIENT DESCENT FOR COMMUNICATION-EFFICIENT DISTRIBUTED LEARNING
This paper addresses this issue by proposing a novel dynamic quantized SGD (DQSGD) framework, which enables us to optimize the quantization strategy for each gradient descent step by exploring the trade-off between communication cost and modeling error.
Unknown Intent Detection Using Gaussian Mixture Model with an Application to Zero-shot Intent Classification
User intent classification plays a vital role in dialogue systems.
