Search Results for author:
Found 20 papers, 9 papers with code
E^2VTS: Energy-Efficient Video Text Spotting from Unmanned Aerial Vehicles
Unmanned Aerial Vehicles (UAVs) based video text spotting has been extensively used in civil and military domains.
Persia: An Open, Hybrid System Scaling Deep Learning-based Recommenders up to 100 Trillion Parameters
Specifically, in order to ensure both the training efficiency and the training accuracy, we design a novel hybrid training algorithm, where the embedding layer and the dense neural network are handled by different synchronization mechanisms; then we build a system called Persia (short for parallel recommendation training system with hybrid acceleration) to support this hybrid training algorithm.
BAGUA: Scaling up Distributed Learning with System Relaxations
Recent years have witnessed a growing list of systems for distributed data-parallel training.
DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learning
Games are abstractions of the real world, where artificial agents learn to compete and cooperate with other agents.
1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed
One of the most effective methods is error-compensated compression, which offers robust convergence speed even under 1-bit compression.
APMSqueeze: A Communication Efficient Adam-Preconditioned Momentum SGD Algorithm
Adam is the important optimization algorithm to guarantee efficiency and accuracy for training many important tasks such as BERT and ImageNet.
Stochastic Recursive Momentum for Policy Gradient Methods
In this paper, we propose a novel algorithm named STOchastic Recursive Momentum for Policy Gradient (STORM-PG), which operates a SARAH-type stochastic recursive variance-reduced policy gradient in an exponential moving average fashion.
Stochastic Recursive Variance Reduction for Efficient Smooth Non-Convex Compositional Optimization
Such a complexity is known to be the best one among IFO complexity results for non-convex stochastic compositional optimization, and is believed to be optimal.
Efficient Smooth Non-Convex Stochastic Compositional Optimization via Stochastic Recursive Gradient Descent
Stochastic compositional optimization arises in many important machine learning tasks such as reinforcement learning and portfolio management.
$\texttt{DeepSqueeze}$: Decentralization Meets Error-Compensated Compression
Since the \emph{decentralized} training has been witnessed to be superior to the traditional \emph{centralized} training in the communication restricted scenario, therefore a natural question to ask is "how to apply the error-compensated technology to the decentralized learning to further reduce the communication cost."
DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compression
For example, under the popular parameter server model for distributed learning, the worker nodes need to send the compressed local gradients to the parameter server, which performs the aggregation.
Revisit Batch Normalization: New Understanding from an Optimization View and a Refinement via Composition Optimization
We show when BN works and when BN does not work by analyzing the optimization problem.
$D^2$: Decentralized Training over Decentralized Data
While training a machine learning model using multiple workers, each of which collects data from its own data source, it would be useful when the data collected from different workers are unique and different.
Ranked #3 on
Multi-view Subspace Clustering
on ORL
D$^2$: Decentralized Training over Decentralized Data
While training a machine learning model using multiple workers, each of which collects data from their own data sources, it would be most useful when the data collected from different workers can be {\em unique} and {\em different}.
PARAMETRIZED DEEP Q-NETWORKS LEARNING: PLAYING ONLINE BATTLE ARENA WITH DISCRETE-CONTINUOUS HYBRID ACTION SPACE
Most existing deep reinforcement learning (DRL) frameworks consider action spaces that are either discrete or continuous space.
Asynchronous Decentralized Parallel Stochastic Gradient Descent
Can we design an algorithm that is robust in a heterogeneous environment, while being communication efficient and maintaining the best-possible convergence rate?
Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent
On network configurations with low bandwidth or high latency, D-PSGD can be up to one order of magnitude faster than its well-optimized centralized counterparts.
Asynchronous Parallel Greedy Coordinate Descent
n this paper, we propose and study an Asynchronous parallel Greedy Coordinate Descent (Asy-GCD) algorithm for minimizing a smooth function with bounded constraints.
Staleness-aware Async-SGD for Distributed Deep Learning
Deep neural networks have been shown to achieve state-of-the-art performance in several machine learning tasks.
Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization
Asynchronous parallel implementations of stochastic gradient (SG) have been broadly used in solving deep neural network and received many successes in practice recently.
