Search Results for author:
Found 31 papers, 11 papers with code
Mean Aggregator Is More Robust Than Robust Aggregators Under Label Poisoning Attacks
Robustness to malicious attacks is of paramount importance for distributed learning.
On the Tradeoff between Privacy Preservation and Byzantine-Robustness in Decentralized Learning
For a class of state-of-the-art robust aggregation rules, we give unified analysis of the "mixing abilities".
Byzantine-Robust Decentralized Stochastic Optimization with Stochastic Gradient Noise-Independent Learning Error
Motivated by this observation, we introduce two variance reduction methods, stochastic average gradient algorithm (SAGA) and loopless stochastic variance-reduced gradient (LSVRG), to Byzantine-robust decentralized stochastic optimization for eliminating the negative effect of the stochastic gradient noise.
Byzantine-Robust Distributed Online Learning: Taming Adversarial Participants in An Adversarial Environment
But we prove that, even with a class of state-of-the-art robust aggregation rules, in an adversarial environment and in the presence of Byzantine participants, distributed online gradient descent can only achieve a linear adversarial regret bound, which is tight.
Spectral Adversarial Training for Robust Graph Neural Network
In this work, we seek to address these challenges and propose Spectral Adversarial Training (SAT), a simple yet effective adversarial training approach for GNNs.
Lazy Queries Can Reduce Variance in Zeroth-order Optimization
A major challenge of applying zeroth-order (ZO) methods is the high query complexity, especially when queries are costly.
Confederated Learning: Federated Learning with Decentralized Edge Servers
Most studies on FL consider a centralized framework, in which a single server is endowed with a central authority to coordinate a number of devices to perform model training in an iterative manner.
Bridging Differential Privacy and Byzantine-Robustness via Model Aggregation
We analyze the trade-off between privacy preservation and learning performance, and show that the influence of our proposed DP mechanisms is decoupled with that of robust stochastic model aggregation.
Stochastic Alternating Direction Method of Multipliers for Byzantine-Robust Distributed Learning
This paper aims to solve a distributed learning problem under Byzantine attacks.
BROADCAST: Reducing Both Stochastic and Compression Noise to Robustify Communication-Efficient Federated Learning
Communication between workers and the master node to collect local stochastic gradients is a key bottleneck in a large-scale federated learning system.
Byzantine-Robust Variance-Reduced Federated Learning over Distributed Non-i.i.d. Data
We prove that the proposed method reaches a neighborhood of the optimal solution at a linear convergence rate and the learning error is determined by the number of Byzantine workers.
Byzantine-Robust Decentralized Stochastic Optimization over Static and Time-Varying Networks
In this paper, we consider the Byzantine-robust stochastic optimization problem defined over decentralized static and time-varying networks, where the agents collaboratively minimize the summation of expectations of stochastic local cost functions, but some of the agents are unreliable due to data corruptions, equipment failures or cyber-attacks.
Conditional Augmentation for Aspect Term Extraction via Masked Sequence-to-Sequence Generation
In this paper, we formulate the data augmentation as a conditional generation task: generating a new sentence while preserving the original opinion targets and labels.
HierTrain: Fast Hierarchical Edge AI Learning with Hybrid Parallelism in Mobile-Edge-Cloud Computing
We develop a novel \textit{hybrid parallelism} method, which is the key to HierTrain, to adaptively assign the DNN model layers and the data samples across the three levels of edge device, edge server and cloud center.
Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attacks
This motivates us to reduce the variance of stochastic gradients as a means of robustifying SGD in the presence of Byzantine attacks.
Convolutional Neural Networks for Space-Time Block Coding Recognition
We apply the latest advances in machine learning with deep neural networks to the tasks of radio modulation recognition, channel coding recognition, and spectrum monitoring.
Communication-Censored Linearized ADMM for Decentralized Consensus Optimization
COLA is proven to be convergent when the local cost functions have Lipschitz continuous gradients and the censoring threshold is summable.
Communication-Censored Distributed Stochastic Gradient Descent
Specifically, in CSGD, the latest mini-batch stochastic gradient at a worker will be transmitted to the server if and only if it is sufficiently informative.
Solving Non-smooth Constrained Programs with Lower Complexity than \mathcal{O}(1/\varepsilon): A Primal-Dual Homotopy Smoothing Approach
In this paper, we show that by leveraging a local error bound condition on the dual function, the proposed algorithm can achieve a better primal convergence time of $\mathcal{O}\l(\varepsilon^{-2/(2+\beta)}\log_2(\varepsilon^{-1})\r)$, where $\beta\in(0, 1]$ is a local error bound parameter.
Asynchronous Stochastic Composition Optimization with Variance Reduction
Composition optimization has drawn a lot of attention in a wide variety of machine learning domains from risk management to reinforcement learning.
RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasets
In this paper, we propose a class of robust stochastic subgradient methods for distributed learning from heterogeneous datasets at presence of an unknown number of Byzantine workers.
DADA: Deep Adversarial Data Augmentation for Extremely Low Data Regime Classification
Given insufficient data, while many techniques have been developed to help combat overfitting, the challenge remains if one tries to train deep networks, especially in the ill-posed extremely low data regimes: only a small set of labeled data are available, and nothing -- including unlabeled data -- else.
An Online Convex Optimization Approach to Dynamic Network Resource Allocation
Performance of an online algorithm in this setting is assessed by: i) the difference of its losses relative to the best dynamic solution with one-slot-ahead information of the loss function and the constraint (that is here termed dynamic regret); and, ii) the accumulated amount of constraint violations (that is here termed dynamic fit).
Stacked Approximated Regression Machine: A Simple Deep Learning Approach
With the agreement of my coauthors, I Zhangyang Wang would like to withdraw the manuscript "Stacked Approximated Regression Machine: A Simple Deep Learning Approach".
D3: Deep Dual-Domain Based Fast Restoration of JPEG-Compressed Images
In this paper, we design a Deep Dual-Domain (D3) based fast restoration model to remove artifacts of JPEG compressed images.
Make Workers Work Harder: Decoupled Asynchronous Proximal Stochastic Gradient Descent
Asynchronous parallel optimization algorithms for solving large-scale machine learning problems have drawn significant attention from academia to industry recently.
Learning A Deep $\ell_\infty$ Encoder for Hashing
We investigate the $\ell_\infty$-constrained representation which demonstrates robustness to quantization errors, utilizing the tool of deep learning.
$\mathbf{D^3}$: Deep Dual-Domain Based Fast Restoration of JPEG-Compressed Images
In this paper, we design a Deep Dual-Domain ($\mathbf{D^3}$) based fast restoration model to remove artifacts of JPEG compressed images.
Learning Deep $\ell_0$ Encoders
We study the $\ell_0$ sparse approximation problem with the tool of deep learning, by proposing Deep $\ell_0$ Encoders.
Decentralized learning for wireless communications and networking
This chapter deals with decentralized learning algorithms for in-network processing of graph-valued data.
EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization
In this paper, we develop a decentralized algorithm for the consensus optimization problem $$\min\limits_{x\in\mathbb{R}^p}~\bar{f}(x)=\frac{1}{n}\sum\limits_{i=1}^n f_i(x),$$ which is defined over a connected network of $n$ agents, where each function $f_i$ is held privately by agent $i$ and encodes the agent's data and objective.
Optimization and Control
