Search Results for author: Tianyi Chen

Found 85 papers, 38 papers with code

FORA: Fast-Forward Caching in Diffusion Transformer Acceleration

1 code implementation1 Jul 2024 Pratheba Selvaraju, Tianyu Ding, Tianyi Chen, Ilya Zharkov, Luming Liang

Diffusion transformers (DiT) have become the de facto choice for generating high-quality images and videos, largely due to their scalability, which enables the construction of larger models for enhanced performance.


Towards Exact Gradient-based Training on Analog In-memory Computing

no code implementations18 Jun 2024 Zhaoxian Wu, Tayfun Gokmen, Malte J. Rasch, Tianyi Chen

Given the high economic and environmental costs of using large vision or language models, analog in-memory accelerators present a promising solution for energy-efficient AI.

A Primal-Dual-Assisted Penalty Approach to Bilevel Optimization with Coupled Constraints

1 code implementation14 Jun 2024 Liuyuan Jiang, Quan Xiao, Victor M. Tenorio, Fernando Real-Rojas, Antonio Marques, Tianyi Chen

Interest in bilevel optimization has grown in recent years, partially due to its applications to tackle challenging machine-learning problems.

Bilevel Optimization

Exploring Key Factors for Long-Term Vessel Incident Risk Prediction

no code implementations30 May 2024 Tianyi Chen, Hua Wang, Yutong Cai, Maohan Liang, Qiang Meng

Most previous studies conduct factor analysis within the framework of incident-related label prediction, where the developed models can be categorized into short-term and long-term prediction models.

feature selection Management

Analytic Federated Learning

1 code implementation25 May 2024 Huiping Zhuang, Run He, Kai Tong, Di Fang, Han Sun, Haoran Li, Tianyi Chen, Ziqian Zeng

In this paper, we introduce analytic federated learning (AFL), a new training paradigm that brings analytical (i. e., closed-form) solutions to the federated learning (FL) community.

Federated Learning

Assessing Image Inpainting via Re-Inpainting Self-Consistency Evaluation

no code implementations25 May 2024 Tianyi Chen, Jianfu Zhang, Yan Hong, Yiyi Zhang, Liqing Zhang

Image inpainting, the task of reconstructing missing segments in corrupted images using available data, faces challenges in ensuring consistency and fidelity, especially under information-scarce conditions.

Image Inpainting

SF-DQN: Provable Knowledge Transfer using Successor Feature for Deep Reinforcement Learning

no code implementations24 May 2024 Shuai Zhang, Heshan Devaka Fernando, Miao Liu, Keerthiram Murugesan, Songtao Lu, Pin-Yu Chen, Tianyi Chen, Meng Wang

This paper studies the transfer reinforcement learning (RL) problem where multiple RL problems have different reward functions but share the same underlying transition dynamics.

Q-Learning Reinforcement Learning (RL) +1

ManiFoundation Model for General-Purpose Robotic Manipulation of Contact Synthesis with Arbitrary Objects and Robots

no code implementations11 May 2024 Zhixuan Xu, Chongkai Gao, Zixuan Liu, Gang Yang, Chenrui Tie, Haozhuo Zheng, Haoyu Zhou, Weikun Peng, Debang Wang, Tianyi Chen, Zhouliang Yu, Lin Shao

Our work introduces a comprehensive framework to develop a foundation model for general robotic manipulation that formalizes a manipulation task as contact synthesis.

Diversity Object

Efficient and Flexible Method for Reducing Moderate-size Deep Neural Networks with Condensation

no code implementations2 May 2024 Tianyi Chen, Zhi-Qin John Xu

In scientific applications, the scale of neural networks is generally moderate-size, mainly to ensure the speed of inference during application.

Image Classification

Optimistic Safety for Online Convex Optimization with Unknown Linear Constraints

no code implementations9 Mar 2024 Spencer Hutchinson, Tianyi Chen, Mahnoosh Alizadeh

We study the problem of online convex optimization (OCO) under unknown linear constraints that are either static, or stochastically time-varying.

Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHF

no code implementations10 Feb 2024 Han Shen, Zhuoran Yang, Tianyi Chen

But bilevel problems such as incentive design, inverse reinforcement learning (RL), and RL from human feedback (RLHF) are often modeled as dynamic objective functions that go beyond the simple static objective structures, which pose significant challenges of using existing bilevel solutions.

Bilevel Optimization reinforcement-learning +1

DistiLLM: Towards Streamlined Distillation for Large Language Models

3 code implementations6 Feb 2024 Jongwoo Ko, Sungnyun Kim, Tianyi Chen, Se-Young Yun

Knowledge distillation (KD) is widely used for compressing a teacher model to a smaller student model, reducing its inference cost and memory footprint while preserving model capabilities.

Instruction Following Knowledge Distillation

Enhancing In-context Learning via Linear Probe Calibration

1 code implementation22 Jan 2024 Momin Abbas, Yi Zhou, Parikshit Ram, Nathalie Baracaldo, Horst Samulowitz, Theodoros Salonidis, Tianyi Chen

However, applying ICL in real cases does not scale with the number of samples, and lacks robustness to different prompt templates and demonstration permutations.

In-Context Learning

Joint Unsupervised and Supervised Training for Automatic Speech Recognition via Bilevel Optimization

1 code implementation13 Jan 2024 A F M Saif, Xiaodong Cui, Han Shen, Songtao Lu, Brian Kingsbury, Tianyi Chen

In this paper, we present a novel bilevel optimization-based training approach to training acoustic models for automatic speech recognition (ASR) tasks that we term {bi-level joint unsupervised and supervised training (BL-JUST)}.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2

Uncertainty Quantification on Clinical Trial Outcome Prediction

1 code implementation7 Jan 2024 Tianyi Chen, Yingzhou Lu, Nan Hao, Capucine van Rechem, Jintai Chen, Tianfan Fu

Selective classification, encompassing a spectrum of methods for uncertainty quantification, empowers the model to withhold decision-making in the face of samples marked by ambiguity or low confidence, thereby amplifying the accuracy of predictions for the instances it chooses to classify.

Decision Making Drug Discovery +2

OTOv3: Automatic Architecture-Agnostic Neural Network Training and Compression from Structured Pruning to Erasing Operators

1 code implementation15 Dec 2023 Tianyi Chen, Tianyu Ding, Zhihui Zhu, Zeyu Chen, HsiangTao Wu, Ilya Zharkov, Luming Liang

Compressing a predefined deep neural network (DNN) into a compact sub-network with competitive performance is crucial in the efficient machine learning realm.

Neural Architecture Search

AttriHuman-3D: Editable 3D Human Avatar Generation with Attribute Decomposition and Indexing

no code implementations CVPR 2024 Fan Yang, Tianyi Chen, Xiaosheng He, Zhongang Cai, Lei Yang, Si Wu, Guosheng Lin

We propose AttriHuman-3D, an editable 3D human generation model, which address the aforementioned problems with attribute decomposition and indexing.

Attribute Disentanglement

The Efficiency Spectrum of Large Language Models: An Algorithmic Survey

1 code implementation1 Dec 2023 Tianyu Ding, Tianyi Chen, Haidong Zhu, Jiachen Jiang, Yiqi Zhong, Jinxin Zhou, Guangzhi Wang, Zhihui Zhu, Ilya Zharkov, Luming Liang

The rapid growth of Large Language Models (LLMs) has been a driving force in transforming various domains, reshaping the artificial general intelligence landscape.

Model Compression

DREAM: Diffusion Rectification and Estimation-Adaptive Models

1 code implementation CVPR 2024 Jinxin Zhou, Tianyu Ding, Tianyi Chen, Jiachen Jiang, Ilya Zharkov, Zhihui Zhu, Luming Liang

We present DREAM, a novel training framework representing Diffusion Rectification and Estimation Adaptive Models, requiring minimal code changes (just three lines) yet significantly enhancing the alignment of training with sampling in diffusion models.

Image Super-Resolution

CaesarNeRF: Calibrated Semantic Representation for Few-shot Generalizable Neural Rendering

1 code implementation27 Nov 2023 Haidong Zhu, Tianyu Ding, Tianyi Chen, Ilya Zharkov, Ram Nevatia, Luming Liang

Generalizability and few-shot learning are key challenges in Neural Radiance Fields (NeRF), often due to the lack of a holistic understanding in pixel-level rendering.

Few-Shot Learning Neural Rendering

LoRAShear: Efficient Large Language Model Structured Pruning and Knowledge Recovery

1 code implementation24 Oct 2023 Tianyi Chen, Tianyu Ding, Badal Yadav, Ilya Zharkov, Luming Liang

Large Language Models (LLMs) have transformed the landscape of artificial intelligence, while their enormous size presents significant challenges in terms of computational costs.

Language Modelling Large Language Model +1

MAT: Mixed-Strategy Game of Adversarial Training in Fine-tuning

no code implementations27 Jun 2023 Zhehua Zhong, Tianyi Chen, Zhen Wang

Fine-tuning large-scale pre-trained language models has been demonstrated effective for various natural language processing (NLP) tasks.

A Generalized Alternating Method for Bilevel Learning under the Polyak-Łojasiewicz Condition

no code implementations4 Jun 2023 Quan Xiao, Songtao Lu, Tianyi Chen

Bilevel optimization has recently regained interest owing to its applications in emerging machine learning fields such as hyperparameter optimization, meta-learning, and reinforcement learning.

Bilevel Optimization Hyperparameter Optimization +1

Automated Search-Space Generation Neural Architecture Search

1 code implementation25 May 2023 Tianyi Chen, Luming Liang, Tianyu Ding, Ilya Zharkov

To search an optimal sub-network within a general deep neural network (DNN), existing neural architecture search (NAS) methods typically rely on handcrafting a search space beforehand.

Neural Architecture Search

MoDA: Modeling Deformable 3D Objects from Casual Videos

1 code implementation17 Apr 2023 Chaoyue Song, Jiacheng Wei, Tianyi Chen, YiWen Chen, Chuan Sheng Foo, Fayao Liu, Guosheng Lin

To solve this problem, we propose neural dual quaternion blend skinning (NeuDBS) to achieve 3D point deformation, which can perform rigid transformation without skin-collapsing artifacts.

OTOV2: Automatic, Generic, User-Friendly

1 code implementation13 Mar 2023 Tianyi Chen, Luming Liang, Tianyu Ding, Zhihui Zhu, Ilya Zharkov

We propose the second generation of Only-Train-Once (OTOv2), which first automatically trains and compresses a general DNN only once from scratch to produce a more compact model with competitive performance without fine-tuning.

Model Compression

On Penalty-based Bilevel Gradient Descent Method

1 code implementation10 Feb 2023 Han Shen, Quan Xiao, Tianyi Chen

Bilevel optimization enjoys a wide range of applications in hyper-parameter optimization, meta-learning and reinforcement learning.

Bilevel Optimization Meta-Learning +2

Exploring Intra-Class Variation Factors With Learnable Cluster Prompts for Semi-Supervised Image Synthesis

no code implementations CVPR 2023 Yunfei Zhang, Xiaoyang Huo, Tianyi Chen, Si Wu, Hau San Wong

Semi-supervised class-conditional image synthesis is typically performed by inferring and injecting class labels into a conditional Generative Adversarial Network (GAN).

Conditional Image Generation Generative Adversarial Network

FSCNN: A Fast Sparse Convolution Neural Network Inference System

no code implementations17 Dec 2022 Bo Ji, Tianyi Chen

Convolution neural networks (CNNs) have achieved remarkable success, but typically accompany high computation cost and numerous redundant weight parameters.

Model Compression

Alternating Implicit Projected SGD and Its Efficient Variants for Equality-constrained Bilevel Optimization

1 code implementation14 Nov 2022 Quan Xiao, Han Shen, Wotao Yin, Tianyi Chen

By leveraging the special structure of the equality constraints problem, the paper first presents an alternating implicit projected SGD approach and establishes the $\tilde{\cal O}(\epsilon^{-2})$ sample complexity that matches the state-of-the-art complexity of ALSET \citep{chen2021closing} for unconstrained bilevel problems.

Bilevel Optimization

Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approach

1 code implementation23 Oct 2022 Heshan Fernando, Han Shen, Miao Liu, Subhajit Chaudhury, Keerthiram Murugesan, Tianyi Chen

Machine learning problems with multiple objective functions appear either in learning with multiple criteria where learning has to make a trade-off between multiple performance metrics such as fairness, safety and accuracy; or, in multi-task learning where multiple tasks are optimized jointly, sharing inductive bias between them.

Fairness Inductive Bias +1

On the Stability Analysis of Open Federated Learning Systems

no code implementations25 Sep 2022 Youbang Sun, Heshan Fernando, Tianyi Chen, Shahin Shahrampour

We consider the open federated learning (FL) systems, where clients may join and/or leave the system during the FL process.

Federated Learning

Sparsity-guided Network Design for Frame Interpolation

1 code implementation9 Sep 2022 Tianyu Ding, Luming Liang, Zhihui Zhu, Tianyi Chen, Ilya Zharkov

As a result, we achieve a considerable performance gain with a quarter of the size of the original AdaCoF.

COOR-PLT: A hierarchical control model for coordinating adaptive platoons of connected and autonomous vehicles at signal-free intersections based on deep reinforcement learning

no code implementations1 Jul 2022 Duowei Li, Jianping Wu, Feng Zhu, Tianyi Chen, Yiik Diew Wong

The simulation results demonstrate that the model is able to: (1) achieve satisfactory convergence performances; (2) adaptively determine platoon size in response to varying traffic conditions; and (3) completely avoid deadlocks at the intersection.

Autonomous Vehicles Fairness

Understanding Benign Overfitting in Gradient-Based Meta Learning

no code implementations27 Jun 2022 Lisha Chen, Songtao Lu, Tianyi Chen

While the conventional statistical learning theory suggests that overparameterized models tend to overfit, empirical evidence reveals that overparameterized meta learning methods still work well -- a phenomenon often called "benign overfitting."

Few-Shot Learning Learning Theory

Modeling Adaptive Platoon and Reservation Based Autonomous Intersection Control: A Deep Reinforcement Learning Approach

no code implementations24 Jun 2022 Duowei Li, Jianping Wu, Feng Zhu, Tianyi Chen, Yiik Diew Wong

As a strategy to reduce travel delay and enhance energy efficiency, platooning of connected and autonomous vehicles (CAVs) at non-signalized intersections has become increasingly popular in academia.

Autonomous Vehicles Reinforcement Learning (RL)

A Single-Timescale Analysis For Stochastic Approximation With Multiple Coupled Sequences

no code implementations21 Jun 2022 Han Shen, Tianyi Chen

Stochastic approximation (SA) with multiple coupled sequences has found broad applications in machine learning such as bilevel learning and reinforcement learning (RL).

Reinforcement Learning (RL)

Lazy Queries Can Reduce Variance in Zeroth-order Optimization

no code implementations14 Jun 2022 Quan Xiao, Qing Ling, Tianyi Chen

A major challenge of applying zeroth-order (ZO) methods is the high query complexity, especially when queries are costly.

Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning

1 code implementation8 Jun 2022 Momin Abbas, Quan Xiao, Lisha Chen, Pin-Yu Chen, Tianyi Chen

Model-agnostic meta learning (MAML) is currently one of the dominating approaches for few-shot meta-learning.


Federated Cross Learning for Medical Image Segmentation

1 code implementation5 Apr 2022 Xuanang Xu, Hannah H. Deng, Tianyi Chen, Tianshu Kuang, Joshua C. Barber, Daeseung Kim, Jaime Gateno, James J. Xia, Pingkun Yan

In this paper, we first conduct a theoretical analysis on the FL algorithm to reveal the problem of model aggregation during training on non-iid data.

Ensemble Learning Federated Learning +3

Is Bayesian Model-Agnostic Meta Learning Better than Model-Agnostic Meta Learning, Provably?

1 code implementation6 Mar 2022 Lisha Chen, Tianyi Chen

In this paper, we aim to provide theoretical justifications for Bayesian MAML's advantageous performance by comparing the meta test risks of MAML and Bayesian MAML.


Spatial-Temporal Sequential Hypergraph Network for Crime Prediction with Dynamic Multiplex Relation Learning

1 code implementation IJCAI 2021 Lianghao Xia, Chao Huang, Yong Xu, Peng Dai, Liefeng Bo, Xiyue Zhang, Tianyi Chen

Crime prediction is crucial for public safety and resource optimization, yet is very challenging due to two aspects: i) the dynamics of criminal patterns across time and space, crime events are distributed unevenly on both spatial and temporal domains; ii) time-evolving dependencies between different types of crimes (e. g., Theft, Robbery, Assault, Damage) which reveal fine-grained semantics of crimes.

Crime Prediction Relation

SphericGAN: Semi-Supervised Hyper-Spherical Generative Adversarial Networks for Fine-Grained Image Synthesis

no code implementations CVPR 2022 Tianyi Chen, Yunfei Zhang, Xiaoyang Huo, Si Wu, Yong Xu, Hau San Wong

To reduce the dependence of generative models on labeled data, we propose a semi-supervised hyper-spherical GAN for class-conditional fine-grained image generation, and our model is referred to as SphericGAN.

Generative Adversarial Network Image Generation

Closing the Gap: Tighter Analysis of Alternating Stochastic Gradient Methods for Bilevel Problems

no code implementations NeurIPS 2021 Tianyi Chen, Yuejiao Sun, Wotao Yin

By leveraging the hidden smoothness of the problem, this paper presents a tighter analysis of ALSET for stochastic nested problems.

Bilevel Optimization

CAFE: Catastrophic Data Leakage in Vertical Federated Learning

1 code implementation26 Oct 2021 Xiao Jin, Pin-Yu Chen, Chia-Yi Hsu, Chia-Mu Yu, Tianyi Chen

We name our proposed method as catastrophic data leakage in vertical federated learning (CAFE).

Vertical Federated Learning

Learning to Coordinate in Multi-Agent Systems: A Coordinated Actor-Critic Algorithm and Finite-Time Guarantees

no code implementations11 Oct 2021 Siliang Zeng, Tianyi Chen, Alfredo Garcia, Mingyi Hong

The flexibility in our design allows the proposed MARL-CAC algorithm to be used in a {\it fully decentralized} setting, where the agents can only communicate with their neighbors, as well as a {\it federated} setting, where the agents occasionally communicate with a server while optimizing their (partially personalized) local models.

Multi-agent Reinforcement Learning

Only Train Once: A One-Shot Neural Network Training And Pruning Framework

1 code implementation NeurIPS 2021 Tianyi Chen, Bo Ji, Tianyu Ding, Biyi Fang, Guanyi Wang, Zhihui Zhu, Luming Liang, Yixin Shi, Sheng Yi, Xiao Tu

Structured pruning is a commonly used technique in deploying deep neural networks (DNNs) onto resource-constrained devices.

Tighter Analysis of Alternating Stochastic Gradient Method for Stochastic Nested Problems

no code implementations25 Jun 2021 Tianyi Chen, Yuejiao Sun, Wotao Yin

By leveraging the hidden smoothness of the problem, this paper presents a tighter analysis of ALSET for stochastic nested problems.

Bilevel Optimization

Mask-Embedded Discriminator With Region-Based Semantic Regularization for Semi-Supervised Class-Conditional Image Synthesis

no code implementations CVPR 2021 Yi Liu, Xiaoyang Huo, Tianyi Chen, Xiangping Zeng, Si Wu, Zhiwen Yu, Hau-San Wong

Semi-supervised generative learning (SSGL) makes use of unlabeled data to achieve a trade-off between the data collection/annotation effort and generation performance, when adequate labeled data are not available.

Generative Adversarial Network Image Generation

Connected and Automated Vehicle Distributed Control for On-ramp Merging Scenario: A Virtual Rotation Approach

no code implementations28 Mar 2021 Tianyi Chen, Meng Wang, Siyuan Gong, Yang Zhou, Bin Ran

In this study, we propose a rotation-based connected automated vehicle (CAV) distributed cooperative control strategy for an on-ramp merging scenario.

A Single-Timescale Method for Stochastic Bilevel Optimization

no code implementations9 Feb 2021 Tianyi Chen, Yuejiao Sun, Quan Xiao, Wotao Yin

This paper develops a new optimization method for a class of stochastic bilevel problems that we term Single-Timescale stochAstic BiLevEl optimization (STABLE) method.

Bilevel Optimization Meta-Learning +1

Semi-Supervised Single-Stage Controllable GANs for Conditional Fine-Grained Image Generation

no code implementations ICCV 2021 Tianyi Chen, Yi Liu, Yunfei Zhang, Si Wu, Yong Xu, Feng Liangbing, Hau San Wong

To ensure disentanglement among the variables, we maximize mutual information between the class-independent variable and synthesized images, map real images to the latent space of a generator to perform consistency regularization of cross-class attributes, and incorporate class semantic-based regularization into a discriminator's feature space.

Disentanglement Image Generation

CAFE: Catastrophic Data Leakage in Federated Learning

no code implementations1 Jan 2021 Xiao Jin, Ruijie Du, Pin-Yu Chen, Tianyi Chen

In this paper, we revisit this defense premise and propose an advanced data leakage attack to efficiently recover batch data from the shared aggregated gradients.

Federated Learning

A Half-Space Stochastic Projected Gradient Method for Group Sparsity Regularization

no code implementations1 Jan 2021 Tianyi Chen, Guanyi Wang, Tianyu Ding, Bo Ji, Sheng Yi, Zhihui Zhu

Optimizing with group sparsity is significant in enhancing model interpretability in machining learning applications, e. g., feature selection, compressed sensing and model compression.

feature selection Model Compression +1

CADA: Communication-Adaptive Distributed Adam

1 code implementation31 Dec 2020 Tianyi Chen, Ziye Guo, Yuejiao Sun, Wotao Yin

This paper proposes an adaptive stochastic gradient descent method for distributed machine learning, which can be viewed as the communication-adaptive counterpart of the celebrated Adam method - justifying its name CADA.

BIG-bench Machine Learning

Towards Understanding Asynchronous Advantage Actor-critic: Convergence and Linear Speedup

no code implementations31 Dec 2020 Han Shen, Kaiqing Zhang, Mingyi Hong, Tianyi Chen

Asynchronous and parallel implementation of standard reinforcement learning (RL) algorithms is a key enabler of the tremendous success of modern RL.

Atari Games OpenAI Gym +1

Hybrid Federated Learning: Algorithms and Implementation

1 code implementation22 Dec 2020 Xinwei Zhang, Wotao Yin, Mingyi Hong, Tianyi Chen

To the best of our knowledge, this is the first formulation and algorithm developed for the hybrid FL.

Federated Learning

Neural Network Compression Via Sparse Optimization

no code implementations10 Nov 2020 Tianyi Chen, Bo Ji, Yixin Shi, Tianyu Ding, Biyi Fang, Sheng Yi, Xiao Tu

The compression of deep neural networks (DNNs) to reduce inference cost becomes increasingly important to meet realistic deployment requirements of various applications.

Neural Network Compression Stochastic Optimization

Byzantine-Robust Variance-Reduced Federated Learning over Distributed Non-i.i.d. Data

2 code implementations17 Sep 2020 Jie Peng, Zhaoxian Wu, Qing Ling, Tianyi Chen

We prove that the proposed method reaches a neighborhood of the optimal solution at a linear convergence rate and the learning error is determined by the number of Byzantine workers.

Federated Learning

Solving Stochastic Compositional Optimization is Nearly as Easy as Solving Stochastic Optimization

no code implementations25 Aug 2020 Tianyi Chen, Yuejiao Sun, Wotao Yin

In particular, we apply Adam to SCSC, and the exhibited rate of convergence matches that of the original Adam on non-compositional stochastic optimization.

Management Meta-Learning +1

Recurrent Exposure Generation for Low-Light Face Detection

1 code implementation21 Jul 2020 Jinxiu Liang, Jingwen Wang, Yuhui Quan, Tianyi Chen, Jiaying Liu, Haibin Ling, Yong Xu

REG produces progressively and efficiently intermediate images corresponding to various exposure settings, and such pseudo-exposures are then fused by MED to detect faces across different lighting conditions.

Face Detection Image Enhancement

VAFL: a Method of Vertical Asynchronous Federated Learning

no code implementations12 Jul 2020 Tianyi Chen, Xiao Jin, Yuejiao Sun, Wotao Yin

Horizontal Federated learning (FL) handles multi-client data that share the same set of features, and vertical FL trains a better predictor that combine all the features from different clients.

Federated Learning

Communication-Efficient Robust Federated Learning Over Heterogeneous Datasets

no code implementations17 Jun 2020 Yanjie Dong, Georgios B. Giannakis, Tianyi Chen, Julian Cheng, Md. Jahangir Hossain, Victor C. M. Leung

For strongly convex loss functions, FRPG and LFRPG have provably faster convergence rates than a benchmark robust stochastic aggregation algorithm.

Federated Learning

Orthant Based Proximal Stochastic Gradient Method for $\ell_1$-Regularized Optimization

1 code implementation7 Apr 2020 Tianyi Chen, Tianyu Ding, Bo Ji, Guanyi Wang, Jing Tian, Yixin Shi, Sheng Yi, Xiao Tu, Zhihui Zhu

Sparsity-inducing regularization problems are ubiquitous in machine learning applications, ranging from feature selection to model compression.

feature selection Model Compression

LASG: Lazily Aggregated Stochastic Gradients for Communication-Efficient Distributed Learning

1 code implementation26 Feb 2020 Tianyi Chen, Yuejiao Sun, Wotao Yin

The new algorithms adaptively choose between fresh and stale stochastic gradients and have convergence rates comparable to the original SGD.

Federated Learning

Adaptive Temporal Difference Learning with Linear Function Approximation

no code implementations20 Feb 2020 Tao Sun, Han Shen, Tianyi Chen, Dongsheng Li

Typically, the performance of TD(0) and TD($\lambda$) is very sensitive to the choice of stepsizes.

OpenAI Gym reinforcement-learning +1

Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attacks

no code implementations29 Dec 2019 Zhaoxian Wu, Qing Ling, Tianyi Chen, Georgios B. Giannakis

This motivates us to reduce the variance of stochastic gradients as a means of robustifying SGD in the presence of Byzantine attacks.

Communication-Efficient Distributed Learning via Lazily Aggregated Quantized Gradients

1 code implementation NeurIPS 2019 Jun Sun, Tianyi Chen, Georgios B. Giannakis, Zaiyue Yang

The present paper develops a novel aggregated gradient approach for distributed machine learning that adaptively compresses the gradient communication.

Communication-Censored Distributed Stochastic Gradient Descent

1 code implementation9 Sep 2019 Weiyu Li, Tianyi Chen, Liping Li, Zhaoxian Wu, Qing Ling

Specifically, in CSGD, the latest mini-batch stochastic gradient at a worker will be transmitted to the server if and only if it is sufficiently informative.

Quantization Stochastic Optimization

Generative Adversarial Network for Handwritten Text

1 code implementation27 Jul 2019 Bo Ji, Tianyi Chen

The main features of the new framework include: (i) A discriminator consists of an integrated CNN-Long-Short-Term- Memory (LSTM) based feature extraction with Path Signature Features (PSF) as input and a Feedforward Neural Network (FNN) based binary classifier; (ii) A recurrent latent variable model as generator for synthesizing sequential handwritten data.

Generative Adversarial Network

Communication-Efficient Policy Gradient Methods for Distributed Reinforcement Learning

no code implementations7 Dec 2018 Tianyi Chen, Kaiqing Zhang, Georgios B. Giannakis, Tamer Başar

This paper deals with distributed policy optimization in reinforcement learning, which involves a central controller and a group of learners.

Distributed Computing Multi-agent Reinforcement Learning +3

RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasets

1 code implementation9 Nov 2018 Liping Li, Wei Xu, Tianyi Chen, Georgios B. Giannakis, Qing Ling

In this paper, we propose a class of robust stochastic subgradient methods for distributed learning from heterogeneous datasets at presence of an unknown number of Byzantine workers.

Bandit Online Learning with Unknown Delays

no code implementations9 Jul 2018 Bingcong Li, Tianyi Chen, Georgios B. Giannakis

This paper deals with bandit online learning problems involving feedback of unknown delay that can emerge in multi-armed bandit (MAB) and bandit convex optimization (BCO) settings.

LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learning

1 code implementation NeurIPS 2018 Tianyi Chen, Georgios B. Giannakis, Tao Sun, Wotao Yin

This paper presents a new class of gradient methods for distributed machine learning that adaptively skip the gradient calculations to learn with reduced communication and computation.

Secure Mobile Edge Computing in IoT via Collaborative Online Learning

no code implementations9 May 2018 Bingcong Li, Tianyi Chen, Georgios B. Giannakis

To accommodate heterogeneous tasks in Internet of Things (IoT), a new communication and computing paradigm termed mobile edge computing emerges that extends computing services from the cloud to edge, but at the same time exposes new challenges on security.


Random Feature-based Online Multi-kernel Learning in Environments with Unknown Dynamics

no code implementations28 Dec 2017 Yanning Shen, Tianyi Chen, Georgios B. Giannakis

To further boost performance in dynamic environments, an adaptive multi-kernel learning scheme (termed AdaRaker) is developed.

Bandit Convex Optimization for Scalable and Dynamic IoT Management

no code implementations27 Jul 2017 Tianyi Chen, Georgios B. Giannakis

Tailored for such human-in-the-loop systems where the loss functions are hard to model, a family of bandit online saddle-point (BanSaP) schemes are developed, which adaptively adjust the online operations based on (possibly multiple) bandit feedback of the loss functions, and the changing environment.


An Online Convex Optimization Approach to Dynamic Network Resource Allocation

no code implementations14 Jan 2017 Tianyi Chen, Qing Ling, Georgios B. Giannakis

Performance of an online algorithm in this setting is assessed by: i) the difference of its losses relative to the best dynamic solution with one-slot-ahead information of the loss function and the constraint (that is here termed dynamic regret); and, ii) the accumulated amount of constraint violations (that is here termed dynamic fit).

Stochastic Averaging for Constrained Optimization with Application to Online Resource Allocation

no code implementations7 Oct 2016 Tianyi Chen, Aryan Mokhtari, Xin Wang, Alejandro Ribeiro, Georgios B. Giannakis

Existing approaches to resource allocation for nowadays stochastic networks are challenged to meet fast convergence and tolerable delay requirements.

Cannot find the paper you are looking for? You can Submit a new open access paper.