Search Results for author: Jianyu Wang

Found 25 papers, 9 papers with code

On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Data

no code implementations9 Jun 2022 Jianyu Wang, Rudrajit Das, Gauri Joshi, Satyen Kale, Zheng Xu, Tong Zhang

Motivated by this observation, we propose a new quantity, average drift at optimum, to measure the effects of data heterogeneity, and explicitly use it to present a new theoretical analysis of FedAvg.

Federated Learning

Federated Learning under Distributed Concept Drift

no code implementations1 Jun 2022 Ellango Jothimurugesan, Kevin Hsieh, Jianyu Wang, Gauri Joshi, Phillip B. Gibbons

Empirical evaluation shows that our solutions achieve significantly higher accuracy than existing baselines, and are comparable to an idealized algorithm with oracle knowledge of the ground-truth clustering of clients to concepts at each time step.

Federated Learning

FedLite: A Scalable Approach for Federated Learning on Resource-constrained Clients

no code implementations28 Jan 2022 Jianyu Wang, Hang Qi, Ankit Singh Rawat, Sashank Reddi, Sagar Waghmare, Felix X. Yu, Gauri Joshi

In classical federated learning, the clients contribute to the overall training by communicating local updates for the underlying model on their private data to a coordinating server.

Federated Learning

Personalized Federated Learning for Heterogeneous Clients with Clustered Knowledge Transfer

no code implementations16 Sep 2021 Yae Jee Cho, Jianyu Wang, Tarun Chiruvolu, Gauri Joshi

Personalized federated learning (FL) aims to train model(s) that can perform well for individual clients that are highly data and system heterogeneous.

Personalized Federated Learning Transfer Learning

DualVGR: A Dual-Visual Graph Reasoning Unit for Video Question Answering

1 code implementation10 Jul 2021 Jianyu Wang, Bing-Kun Bao, Changsheng Xu

However, existing graph-based methods fail to perform multi-step reasoning well, neglecting two properties of VideoQA: (1) Even for the same video, different questions may require different amount of video clips or objects to infer the answer with relational reasoning; (2) During reasoning, appearance and motion features have complicated interdependence which are correlated and complementary to each other.

Graph Attention Question Answering +3

Local Adaptivity in Federated Learning: Convergence and Consistency

no code implementations4 Jun 2021 Jianyu Wang, Zheng Xu, Zachary Garrett, Zachary Charles, Luyang Liu, Gauri Joshi

Popular optimization algorithms of FL use vanilla (stochastic) gradient descent for both local updates at clients and global updates at the aggregating server.

Federated Learning

Deep NMF Topic Modeling

no code implementations24 Feb 2021 Jianyu Wang, Xiao-Lei Zhang

In this paper, we propose a deep NMF (DNMF) topic modeling framework to alleviate the aforementioned problems.

Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies

no code implementations3 Oct 2020 Yae Jee Cho, Jianyu Wang, Gauri Joshi

Federated learning is a distributed optimization paradigm that enables a large number of resource-limited client nodes to cooperatively train a model without data sharing.

Distributed Optimization Federated Learning +1

Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization

no code implementations NeurIPS 2020 Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, H. Vincent Poor

In federated optimization, heterogeneity in the clients' local datasets and computation speeds results in large variations in the number of local updates performed by each client in each communication round.

Slow and Stale Gradients Can Win the Race

no code implementations23 Mar 2020 Sanghamitra Dutta, Jianyu Wang, Gauri Joshi

Distributed Stochastic Gradient Descent (SGD) when run in a synchronous manner, suffers from delays in runtime as it waits for the slowest workers (stragglers).

Overlap Local-SGD: An Algorithmic Approach to Hide Communication Delays in Distributed SGD

1 code implementation21 Feb 2020 Jianyu Wang, Hao Liang, Gauri Joshi

In this paper, we propose an algorithmic approach named Overlap-Local-SGD (and its momentum variant) to overlap the communication and computation so as to speedup the distributed training procedure.

Deep topic modeling by multilayer bootstrap network and lasso

no code implementations24 Oct 2019 Jianyu Wang, Xiao-Lei Zhang

Specifically, we first apply multilayer bootstrap network (MBN), which is an unsupervised deep model, to reduce the dimension of documents, and then use the low-dimensional data representations or their clustering results as the target of supervised Lasso for topic word discovery.

Dimensionality Reduction Topic Models

MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Sampling

3 code implementations23 May 2019 Jianyu Wang, Anit Kumar Sahu, Zhouyi Yang, Gauri Joshi, Soummya Kar

This paper studies the problem of error-runtime trade-off, typically encountered in decentralized training based on stochastic gradient descent (SGD) using a given network.

Bilateral Adversarial Training: Towards Fast Training of More Robust Models Against Adversarial Attacks

1 code implementation ICCV 2019 Jianyu Wang, Haichao Zhang

To generate the adversarial image, we use one-step targeted attack with the target label being the most confusing class.

Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGD

no code implementations19 Oct 2018 Jianyu Wang, Gauri Joshi

Large-scale machine learning training, in particular distributed stochastic gradient descent, needs to be robust to inherent system variability such as node straggling and random communication delays.

Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms

no code implementations22 Aug 2018 Jianyu Wang, Gauri Joshi

Communication-efficient SGD algorithms, which allow nodes to perform local updates and periodically synchronize local models, are highly effective in improving the speed and scalability of distributed SGD.

Visual Concepts and Compositional Voting

no code implementations13 Nov 2017 Jianyu Wang, Zhishuai Zhang, Cihang Xie, Yuyin Zhou, Vittal Premachandran, Jun Zhu, Lingxi Xie, Alan Yuille

We use clustering algorithms to study the population activities of the features and extract a set of visual concepts which we show are visually tight and correspond to semantic parts of vehicles.

Semantic Part Detection

Detecting Semantic Parts on Partially Occluded Objects

no code implementations25 Jul 2017 Jianyu Wang, Cihang Xie, Zhishuai Zhang, Jun Zhu, Lingxi Xie, Alan Yuille

Our approach detects semantic parts by accumulating the confidence of local visual cues.

Semantic Part Detection

Unsupervised learning of object semantic parts from internal states of CNNs by population encoding

1 code implementation21 Nov 2015 Jianyu Wang, Zhishuai Zhang, Cihang Xie, Vittal Premachandran, Alan Yuille

We address the key question of how object part representations can be found from the internal states of CNNs that are trained for high-level tasks, such as object classification.

Keypoint Detection

Semantic Part Segmentation using Compositional Model combining Shape and Appearance

no code implementations CVPR 2015 Jianyu Wang, Alan Yuille

This is more challenging than standard object detection, object segmentation and pose estimation tasks because semantic parts of animals often have similar appearance and highly varying shapes.

object-detection Object Detection +2

Cannot find the paper you are looking for? You can Submit a new open access paper.