Search Results for author: Zheng Xu

Found 36 papers, 14 papers with code

Analyzing the effect of neural network architecture on training performance

no code implementations ICML 2020 Karthik Abinav Sankararaman, Soham De, Zheng Xu, W. Ronny Huang, Tom Goldstein

Through novel theoretical and experimental results, we show how the neural net architecture affects gradient confusion, and thus the efficiency of training.

Beyond Uniform Lipschitz Condition in Differentially Private Optimization

no code implementations21 Jun 2022 Rudrajit Das, Satyen Kale, Zheng Xu, Tong Zhang, Sujay Sanghavi

Most prior convergence results on differentially private stochastic gradient descent (DP-SGD) are derived under the simplistic assumption of uniform Lipschitzness, i. e., the per-sample gradients are uniformly bounded.

On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Data

no code implementations9 Jun 2022 Jianyu Wang, Rudrajit Das, Gauri Joshi, Satyen Kale, Zheng Xu, Tong Zhang

Motivated by this observation, we propose a new quantity, average drift at optimum, to measure the effects of data heterogeneity, and explicitly use it to present a new theoretical analysis of FedAvg.

Federated Learning

Digging into Primary Financial Market: Challenges and Opportunities of Adopting Blockchain

no code implementations20 Apr 2022 Ji Liu, Zheng Xu, Yanmei Zhang, Wei Dai, Hao Wu, Shiping Chen

Since the emergence of blockchain technology, its application in the financial market has always been an area of focus and exploration by all parties.

Efficient and Private Federated Learning with Partially Trainable Networks

no code implementations6 Oct 2021 Hakim Sidahmed, Zheng Xu, Ankush Garg, Yuan Cao, Mingqing Chen

Through extensive experiments, we empirically show that Federated learning of Partially Trainable neural networks (FedPT) can result in superior communication-accuracy trade-offs, with up to $46\times$ reduction in communication cost, at a small accuracy cost.

Federated Learning

Local-Global Knowledge Distillation in Heterogeneous Federated Learning with Non-IID Data

no code implementations30 Jun 2021 Dezhong Yao, Wanning Pan, Yutong Dai, Yao Wan, Xiaofeng Ding, Hai Jin, Zheng Xu, Lichao Sun

Federated learning enables multiple clients to collaboratively learn a global model by periodically aggregating the clients' models without transferring the local data.

Federated Learning Knowledge Distillation

Local Adaptivity in Federated Learning: Convergence and Consistency

no code implementations4 Jun 2021 Jianyu Wang, Zheng Xu, Zachary Garrett, Zachary Charles, Luyang Liu, Gauri Joshi

Popular optimization algorithms of FL use vanilla (stochastic) gradient descent for both local updates at clients and global updates at the aggregating server.

Federated Learning

Towards Accurate Quantization and Pruning via Data-free Knowledge Transfer

no code implementations14 Oct 2020 Chen Zhu, Zheng Xu, Ali Shafahi, Manli Shu, Amin Ghiasi, Tom Goldstein

Further, we demonstrate that the compact structure and corresponding initialization from the Lottery Ticket Hypothesis can also help in data-free training.

Data Free Quantization Transfer Learning

Exploring Model Robustness with Adaptive Networks and Improved Adversarial Training

no code implementations30 May 2020 Zheng Xu, Ali Shafahi, Tom Goldstein

Our adaptive networks also outperform larger widened non-adaptive architectures that have 1. 5 times more parameters.

The Effect of Neural Net Architecture on Gradient Confusion & Training Performance

no code implementations25 Sep 2019 Karthik A. Sankararaman, Soham De, Zheng Xu, W. Ronny Huang, Tom Goldstein

Through novel theoretical and experimental results, we show how the neural net architecture affects gradient confusion, and thus the efficiency of training.

The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent

no code implementations15 Apr 2019 Karthik A. Sankararaman, Soham De, Zheng Xu, W. Ronny Huang, Tom Goldstein

Our results show that, for popular initialization techniques, increasing the width of neural networks leads to lower gradient confusion, and thus faster model training.

Universal Adversarial Training

no code implementations27 Nov 2018 Ali Shafahi, Mahyar Najibi, Zheng Xu, John Dickerson, Larry S. Davis, Tom Goldstein

Standard adversarial attacks change the predicted class label of a selected image by adding specially tailored small perturbations to its pixels.

Accurate and efficient video de-fencing using convolutional neural networks and temporal information

1 code implementation28 Jun 2018 Chen Du, Byeongkeun Kang, Zheng Xu, Ji Dai, Truong Nguyen

To overcome these problems, we propose a novel method consisting of segmentation using convolutional neural networks and a fast/robust recovery algorithm.

object-detection Object Detection +1

Learning from Multi-domain Artistic Images for Arbitrary Style Transfer

1 code implementation25 May 2018 Zheng Xu, Michael Wilber, Chen Fang, Aaron Hertzmann, Hailin Jin

We propose a fast feed-forward network for arbitrary style transfer, which can generate stylized image for previously unseen content and style image pairs.

Style Transfer

The Effectiveness of Instance Normalization: a Strong Baseline for Single Image Dehazing

no code implementations8 May 2018 Zheng Xu, Xitong Yang, Xue Li, Xiaoshuai Sun

We propose a novel deep neural network architecture for the challenging problem of single image dehazing, which aims to recover the clear image from a degraded hazy image.

Image Dehazing Single Image Dehazing

Learning to Cluster for Proposal-Free Instance Segmentation

1 code implementation17 Mar 2018 Yen-Chang Hsu, Zheng Xu, Zsolt Kira, Jiawei Huang

We utilize the most fundamental property of instance labeling -- the pairwise relationship between pixels -- as the supervision to formulate the learning objective, then apply it to train a fully convolutional network (FCN) for learning to perform pixel-wise clustering.

Autonomous Driving Instance Segmentation +4

Visualizing the Loss Landscape of Neural Nets

7 code implementations ICLR 2018 Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, Tom Goldstein

Neural network training relies on our ability to find "good" minimizers of highly non-convex loss functions.

Group-driven Reinforcement Learning for Personalized mHealth Intervention

1 code implementation14 Aug 2017 Feiyun Zhu, Jun Guo, Zheng Xu, Peng Liao, Junzhou Huang

Due to the popularity of smartphones and wearable devices nowadays, mobile health (mHealth) technologies are promising to bring positive and wide impacts on people's health.

Decision Making reinforcement-learning

Adaptive Consensus ADMM for Distributed Optimization

no code implementations ICML 2017 Zheng Xu, Gavin Taylor, Hao Li, Mario Figueiredo, Xiaoming Yuan, Tom Goldstein

The alternating direction method of multipliers (ADMM) is commonly used for distributed model fitting problems, but its performance and reliability depend strongly on user-defined penalty parameters.

Distributed Optimization

Training Quantized Nets: A Deeper Understanding

no code implementations NeurIPS 2017 Hao Li, Soham De, Zheng Xu, Christoph Studer, Hanan Samet, Tom Goldstein

Currently, deep neural networks are deployed on low-power portable devices by first training a full-precision model using powerful hardware, and then deriving a corresponding low-precision model for efficient inference on such systems.

Stabilizing Adversarial Nets With Prediction Methods

1 code implementation ICLR 2018 Abhay Yadav, Sohil Shah, Zheng Xu, David Jacobs, Tom Goldstein

Adversarial neural networks solve many important problems in data science, but are notoriously difficult to train.

Adaptive Relaxed ADMM: Convergence Theory and Practical Implementation

no code implementations CVPR 2017 Zheng Xu, Mario A. T. Figueiredo, Xiaoming Yuan, Christoph Studer, Tom Goldstein

Relaxed ADMM is a generalization of ADMM that often achieves better performance, but its efficiency depends strongly on algorithm parameters that must be chosen by an expert user.

Non-negative Factorization of the Occurrence Tensor from Financial Contracts

1 code implementation10 Dec 2016 Zheng Xu, Furong Huang, Louiqa Raschid, Tom Goldstein

We propose an algorithm for the non-negative factorization of an occurrence tensor built from heterogeneous networks.

An Empirical Study of ADMM for Nonconvex Problems

no code implementations10 Dec 2016 Zheng Xu, Soham De, Mario Figueiredo, Christoph Studer, Tom Goldstein

The alternating direction method of multipliers (ADMM) is a common optimization tool for solving constrained and non-differentiable problems.

Image Denoising

Adaptive ADMM with Spectral Penalty Parameter Selection

no code implementations24 May 2016 Zheng Xu, Mario A. T. Figueiredo, Tom Goldstein

The alternating direction method of multipliers (ADMM) is a versatile tool for solving a wide range of constrained optimization problems, with differentiable or non-differentiable objective functions.

Training Neural Networks Without Gradients: A Scalable ADMM Approach

2 code implementations6 May 2016 Gavin Taylor, Ryan Burmeister, Zheng Xu, Bharat Singh, Ankit Patel, Tom Goldstein

With the growing importance of large network models and enormous training datasets, GPUs have become increasingly necessary to train neural networks.

Exploiting Lists of Names for Named Entity Identification of Financial Institutions from Unstructured Documents

no code implementations14 Feb 2016 Zheng Xu, Douglas Burdick, Louiqa Raschid

To our knowledge, our proposed solutions, Dict-based NER and Rank-based ER, and the root and suffix dictionaries, are the first attempt to exploit specialized knowledge, i. e., lists of FI names, for rule-based NER and

Entity Resolution named-entity-recognition +1

Cannot find the paper you are looking for? You can Submit a new open access paper.