Search Results for author: Shuai Zheng

Found 57 papers, 20 papers with code

Disentangling Orthogonal Planes for Indoor Panoramic Room Layout Estimation with Cross-Scale Distortion Awareness

1 code implementation CVPR 2023 Zhijie Shen, Zishuo Zheng, Chunyu Lin, Lang Nie, Kang Liao, Shuai Zheng, Yao Zhao

Based on the Manhattan World assumption, most existing indoor layout estimation schemes focus on recovering layouts from vertically compressed 1D sequences.

Room Layout Estimation

Decoupled Model Schedule for Deep Learning Training

no code implementations16 Feb 2023 Hongzheng Chen, Cody Hao Yu, Shuai Zheng, Zhen Zhang, Zhiru Zhang, Yida Wang

Specifically, the schedule works on a PyTorch model and uses a set of schedule primitives to convert the model for common model training optimizations such as high-performance kernels, effective 3D parallelism, and efficient activation checkpointing.


SPT: Semi-Parametric Prompt Tuning for Multitask Prompted Learning

no code implementations21 Dec 2022 M Saiful Bari, Aston Zhang, Shuai Zheng, Xingjian Shi, Yi Zhu, Shafiq Joty, Mu Li

Pre-trained large language models can efficiently interpolate human-written prompts in a natural way.

Language Modelling

SMILE: Scaling Mixture-of-Experts with Efficient Bi-level Routing

no code implementations10 Dec 2022 Chaoyang He, Shuai Zheng, Aston Zhang, George Karypis, Trishul Chilimbi, Mahdi Soltanolkotabi, Salman Avestimehr

The mixture of Expert (MoE) parallelism is a recent advancement that scales up the model size with constant computational cost.

Technical Report of Mixing Local Patterns

no code implementations7 Dec 2022 Shuai Zheng

Graph neural networks (GNNs) have shown remarkable performance on homophilic graph data while being far less impressive when handling non-homophilic graph data due to the inherent low-pass filtering property of GNNs.

Removing Batch Normalization Boosts Adversarial Training

1 code implementation4 Jul 2022 Haotao Wang, Aston Zhang, Shuai Zheng, Xingjian Shi, Mu Li, Zhangyang Wang

In addition, NoFrost achieves a $23. 56\%$ adversarial robustness against PGD attack, which improves the $13. 57\%$ robustness in BN-based AT.

Adversarial Robustness

Multi-modal Graph Learning for Disease Prediction

1 code implementation11 Mar 2022 Shuai Zheng, Zhenfeng Zhu, Zhizhe Liu, Zhenyu Guo, Yang Liu, Yuchen Yang, Yao Zhao

For disease prediction tasks, most existing graph-based methods tend to define the graph manually based on specified modality (e. g., demographic information), and then integrated other modalities to obtain the patient representation by Graph Representation Learning (GRL).

Disease Prediction Graph Learning +1

Contractive error feedback for gradient compression

no code implementations29 Sep 2021 Bingcong Li, Shuai Zheng, Parameswaran Raman, Anshumali Shrivastava, Georgios B. Giannakis

On-device memory concerns in distributed deep learning are becoming more severe due to i) the growth of model size in multi-GPU training, and ii) the adoption of neural networks for federated learning on IoT devices with limited storage.

Federated Learning Image Classification +3

Zero-shot Object Detection Through Vision-Language Embedding Alignment

1 code implementation24 Sep 2021 Johnathan Xie, Shuai Zheng

Recent approaches have shown that training deep neural networks directly on large-scale image-text pair collections enables zero-shot transfer on various recognition tasks.

object-detection Zero-Shot Object Detection

CETransformer: Casual Effect Estimation via Transformer Based Representation Learning

no code implementations19 Jul 2021 Zhenyu Guo, Shuai Zheng, Zhizhe Liu, Kun Yan, Zhenfeng Zhu

Treatment effect estimation, which refers to the estimation of causal effects and aims to measure the strength of the causal relationship, is of great importance in many fields but is a challenging problem in practice.

Representation Learning Selection bias

Compressed Communication for Distributed Training: Adaptive Methods and System

1 code implementation17 May 2021 Yuchen Zhong, Cong Xie, Shuai Zheng, Haibin Lin

Recently, there has been a growing interest in using gradient compression to reduce the communication overhead of the distributed training.

Deep Time Series Models for Scarce Data

no code implementations16 Mar 2021 Qiyao Wang, Ahmed Farahat, Chetan Gupta, Shuai Zheng

Time series data have grown at an explosive rate in numerous domains and have stimulated a surge of time series modeling research.

Model Selection Time Series Analysis

Margin Preserving Self-paced Contrastive Learning Towards Domain Adaptation for Medical Image Segmentation

1 code implementation15 Mar 2021 Zhizhe Liu, Zhenfeng Zhu, Shuai Zheng, Yang Liu, Jiayu Zhou, Yao Zhao

To bridge the gap between the source and target domains in unsupervised domain adaptation (UDA), the most common strategy puts focus on matching the marginal distributions in the feature space through adversarial learning.

Cardiac Segmentation Contrastive Learning +3

Graph Representation Learning via Diversity-preserving Graph Refinement

no code implementations12 Mar 2021 Shuai Zheng

For real-world graph data, the complex relationship between nodes is often represented as a hard binary link.

Disentanglement Graph Representation Learning

Taking Modality-free Human Identification as Zero-shot Learning

no code implementations2 Oct 2020 Zhizhe Liu, Xingxing Zhang, Zhenfeng Zhu, Shuai Zheng, Yao Zhao, Jian Cheng

There have been numerous methods proposed for human identification, such as face identification, person re-identification, and gait identification.

Event Detection Face Identification +3

CSER: Communication-efficient SGD with Error Reset

no code implementations NeurIPS 2020 Cong Xie, Shuai Zheng, Oluwasanmi Koyejo, Indranil Gupta, Mu Li, Haibin Lin

The scalability of Distributed Stochastic Gradient Descent (SGD) is today limited by communication bottlenecks.

Accelerated Large Batch Optimization of BERT Pretraining in 54 minutes

1 code implementation24 Jun 2020 Shuai Zheng, Haibin Lin, Sheng Zha, Mu Li

Using the proposed LANS method and the learning rate scheme, we scaled up the mini-batch sizes to 96K and 33K in phases 1 and 2 of BERT pretraining, respectively.

Natural Language Understanding

Distribution-induced Bidirectional Generative Adversarial Network for Graph Representation Learning

1 code implementation CVPR 2020 Shuai Zheng, Zhenfeng Zhu, Xingxing Zhang, Zhizhe Liu, Jian Cheng, Yao Zhao

Graph representation learning aims to encode all nodes of a graph into low-dimensional vectors that will serve as input of many compute vision tasks.

Graph Representation Learning

Convolutional Prototype Learning for Zero-Shot Recognition

no code implementations22 Oct 2019 Zhizhe Liu, Xingxing Zhang, Zhenfeng Zhu, Shuai Zheng, Yao Zhao, Jian Cheng

The key to ZSL is to transfer knowledge from the seen to the unseen classes via auxiliary class attribute vectors.

Image Captioning Object Recognition +2

Generative Adversarial Networks for Failure Prediction

no code implementations4 Oct 2019 Shuai Zheng, Ahmed Farahat, Chetan Gupta

GAN-FP first utilizes two GAN networks to simultaneously generate training samples and build an inference network that can be used to predict failures for new samples.

imbalanced classification Management

Edge Heuristic GAN for Non-uniform Blind Deblurring

no code implementations11 Jul 2019 Shuai Zheng, Zhenfeng Zhu, Jian Cheng, Yandong Guo, Yao Zhao

Non-uniform blur, mainly caused by camera shake and motions of multiple objects, is one of the most common causes of image quality degradation.


GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing

4 code implementations9 Jul 2019 Jian Guo, He He, Tong He, Leonard Lausen, Mu Li, Haibin Lin, Xingjian Shi, Chenguang Wang, Junyuan Xie, Sheng Zha, Aston Zhang, Hang Zhang, Zhi Zhang, Zhongyue Zhang, Shuai Zheng, Yi Zhu

We present GluonCV and GluonNLP, the deep learning toolkits for computer vision and natural language processing based on Apache MXNet (incubating).

Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback

1 code implementation NeurIPS 2019 Shuai Zheng, Ziyue Huang, James T. Kwok

In particular, on distributed ResNet training with 7 workers on the ImageNet, the proposed algorithm achieves the same testing accuracy as momentum SGD using full-precision gradients, but with $46\%$ less wall clock time.


Blockwise Adaptivity: Faster Training and Better Generalization in Deep Learning

no code implementations23 May 2019 Shuai Zheng, James T. Kwok

Stochastic methods with coordinate-wise adaptive stepsize (such as RMSprop and Adam) have been widely used in training deep neural networks.

Remaining Useful Life Estimation Using Functional Data Analysis

no code implementations12 Apr 2019 Qiyao Wang, Shuai Zheng, Ahmed Farahat, Susumu Serita, Chetan Gupta

In this work, we propose a novel Functional Data Analysis (FDA) method called functional Multilayer Perceptron (functional MLP) for RUL estimation.

Management Time Series Analysis

Give me a hint! Navigating Image Databases using Human-in-the-loop Feedback

no code implementations24 Sep 2018 Bryan A. Plummer, M. Hadi Kiapour, Shuai Zheng, Robinson Piramuthu

In this paper, we introduce an attribute-based interactive image search which can leverage human-in-the-loop feedback to iteratively refine image search results.

Image Retrieval

Lightweight Stochastic Optimization for Minimizing Finite Sums with Infinite Data

no code implementations ICML 2018 Shuai Zheng, James T. Kwok

The memory cost of SSAG does not depend on the sample size, while that of S-SAGA is the same as those of variance reduction methods on un- perturbed data.

Data Augmentation Stochastic Optimization

Minimal Support Vector Machine

no code implementations6 Apr 2018 Shuai Zheng, Chris Ding

Support Vector Machine (SVM) is an efficient classification approach, which finds a hyperplane to separate data from different classes.

Classification General Classification

Fast End-to-End Trainable Guided Filter

1 code implementation CVPR 2018 Huikai Wu, Shuai Zheng, Junge Zhang, Kaiqi Huang

To address the problem, we present a novel building block for FCNs, namely guided filtering layer, which is designed for efficiently generating a high-resolution output given the corresponding low-resolution one and a high-resolution guidance map.

Conditional Image-Text Embedding Networks

1 code implementation ECCV 2018 Bryan A. Plummer, Paige Kordas, M. Hadi Kiapour, Shuai Zheng, Robinson Piramuthu, Svetlana Lazebnik

This paper presents an approach for grounding phrases in images which jointly learns multiple text-conditioned embeddings in a single end-to-end model.

Phrase Grounding

GP-GAN: Towards Realistic High-Resolution Image Blending

2 code implementations21 Mar 2017 Huikai Wu, Shuai Zheng, Junge Zhang, Kaiqi Huang

Concretely, we propose Gaussian-Poisson Equation to formulate the high-resolution image blending problem, which is a joint optimization constrained by the gradient and color information.

Conditional Image Generation Vocal Bursts Intensity Prediction

A Projected Gradient Descent Method for CRF Inference allowing End-To-End Training of Arbitrary Pairwise Potentials

no code implementations24 Jan 2017 Måns Larsson, Anurag Arnab, Fredrik Kahl, Shuai Zheng, Philip Torr

It is empirically demonstrated that such learned potentials can improve segmentation accuracy and that certain label class interactions are indeed better modelled by a non-Gaussian potential.

Semantic Segmentation Structured Prediction

Kernel Alignment Inspired Linear Discriminant Analysis

no code implementations14 Oct 2016 Shuai Zheng, Chris Ding

The problem is to find a subspace to maximize the alignment between subspace-transformed data kernel and class indicator kernel.

A Harmonic Mean Linear Discriminant Analysis for Robust Image Classification

no code implementations14 Oct 2016 Shuai Zheng, Feiping Nie, Chris Ding, Heng Huang

In null space based LDA (NLDA), a well-known LDA extension, between-class distance is maximized in the null space of the within-class scatter matrix.

Classification General Classification +2

Fully-Trainable Deep Matching

1 code implementation12 Sep 2016 James Thewlis, Shuai Zheng, Philip H. S. Torr, Andrea Vedaldi

Deep Matching (DM) is a popular high-quality method for quasi-dense image matching.

Image Segmentation Semantic Segmentation

Stochastic Variance-Reduced ADMM

no code implementations24 Apr 2016 Shuai Zheng, James T. Kwok

The alternating direction method of multipliers (ADMM) is a powerful optimization solver in machine learning.

Fast Nonsmooth Regularized Risk Minimization with Continuation

no code implementations25 Feb 2016 Shuai Zheng, Ruiliang Zhang, James T. Kwok

In regularized risk minimization, the associated optimization problem becomes particularly difficult when both the loss and regularizer are nonsmooth.

Higher Order Conditional Random Fields in Deep Neural Networks

1 code implementation25 Nov 2015 Anurag Arnab, Sadeep Jayasumana, Shuai Zheng, Philip Torr

Recent deep learning approaches have incorporated CRFs into Convolutional Neural Networks (CNNs), with some even training the CRF end-to-end with the rest of the network.

Semantic Segmentation Superpixels

Joint Training of Generic CNN-CRF Models with Stochastic Optimization

no code implementations16 Nov 2015 Alexander Kirillov, Dmitrij Schlesinger, Shuai Zheng, Bogdan Savchynskyy, Philip H. S. Torr, Carsten Rother

We propose a new CNN-CRF end-to-end learning framework, which is based on joint stochastic optimization with respect to both Convolutional Neural Network (CNN) and Conditional Random Field (CRF) parameters.

Stochastic Optimization

Asynchronous Distributed Semi-Stochastic Gradient Optimization

no code implementations7 Aug 2015 Ruiliang Zhang, Shuai Zheng, James T. Kwok

With the recent proliferation of large-scale learning problems, there have been a lot of interest on distributed machine learning algorithms, particularly those that are based on stochastic gradient descent (SGD) and its variants.

Dense Semantic Image Segmentation with Objects and Attributes

no code implementations CVPR 2014 Shuai Zheng, Ming-Ming Cheng, Jonathan Warrell, Paul Sturgess, Vibhav Vineet, Carsten Rother, Philip H. S. Torr

The concepts of objects and attributes are both important for describing images precisely, since verbal descriptions often contain both adjectives and nouns (e. g. "I see a shiny red chair').

Image Segmentation Semantic Segmentation

ImageSpirit: Verbal Guided Image Parsing

no code implementations16 Oct 2013 Ming-Ming Cheng, Shuai Zheng, Wen-Yan Lin, Jonathan Warrell, Vibhav Vineet, Paul Sturgess, Nigel Crook, Niloy Mitra, Philip Torr

This allows us to formulate the image parsing problem as one of jointly estimating per-pixel object and attribute labels from a set of training images.

Cannot find the paper you are looking for? You can Submit a new open access paper.