Search Results for author: Shuai Zheng

Found 58 papers, 21 papers with code

ImageSpirit: Verbal Guided Image Parsing

no code implementations16 Oct 2013 Ming-Ming Cheng, Shuai Zheng, Wen-Yan Lin, Jonathan Warrell, Vibhav Vineet, Paul Sturgess, Nigel Crook, Niloy Mitra, Philip Torr

This allows us to formulate the image parsing problem as one of jointly estimating per-pixel object and attribute labels from a set of training images.

Attribute Object

Dense Semantic Image Segmentation with Objects and Attributes

no code implementations CVPR 2014 Shuai Zheng, Ming-Ming Cheng, Jonathan Warrell, Paul Sturgess, Vibhav Vineet, Carsten Rother, Philip H. S. Torr

The concepts of objects and attributes are both important for describing images precisely, since verbal descriptions often contain both adjectives and nouns (e. g. "I see a shiny red chair').

Attribute Image Segmentation +2

Asynchronous Distributed Semi-Stochastic Gradient Optimization

no code implementations7 Aug 2015 Ruiliang Zhang, Shuai Zheng, James T. Kwok

With the recent proliferation of large-scale learning problems, there have been a lot of interest on distributed machine learning algorithms, particularly those that are based on stochastic gradient descent (SGD) and its variants.

Cloud Computing

Joint Training of Generic CNN-CRF Models with Stochastic Optimization

no code implementations16 Nov 2015 Alexander Kirillov, Dmitrij Schlesinger, Shuai Zheng, Bogdan Savchynskyy, Philip H. S. Torr, Carsten Rother

We propose a new CNN-CRF end-to-end learning framework, which is based on joint stochastic optimization with respect to both Convolutional Neural Network (CNN) and Conditional Random Field (CRF) parameters.

Stochastic Optimization

Higher Order Conditional Random Fields in Deep Neural Networks

1 code implementation25 Nov 2015 Anurag Arnab, Sadeep Jayasumana, Shuai Zheng, Philip Torr

Recent deep learning approaches have incorporated CRFs into Convolutional Neural Networks (CNNs), with some even training the CRF end-to-end with the rest of the network.

Segmentation Semantic Segmentation +1

Fast Nonsmooth Regularized Risk Minimization with Continuation

no code implementations25 Feb 2016 Shuai Zheng, Ruiliang Zhang, James T. Kwok

In regularized risk minimization, the associated optimization problem becomes particularly difficult when both the loss and regularizer are nonsmooth.

Stochastic Variance-Reduced ADMM

no code implementations24 Apr 2016 Shuai Zheng, James T. Kwok

The alternating direction method of multipliers (ADMM) is a powerful optimization solver in machine learning.

Fully-Trainable Deep Matching

1 code implementation12 Sep 2016 James Thewlis, Shuai Zheng, Philip H. S. Torr, Andrea Vedaldi

Deep Matching (DM) is a popular high-quality method for quasi-dense image matching.

Image Segmentation Semantic Segmentation

A Harmonic Mean Linear Discriminant Analysis for Robust Image Classification

no code implementations14 Oct 2016 Shuai Zheng, Feiping Nie, Chris Ding, Heng Huang

In null space based LDA (NLDA), a well-known LDA extension, between-class distance is maximized in the null space of the within-class scatter matrix.

Classification General Classification +2

Kernel Alignment Inspired Linear Discriminant Analysis

no code implementations14 Oct 2016 Shuai Zheng, Chris Ding

The problem is to find a subspace to maximize the alignment between subspace-transformed data kernel and class indicator kernel.

A Projected Gradient Descent Method for CRF Inference allowing End-To-End Training of Arbitrary Pairwise Potentials

no code implementations24 Jan 2017 Måns Larsson, Anurag Arnab, Fredrik Kahl, Shuai Zheng, Philip Torr

It is empirically demonstrated that such learned potentials can improve segmentation accuracy and that certain label class interactions are indeed better modelled by a non-Gaussian potential.

Segmentation Semantic Segmentation +1

GP-GAN: Towards Realistic High-Resolution Image Blending

2 code implementations21 Mar 2017 Huikai Wu, Shuai Zheng, Junge Zhang, Kaiqi Huang

Concretely, we propose Gaussian-Poisson Equation to formulate the high-resolution image blending problem, which is a joint optimization constrained by the gradient and color information.

Conditional Image Generation Generative Adversarial Network +1

Conditional Image-Text Embedding Networks

1 code implementation ECCV 2018 Bryan A. Plummer, Paige Kordas, M. Hadi Kiapour, Shuai Zheng, Robinson Piramuthu, Svetlana Lazebnik

This paper presents an approach for grounding phrases in images which jointly learns multiple text-conditioned embeddings in a single end-to-end model.

Phrase Grounding

Fast End-to-End Trainable Guided Filter

1 code implementation CVPR 2018 Huikai Wu, Shuai Zheng, Junge Zhang, Kaiqi Huang

To address the problem, we present a novel building block for FCNs, namely guided filtering layer, which is designed for efficiently generating a high-resolution output given the corresponding low-resolution one and a high-resolution guidance map.

Minimal Support Vector Machine

no code implementations6 Apr 2018 Shuai Zheng, Chris Ding

Support Vector Machine (SVM) is an efficient classification approach, which finds a hyperplane to separate data from different classes.

Classification General Classification

Lightweight Stochastic Optimization for Minimizing Finite Sums with Infinite Data

no code implementations ICML 2018 Shuai Zheng, James T. Kwok

The memory cost of SSAG does not depend on the sample size, while that of S-SAGA is the same as those of variance reduction methods on un- perturbed data.

Data Augmentation Stochastic Optimization

Give me a hint! Navigating Image Databases using Human-in-the-loop Feedback

no code implementations24 Sep 2018 Bryan A. Plummer, M. Hadi Kiapour, Shuai Zheng, Robinson Piramuthu

In this paper, we introduce an attribute-based interactive image search which can leverage human-in-the-loop feedback to iteratively refine image search results.

Attribute Image Retrieval

Remaining Useful Life Estimation Using Functional Data Analysis

no code implementations12 Apr 2019 Qiyao Wang, Shuai Zheng, Ahmed Farahat, Susumu Serita, Chetan Gupta

In this work, we propose a novel Functional Data Analysis (FDA) method called functional Multilayer Perceptron (functional MLP) for RUL estimation.

Management Time Series +1

Blockwise Adaptivity: Faster Training and Better Generalization in Deep Learning

no code implementations23 May 2019 Shuai Zheng, James T. Kwok

Stochastic methods with coordinate-wise adaptive stepsize (such as RMSprop and Adam) have been widely used in training deep neural networks.

Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback

1 code implementation NeurIPS 2019 Shuai Zheng, Ziyue Huang, James T. Kwok

In particular, on distributed ResNet training with 7 workers on the ImageNet, the proposed algorithm achieves the same testing accuracy as momentum SGD using full-precision gradients, but with $46\%$ less wall clock time.

Quantization

GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing

4 code implementations9 Jul 2019 Jian Guo, He He, Tong He, Leonard Lausen, Mu Li, Haibin Lin, Xingjian Shi, Chenguang Wang, Junyuan Xie, Sheng Zha, Aston Zhang, Hang Zhang, Zhi Zhang, Zhongyue Zhang, Shuai Zheng, Yi Zhu

We present GluonCV and GluonNLP, the deep learning toolkits for computer vision and natural language processing based on Apache MXNet (incubating).

Edge Heuristic GAN for Non-uniform Blind Deblurring

no code implementations11 Jul 2019 Shuai Zheng, Zhenfeng Zhu, Jian Cheng, Yandong Guo, Yao Zhao

Non-uniform blur, mainly caused by camera shake and motions of multiple objects, is one of the most common causes of image quality degradation.

Deblurring Generative Adversarial Network

Generative Adversarial Networks for Failure Prediction

no code implementations4 Oct 2019 Shuai Zheng, Ahmed Farahat, Chetan Gupta

GAN-FP first utilizes two GAN networks to simultaneously generate training samples and build an inference network that can be used to predict failures for new samples.

imbalanced classification Management

Convolutional Prototype Learning for Zero-Shot Recognition

no code implementations22 Oct 2019 Zhizhe Liu, Xingxing Zhang, Zhenfeng Zhu, Shuai Zheng, Yao Zhao, Jian Cheng

The key to ZSL is to transfer knowledge from the seen to the unseen classes via auxiliary class attribute vectors.

Attribute Image Captioning +3

Accelerated Large Batch Optimization of BERT Pretraining in 54 minutes

1 code implementation24 Jun 2020 Shuai Zheng, Haibin Lin, Sheng Zha, Mu Li

Using the proposed LANS method and the learning rate scheme, we scaled up the mini-batch sizes to 96K and 33K in phases 1 and 2 of BERT pretraining, respectively.

Natural Language Understanding

CSER: Communication-efficient SGD with Error Reset

no code implementations NeurIPS 2020 Cong Xie, Shuai Zheng, Oluwasanmi Koyejo, Indranil Gupta, Mu Li, Haibin Lin

The scalability of Distributed Stochastic Gradient Descent (SGD) is today limited by communication bottlenecks.

Taking Modality-free Human Identification as Zero-shot Learning

no code implementations2 Oct 2020 Zhizhe Liu, Xingxing Zhang, Zhenfeng Zhu, Shuai Zheng, Yao Zhao, Jian Cheng

There have been numerous methods proposed for human identification, such as face identification, person re-identification, and gait identification.

Attribute Event Detection +4

Adversarial Graph Disentanglement

1 code implementation12 Mar 2021 Shuai Zheng, Zhenfeng Zhu, Zhizhe Liu, Jian Cheng, Yao Zhao

For them, a component-specific aggregation approach is proposed to achieve micro-disentanglement by inferring latent components that cause the links between nodes.

Disentanglement Graph Representation Learning

Margin Preserving Self-paced Contrastive Learning Towards Domain Adaptation for Medical Image Segmentation

1 code implementation15 Mar 2021 Zhizhe Liu, Zhenfeng Zhu, Shuai Zheng, Yang Liu, Jiayu Zhou, Yao Zhao

To bridge the gap between the source and target domains in unsupervised domain adaptation (UDA), the most common strategy puts focus on matching the marginal distributions in the feature space through adversarial learning.

Cardiac Segmentation Contrastive Learning +4

Deep Time Series Models for Scarce Data

no code implementations16 Mar 2021 Qiyao Wang, Ahmed Farahat, Chetan Gupta, Shuai Zheng

Time series data have grown at an explosive rate in numerous domains and have stimulated a surge of time series modeling research.

Model Selection Time Series +1

Compressed Communication for Distributed Training: Adaptive Methods and System

1 code implementation17 May 2021 Yuchen Zhong, Cong Xie, Shuai Zheng, Haibin Lin

Recently, there has been a growing interest in using gradient compression to reduce the communication overhead of the distributed training.

CETransformer: Casual Effect Estimation via Transformer Based Representation Learning

no code implementations19 Jul 2021 Zhenyu Guo, Shuai Zheng, Zhizhe Liu, Kun Yan, Zhenfeng Zhu

Treatment effect estimation, which refers to the estimation of causal effects and aims to measure the strength of the causal relationship, is of great importance in many fields but is a challenging problem in practice.

counterfactual Representation Learning +1

Zero-shot Object Detection Through Vision-Language Embedding Alignment

1 code implementation24 Sep 2021 Johnathan Xie, Shuai Zheng

Recent approaches have shown that training deep neural networks directly on large-scale image-text pair collections enables zero-shot transfer on various recognition tasks.

Object object-detection +1

Multi-modal Graph Learning for Disease Prediction

1 code implementation11 Mar 2022 Shuai Zheng, Zhenfeng Zhu, Zhizhe Liu, Zhenyu Guo, Yang Liu, Yuchen Yang, Yao Zhao

For disease prediction tasks, most existing graph-based methods tend to define the graph manually based on specified modality (e. g., demographic information), and then integrated other modalities to obtain the patient representation by Graph Representation Learning (GRL).

Disease Prediction Graph Learning +1

Removing Batch Normalization Boosts Adversarial Training

1 code implementation4 Jul 2022 Haotao Wang, Aston Zhang, Shuai Zheng, Xingjian Shi, Mu Li, Zhangyang Wang

In addition, NoFrost achieves a $23. 56\%$ adversarial robustness against PGD attack, which improves the $13. 57\%$ robustness in BN-based AT.

Adversarial Robustness

Node-oriented Spectral Filtering for Graph Neural Networks

no code implementations7 Dec 2022 Shuai Zheng, Zhenfeng Zhu, Zhizhe Liu, Youru Li, Yao Zhao

Graph neural networks (GNNs) have shown remarkable performance on homophilic graph data while being far less impressive when handling non-homophilic graph data due to the inherent low-pass filtering property of GNNs.

SMILE: Scaling Mixture-of-Experts with Efficient Bi-level Routing

no code implementations10 Dec 2022 Chaoyang He, Shuai Zheng, Aston Zhang, George Karypis, Trishul Chilimbi, Mahdi Soltanolkotabi, Salman Avestimehr

The mixture of Expert (MoE) parallelism is a recent advancement that scales up the model size with constant computational cost.

Slapo: A Schedule Language for Progressive Optimization of Large Deep Learning Model Training

no code implementations16 Feb 2023 Hongzheng Chen, Cody Hao Yu, Shuai Zheng, Zhen Zhang, Zhiru Zhang, Yida Wang

Specifically, Slapo works on a PyTorch model and uses a set of schedule primitives to convert the model for common model training optimizations such as high-performance kernels, effective 3D parallelism, and efficient activation checkpointing.

Scheduling

Disentangling Orthogonal Planes for Indoor Panoramic Room Layout Estimation with Cross-Scale Distortion Awareness

1 code implementation CVPR 2023 Zhijie Shen, Zishuo Zheng, Chunyu Lin, Lang Nie, Kang Liao, Shuai Zheng, Yao Zhao

Based on the Manhattan World assumption, most existing indoor layout estimation schemes focus on recovering layouts from vertically compressed 1D sequences.

Room Layout Estimation Segmentation

Unleashing the potential of GNNs via Bi-directional Knowledge Transfer

no code implementations26 Oct 2023 Shuai Zheng, Zhizhe Liu, Zhenfeng Zhu, Xingxing Zhang, JianXin Li, Yao Zhao

On this basis, BiKT not only allows us to acquire knowledge from both the GNN and its derived model but promotes each other by injecting the knowledge into the other.

Domain Adaptation Representation Learning +1

DynaPipe: Optimizing Multi-task Training through Dynamic Pipelines

2 code implementations17 Nov 2023 Chenyu Jiang, Zhen Jia, Shuai Zheng, Yida Wang, Chuan Wu

This paper proposes a dynamic micro-batching approach to tackle sequence length variation and enable efficient multi-task model training.

Language Modelling Large Language Model +3

Contractive error feedback for gradient compression

no code implementations13 Dec 2023 Bingcong Li, Shuai Zheng, Parameswaran Raman, Anshumali Shrivastava, Georgios B. Giannakis

On-device memory concerns in distributed deep learning have become severe due to (i) the growth of model size in multi-GPU training, and (ii) the wide adoption of deep neural networks for federated learning on IoT devices which have limited storage.

Federated Learning Image Classification +2

Cannot find the paper you are looking for? You can Submit a new open access paper.