Search Results for author: Guangrun Wang

Found 28 papers, 13 papers with code

Understanding Weight Similarity of Neural Networks via Chain Normalization Rule and Hypothesis-Training-Testing

no code implementations8 Aug 2022 Guangcong Wang, Guangrun Wang, Wenqi Liang, JianHuang Lai

We extend the traditional hypothesis-testing method to a hypothesis-training-testing statistical inference method to validate the hypothesis on the weight similarity of neural networks.

Representation Learning

Automated Progressive Learning for Efficient Training of Vision Transformers

1 code implementation CVPR 2022 Changlin Li, Bohan Zhuang, Guangrun Wang, Xiaodan Liang, Xiaojun Chang, Yi Yang

First, we develop a strong manual baseline for progressive learning of ViTs, by introducing momentum growth (MoGrow) to bridge the gap brought by model growth.

Beyond Fixation: Dynamic Window Visual Transformer

1 code implementation CVPR 2022 Pengzhen Ren, Changlin Li, Guangrun Wang, Yun Xiao, Qing Du, Xiaodan Liang, Xiaojun Chang

Recently, a surge of interest in visual transformers is to reduce the computational cost by limiting the calculation of self-attention to a local window.

Semantic-Aware Auto-Encoders for Self-Supervised Representation Learning

no code implementations CVPR 2022 Guangrun Wang, Yansong Tang, Liang Lin, Philip H.S. Torr

Inspired by perceptual learning that could use cross-view learning to perceive concepts and semantics, we propose a novel AE that could learn semantic-aware representation via cross-view image reconstruction.

Image Reconstruction Representation Learning +1

DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Transformers

1 code implementation21 Sep 2021 Changlin Li, Guangrun Wang, Bing Wang, Xiaodan Liang, Zhihui Li, Xiaojun Chang

Dynamic networks have shown their promising capability in reducing theoretical computation complexity by adapting their architectures to the input during inference.

Fairness Model Compression

EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation

1 code implementation Findings (EMNLP) 2021 Chenhe Dong, Guangrun Wang, Hang Xu, Jiefeng Peng, Xiaozhe Ren, Xiaodan Liang

In this paper, we have a critical insight that improving the feed-forward network (FFN) in BERT has a higher gain than improving the multi-head attention (MHA) since the computational cost of FFN is 2$\sim$3 times larger than MHA.

Data Augmentation Knowledge Distillation

Solving Inefficiency of Self-supervised Representation Learning

1 code implementation ICCV 2021 Guangrun Wang, Keze Wang, Guangcong Wang, Philip H. S. Torr, Liang Lin

In this paper, we reveal two contradictory phenomena in contrastive learning that we call under-clustering and over-clustering problems, which are major obstacles to learning efficiency.

Contrastive Learning Representation Learning +3

Joint Learning of Neural Transfer and Architecture Adaptation for Image Recognition

no code implementations31 Mar 2021 Guangrun Wang, Liang Lin, Rongcong Chen, Guangcong Wang, Jiqi Zhang

In this work, we prove that dynamically adapting network architectures tailored for each domain task along with weight finetuning benefits in both efficiency and effectiveness, compared to the existing image recognition pipeline that only tunes the weights regardless of the architecture.

Age Estimation Image Classification +4

Dynamic Slimmable Network

1 code implementation CVPR 2021 Changlin Li, Guangrun Wang, Bing Wang, Xiaodan Liang, Zhihui Li, Xiaojun Chang

Here, we explore a dynamic network slimming regime, named Dynamic Slimmable Network (DS-Net), which aims to achieve good hardware-efficiency via dynamically adjusting filter numbers of networks at test time with respect to different inputs, while keeping filters stored statically and contiguously in hardware to prevent the extra burden.

Fairness Model Compression

BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search

1 code implementation ICCV 2021 Changlin Li, Tao Tang, Guangrun Wang, Jiefeng Peng, Bing Wang, Xiaodan Liang, Xiaojun Chang

In this work, we present Block-wisely Self-supervised Neural Architecture Search (BossNAS), an unsupervised NAS method that addresses the problem of inaccurate architecture rating caused by large weight-sharing space and biased supervision in previous methods.

Image Classification Neural Architecture Search

Heterogeneous Model Transfer between Different Neural Networks

no code implementations1 Jan 2021 Guangcong Wang, JianHuang Lai, Wenqi Liang, Guangrun Wang

Specifically, we select the longest chain from the source model and transfer it to the longest chain of the target model.

Neural Architecture Search

EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning

1 code implementation ECCV 2020 Bailin Li, Bowen Wu, Jiang Su, Guangrun Wang, Liang Lin

Many algorithms try to predict model performance of the pruned sub-nets by introducing various evaluation methods.

Network Pruning

Transferable, Controllable, and Inconspicuous Adversarial Attacks on Person Re-identification With Deep Mis-Ranking

1 code implementation CVPR 2020 Hongjun Wang, Guangrun Wang, Ya Li, Dongyu Zhang, Liang Lin

To examine the robustness of ReID systems is rather important because the insecurity of ReID systems may cause severe losses, e. g., the criminals may use the adversarial perturbations to cheat the CCTV systems.

Adversarial Attack Person Re-Identification

Function Feature Learning of Neural Networks

no code implementations25 Sep 2019 Guangcong Wang, JianHuang Lai, Guangrun Wang, Wenqi Liang

We present a Function Feature Learning (FFL) method that can measure the similarity of non-convex neural networks.

Learnable Parameter Similarity

no code implementations27 Jul 2019 Guangcong Wang, Jian-Huang Lai, Wenqi Liang, Guangrun Wang

Most of the existing approaches focus on specific visual tasks while ignoring the relations between them.

Transfer Learning

Adaptively Connected Neural Networks

1 code implementation CVPR 2019 Guangrun Wang, Keze Wang, Liang Lin

This paper presents a novel adaptively connected neural network (ACNet) to improve the traditional convolutional neural networks (CNNs) {in} two aspects.

Document Classification Image Classification +1

Kalman Normalization: Normalizing Internal Representations Across Network Layers

no code implementations NeurIPS 2018 Guangrun Wang, Jiefeng Peng, Ping Luo, Xinjiang Wang, Liang Lin

In this paper, we present a novel normalization method, called Kalman Normalization (KN), for improving and accelerating the training of DNNs, particularly under the context of micro-batches.

object-detection Object Detection

Batch Kalman Normalization: Towards Training Deep Neural Networks with Micro-Batches

no code implementations9 Feb 2018 Guangrun Wang, Jiefeng Peng, Ping Luo, Xinjiang Wang, Liang Lin

As an indispensable component, Batch Normalization (BN) has successfully improved the training of deep neural networks (DNNs) with mini-batches, by normalizing the distribution of the internal representation for each hidden layer.

Image Classification

Deep Dual Learning for Semantic Image Segmentation

no code implementations ICCV 2017 Ping Luo, Guangrun Wang, Liang Lin, Xiaogang Wang

The estimated labelmaps that capture accurate object classes and boundaries are used as ground truths in training to boost performance.

Image Segmentation Semantic Segmentation

Hierarchical Scene Parsing by Weakly Supervised Learning with Image Descriptions

no code implementations27 Sep 2017 Ruimao Zhang, Liang Lin, Guangrun Wang, Meng Wang, WangMeng Zuo

Rather than relying on elaborative annotations (e. g., manually labeled semantic maps and relations), we train our deep model in a weakly-supervised learning manner by leveraging the descriptive sentences of the training images.

Scene Labeling Scene Understanding

Learning Object Interactions and Descriptions for Semantic Image Segmentation

no code implementations CVPR 2017 Guangrun Wang, Ping Luo, Liang Lin, Xiaogang Wang

This work significantly increases segmentation accuracy of CNNs by learning from an Image Descriptions in the Wild (IDW) dataset.

Image Captioning Image Segmentation +1

Cross-Domain Visual Matching via Generalized Similarity Measure and Feature Learning

no code implementations13 May 2016 Liang Lin, Guangrun Wang, WangMeng Zuo, Xiangchu Feng, Lei Zhang

Cross-domain visual data matching is one of the fundamental problems in many real-world vision tasks, e. g., matching persons across ID photos and surveillance videos.

Face Verification Model Optimization +2

DARI: Distance metric And Representation Integration for Person Verification

no code implementations15 Apr 2016 Guangrun Wang, Liang Lin, Shengyong Ding, Ya Li, Qing Wang

The past decade has witnessed the rapid development of feature representation learning and distance metric learning, whereas the two steps are often discussed separately.

Ranked #7 on Person Re-Identification on SYSU-30k (using extra training data)

Metric Learning Person Re-Identification +1

Deep Structured Scene Parsing by Learning with Image Descriptions

no code implementations CVPR 2016 Liang Lin, Guangrun Wang, Rui Zhang, Ruimao Zhang, Xiaodan Liang, WangMeng Zuo

This paper addresses a fundamental problem of scene understanding: How to parse the scene image into a structured configuration (i. e., a semantic object hierarchy with object interaction relations) that finely accords with human perception.

Scene Labeling Scene Understanding

Deep Feature Learning with Relative Distance Comparison for Person Re-identification

no code implementations11 Dec 2015 Shengyong Ding, Liang Lin, Guangrun Wang, Hongyang Chao

Identifying the same individual across different scenes is an important yet difficult task in intelligent video surveillance.

Ranked #9 on Person Re-Identification on SYSU-30k (using extra training data)

Person Re-Identification

Cannot find the paper you are looking for? You can Submit a new open access paper.