Search Results for author: Anbang Yao

Found 34 papers, 23 papers with code

NOAH: Learning Pairwise Object Category Attentions for Image Classification

1 code implementation4 Feb 2024 Chao Li, Aojun Zhou, Anbang Yao

We observe that the head structures of mainstream DNNs adopt a similar feature encoding pipeline, exploiting global feature dependencies while disregarding local ones.

Classification Multi-Label Image Classification +1

Augmentation-Free Dense Contrastive Knowledge Distillation for Efficient Semantic Segmentation

1 code implementation7 Dec 2023 Jiawei Fan, Chao Li, Xiaolong Liu, Meina Song, Anbang Yao

In order to address this problem, we present Augmentation-free Dense Contrastive Knowledge Distillation (Af-DCD), a new contrastive distillation learning paradigm to train compact and accurate deep neural networks for semantic segmentation applications.

Contrastive Learning Data Augmentation +6

KernelWarehouse: Towards Parameter-Efficient Dynamic Convolution

1 code implementation16 Aug 2023 Chao Li, Anbang Yao

Dynamic convolution learns a linear mixture of $n$ static kernels weighted with their sample-dependent attentions, demonstrating superior performance compared to normal convolution.

Ske2Grid: Skeleton-to-Grid Representation Learning for Action Recognition

1 code implementation15 Aug 2023 Dongqi Cai, Yangyuxuan Kang, Anbang Yao, Yurong Chen

This paper presents Ske2Grid, a new representation learning framework for improved skeleton-based action recognition.

Action Recognition Representation Learning +1

NORM: Knowledge Distillation via N-to-One Representation Matching

1 code implementation23 May 2023 Xiaolong Liu, Lujun Li, Chao Li, Anbang Yao

By sequentially splitting the expanded student representation into N non-overlapping feature segments having the same number of feature channels as the teacher's, they can be readily forced to approximate the intact teacher representation simultaneously, formulating a novel many-to-one representation matching mechanism conditioned on a single teacher-student layer pair.

Knowledge Distillation

Compacting Binary Neural Networks by Sparse Kernel Selection

no code implementations CVPR 2023 Yikai Wang, Wenbing Huang, Yinpeng Dong, Fuchun Sun, Anbang Yao

Binary Neural Network (BNN) represents convolution weights with 1-bit values, which enhances the efficiency of storage and computation.

Binarization

3D Human Pose Lifting with Grid Convolution

1 code implementation17 Feb 2023 Yangyuxuan Kang, Yuyang Liu, Anbang Yao, Shandong Wang, Enhua Wu

Existing lifting networks for regressing 3D human poses from 2D single-view poses are typically constructed with linear layers based on graph-structured representation learning.

Representation Learning

Omni-Dimensional Dynamic Convolution

1 code implementation ICLR 2022 Chao Li, Aojun Zhou, Anbang Yao

Learning a single static convolutional kernel in each convolutional layer is the common training paradigm of modern Convolutional Neural Networks (CNNs).

Efficient Meta-Tuning for Content-aware Neural Video Delivery

1 code implementation20 Jul 2022 Xiaoqi Li, Jiaming Liu, Shizun Wang, Cheng Lyu, Ming Lu, Yurong Chen, Anbang Yao, Yandong Guo, Shanghang Zhang

Our method significantly reduces the computational cost and achieves even better performance, paving the way for applying neural video delivery techniques to practical applications.

Super-Resolution

Dynamic Normalization and Relay for Video Action Recognition

1 code implementation NeurIPS 2021 Dongqi Cai, Anbang Yao, Yurong Chen

In this paper, we present Dynamic Normalization and Relay (DNR), an improved normalization design, to augment the spatial-temporal representation learning of any deep action recognition model, adapting to small batch size training settings.

Action Recognition Representation Learning +1

Sub-bit Neural Networks: Learning to Compress and Accelerate Binary Neural Networks

1 code implementation ICCV 2021 Yikai Wang, Yi Yang, Fuchun Sun, Anbang Yao

In the low-bit quantization field, training Binary Neural Networks (BNNs) is the extreme solution to ease the deployment of deep models on resource-constrained devices, having the lowest storage cost and significantly cheaper bit-wise operations compared to 32-bit floating-point counterparts.

Quantization

Explicit Connection Distillation

no code implementations1 Jan 2021 Lujun Li, Yikai Wang, Anbang Yao, Yi Qian, Xiao Zhou, Ke He

In this paper, we present Explicit Connection Distillation (ECD), a new KD framework, which addresses the knowledge distillation problem in a novel perspective of bridging dense intermediate feature connections between a student network and its corresponding teacher generated automatically in the training, achieving knowledge transfer goal via direct cross-network layer-to-layer gradients propagation, without need to define complex distillation losses and assume a pre-trained teacher model to be available.

Image Classification Knowledge Distillation +1

Weights Having Stable Signs Are Important: Finding Primary Subnetworks and Kernels to Compress Binary Weight Networks

no code implementations1 Jan 2021 Zhaole Sun, Anbang Yao

Binary Weight Networks (BWNs) have significantly lower computational and memory costs compared to their full-precision counterparts.

Quantization

Knowledge Transfer via Dense Cross-Layer Mutual-Distillation

1 code implementation ECCV 2020 Anbang Yao, Dawei Sun

Knowledge Distillation (KD) based methods adopt the one-way Knowledge Transfer (KT) scheme in which training a lower-capacity student network is guided by a pre-trained high-capacity teacher network.

Knowledge Distillation Representation Learning +1

Resolution Switchable Networks for Runtime Efficient Image Recognition

1 code implementation ECCV 2020 Yikai Wang, Fuchun Sun, Duo Li, Anbang Yao

We propose a general method to train a single convolutional neural network which is capable of switching image resolutions at inference.

Knowledge Distillation Quantization

CASNet: Common Attribute Support Network for image instance and panoptic segmentation

no code implementations17 Jul 2020 Xiaolong Liu, Yuqing Hou, Anbang Yao, Yurong Chen, Keqiang Li

Given the insight that pixels belonging to one instance have one or more common attributes of current instance, we bring up an one-stage instance segmentation network named Common Attribute Support Network (CASNet), which realizes instance segmentation by predicting and clustering common attributes.

Attribute Clustering +5

PSConv: Squeezing Feature Pyramid into One Compact Poly-Scale Convolutional Layer

1 code implementation ECCV 2020 Duo Li, Anbang Yao, Qifeng Chen

Despite their strong modeling capacities, Convolutional Neural Networks (CNNs) are often scale-sensitive.

Representation Learning

Learning to Learn Parameterized Classification Networks for Scalable Input Images

1 code implementation ECCV 2020 Duo Li, Anbang Yao, Qifeng Chen

To achieve efficient and flexible image classification at runtime, we employ meta learners to generate convolutional weights of main networks for various input scales and maintain privatized Batch Normalization layers per scale.

Classification General Classification +2

Learning Two-View Correspondences and Geometry Using Order-Aware Network

1 code implementation ICCV 2019 Jiahui Zhang, Dawei Sun, Zixin Luo, Anbang Yao, Lei Zhou, Tianwei Shen, Yurong Chen, Long Quan, Hongen Liao

First, to capture the local context of sparse correspondences, the network clusters unordered input correspondences by learning a soft assignment matrix.

Vocal Bursts Valence Prediction

HBONet: Harmonious Bottleneck on Two Orthogonal Dimensions

1 code implementation ICCV 2019 Duo Li, Aojun Zhou, Anbang Yao

MobileNets, a class of top-performing convolutional neural network architectures in terms of accuracy and efficiency trade-off, are increasingly used in many resourceaware vision applications.

object-detection Object Detection +2

Deeply-supervised Knowledge Synergy

1 code implementation CVPR 2019 Dawei Sun, Anbang Yao, Aojun Zhou, Hao Zhao

Convolutional Neural Networks (CNNs) have become deeper and more complicated compared with the pioneering AlexNet.

General Classification Image Classification

A Closed-form Solution to Universal Style Transfer

2 code implementations ICCV 2019 Ming Lu, Hao Zhao, Anbang Yao, Yurong Chen, Feng Xu, Li Zhang

Although plenty of methods have been proposed, a theoretical analysis of feature transform is still missing.

Style Transfer

SnapQuant: A Probabilistic and Nested Parameterization for Binary Networks

no code implementations27 Sep 2018 Kuan Wang, Hao Zhao, Anbang Yao, Aojun Zhou, Dawei Sun, Yurong Chen

During the training phase, we generate binary weights on-the-fly since what we actually maintain is the policy network, and all the binary weights are used in a burn-after-reading style.

Explicit Loss-Error-Aware Quantization for Low-Bit Deep Neural Networks

no code implementations CVPR 2018 Aojun Zhou, Anbang Yao, Kuan Wang, Yurong Chen

Through explicitly regularizing the loss perturbation and the weight approximation error in an incremental way, we show that such a new optimization method is theoretically reasonable and practically effective.

Quantization

Network Sketching: Exploiting Binary Structure in Deep CNNs

no code implementations CVPR 2017 Yiwen Guo, Anbang Yao, Hao Zhao, Yurong Chen

Convolutional neural networks (CNNs) with deep architectures have substantially advanced the state-of-the-art in computer vision tasks.

Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights

3 code implementations10 Feb 2017 Aojun Zhou, Anbang Yao, Yiwen Guo, Lin Xu, Yurong Chen

The weights in the other group are responsible to compensate for the accuracy loss from the quantization, thus they are the ones to be re-trained.

Quantization

Dynamic Network Surgery for Efficient DNNs

3 code implementations NeurIPS 2016 Yiwen Guo, Anbang Yao, Yurong Chen

In this paper, we propose a novel network compression method called dynamic network surgery, which can remarkably reduce the network complexity by making on-the-fly connection pruning.

HyperNet: Towards Accurate Region Proposal Generation and Joint Object Detection

no code implementations CVPR 2016 Tao Kong, Anbang Yao, Yurong Chen, Fuchun Sun

Almost all of the current top-performing object detection networks employ region proposals to guide the search for object instances.

Object object-detection +2

Cannot find the paper you are looking for? You can Submit a new open access paper.