Search Results for author:
Found 22 papers, 13 papers with code
Stereo Neural Vernier Caliper
We propose a new object-centric framework for learning-based stereo 3D object detection.
Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Space
This paper explores the feasibility of finding an optimal sub-model from a vision transformer and introduces a pure vision transformer slimming (ViT-Slim) framework.
Data-Free Neural Architecture Search via Recursive Label Calibration
Then we use the synthesized data and their predicted soft-labels to guide neural architecture search.
Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation
The nonuniform quantization strategy for compressing neural networks usually achieves better performance than its counterpart, i. e., uniform strategy, due to its superior representational capacity.
Sliced Recursive Transformer
The proposed sliced recursive operation allows us to build a transformer with more than 100 or even 1000 layers effortlessly under a still small size (13~15M), to avoid difficulties in optimization when the model size is too large.
Ranked #141 on
Image Classification
on ImageNet
How Do Adam and Training Strategies Help BNNs Optimization?
We show the regularization effect of second-order momentum in Adam is crucial to revitalize the weights that are dead due to the activation saturation in BNNs.
"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization
However, the BN layer is costly to calculate and is typically implemented with non-binary parameters, leaving a hurdle for the efficient implementation of BNN training.
Ranked #139 on
Image Classification
on CIFAR-100
Is Label Smoothing Truly Incompatible with Knowledge Distillation: An Empirical Study
This work aims to empirically clarify a recently discovered perspective that label smoothing is incompatible with knowledge distillation.
S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration
In this paper, we focus on this more difficult scenario: learning networks where both weights and activations are binary, meanwhile, without any human annotated labels.
Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning
A common practice for this task is to train a model on the base set first and then transfer to novel classes through fine-tuning (Here fine-tuning procedure is defined as transferring knowledge from base to novel data, i. e. learning to transfer in few-shot scenario.)
Conditional Link Prediction of Category-Implicit Keypoint Detection
Keypoints of objects reflect their concise abstractions, while the corresponding connection links (CL) build the skeleton by detecting the intrinsic relations between keypoints.
Weight-dependent Gates for Network Pruning
In this paper, a simple yet effective network pruning framework is proposed to simultaneously address the problems of pruning indicator, pruning ratio, and efficiency constraint.
Joint Multi-Dimension Pruning via Numerical Gradient Update
To tackle these three naturally different dimensions, we proposed a general framework by defining pruning as seeking the best pruning vector (i. e., the numerical value of layer-wise channel number, spacial size, depth) and construct a unique mapping from the pruning vector to the pruned network structures.
Binarizing MobileNet via Evolution-based Searching
Inspired by one-shot architecture search frameworks, we manipulate the idea of group convolution to design efficient 1-Bit Convolutional Neural Networks (CNNs), assuming an approximately optimal trade-off between computational cost and model accuracy.
Attentive CutMix: An Enhanced Data Augmentation Approach for Deep Learning Based Image Classification
In this paper, we propose Attentive CutMix, a naturally enhanced augmentation strategy based on CutMix.
Un-Mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learning
This drawback hinders the model from learning subtle variance and fine-grained information.
ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions
In this paper, we propose several ideas for enhancing a binary network to close its accuracy gap from real-valued networks without incurring any additional computational cost.
Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization
Together, the redefinition of latent weights as inertia and the introduction of Bop enable a better understanding of BNN optimization and open up the way for further improvements in training methodologies for BNNs.
Single Path One-Shot Neural Architecture Search with Uniform Sampling
It is easy to train and fast to search.
Ranked #58 on
Neural Architecture Search
on ImageNet
(MACs metric)
MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning
In this paper, we propose a novel meta learning approach for automatic channel pruning of very deep neural networks.
Bi-Real Net: Binarizing Deep Network Towards Real-Network Performance
To address the training difficulty, we propose a training algorithm using a tighter approximation to the derivative of the sign function, a magnitude-aware gradient for weight updating, a better initialization method, and a two-step scheme for training a deep network.
Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm
In this work, we study the 1-bit convolutional neural networks (CNNs), of which both the weights and activations are binary.
