Search Results for author:
Found 27 papers, 21 papers with code
GPT4Image: Can Large Pre-trained Models Help Vision Models on Perception Tasks?
We present a new learning paradigm in which the knowledge extracted from large pre-trained models are utilized to help models like CNN and ViT learn enhanced representations and achieve better performance.
Masked Image Modeling with Local Multi-Scale Reconstruction
The lower layers are not explicitly guided and the interaction among their patches is only used for calculating new activations.
Network Expansion for Practical Training Acceleration
Recently, the sizes of deep neural networks and training datasets both increase drastically to pursue better performance in a practical sense.
FastMIM: Expediting Masked Image Modeling Pre-training for Vision
This paper presents FastMIM, a simple and generic framework for expediting masked image modeling with the following two steps: (i) pre-training vision backbones with low-resolution input images; and (ii) reconstructing Histograms of Oriented Gradients (HOG) feature instead of original RGB values of the input images.
GhostNetV2: Enhance Cheap Operation with Long-Range Attention
The convolutional operation can only capture local information in a window region, which prevents performance from being further improved.
Spatial-Channel Token Distillation for Vision MLPs
Recently, neural architectures with all Multi-layer Perceptrons (MLPs) have attracted great research interest from the computer vision community.
Ranked #455 on
Image Classification
on ImageNet
Vision GNN: An Image is Worth Graph of Nodes
In this paper, we propose to represent the image as a graph structure and introduce a new Vision GNN (ViG) architecture to extract graph-level feature for visual tasks.
Ranked #335 on
Image Classification
on ImageNet
Source-Free Domain Adaptation via Distribution Estimation
Domain Adaptation aims to transfer the knowledge learned from a labeled source domain to an unlabeled target domain whose data distributions are different.
From Quantum Graph Computing to Quantum Graph Learning: A Survey
Quantum computing (QC) is a new computational paradigm whose foundations relate to quantum physics.
PyramidTNT: Improved Transformer-in-Transformer Baselines with Pyramid Architecture
We hope this new baseline will be helpful to the further research and application of vision transformer.
An Image Patch is a Wave: Phase-Aware Vision MLP
To dynamically aggregate tokens, we propose to represent each token as a wave function with two parts, amplitude and phase.
Hire-MLP: Vision MLP via Hierarchical Rearrangement
Previous vision MLPs such as MLP-Mixer and ResMLP accept linearly flattened image patches as input, making them inflexible for different input sizes and hard to capture spatial information.
Homogeneous Architecture Augmentation for Neural Predictor
Specifically, a homogeneous architecture augmentation algorithm is proposed in HAAP to generate sufficient training data taking the use of homogeneous representation.
CMT: Convolutional Neural Networks Meet Vision Transformers
Vision transformers have been successfully applied to image recognition tasks due to their ability to capture long-range dependencies within an image.
Learning Efficient Vision Transformers via Fine-Grained Manifold Distillation
Specifically, we train a tiny student model to match a pre-trained teacher model in the patch-level manifold space.
Augmented Shortcuts for Vision Transformers
Transformer models have achieved great progress on computer vision tasks recently.
ReNAS: Relativistic Evaluation of Neural Architecture Search
An effective and efficient architecture performance evaluation scheme is essential for the success of Neural Architecture Search (NAS).
Patch Slimming for Efficient Vision Transformers
We first identify the effective patches in the last layer and then use them to guide the patch selection process of previous layers.
Vision Transformer Pruning
Vision transformer has achieved competitive performance on a variety of computer vision applications.
Manifold Regularized Dynamic Network Pruning
Then, the manifold relationship between instances and the pruned sub-networks will be aligned in the training procedure.
Learning Frequency Domain Approximation for Binary Neural Networks
Binary neural networks (BNNs) represent original full-precision weights and activations into 1-bit with sign function.
A Survey on Visual Transformer
Transformer, first applied to the field of natural language processing, is a type of deep neural network mainly based on the self-attention mechanism.
SCOP: Scientific Control for Reliable Neural Network Pruning
To increase the reliability of the results, we prefer to have a more rigorous research design by including a scientific control group as an essential part to minimize the effect of all factors except the association between the filter and expected network output.
A Semi-Supervised Assessor of Neural Architectures
A graph convolutional neural network is introduced to predict the performance of architectures based on the learned representations and their relation modeled by the graph.
Beyond Dropout: Feature Map Distortion to Regularize Deep Neural Networks
On one hand, massive trainable parameters significantly enhance the performance of these deep networks.
ReNAS:Relativistic Evaluation of Neural Architecture Search
An effective and efficient architecture performance evaluation scheme is essential for the success of Neural Architecture Search (NAS).
Bringing Giant Neural Networks Down to Earth with Unlabeled Data
Specifically, we exploit the unlabeled data to mimic the classification characteristics of giant networks, so that the original capacity can be preserved nicely.
