Search Results for author:
Found 36 papers, 18 papers with code
CvT: Introducing Convolutions to Vision Transformers
We present in this paper a new architecture, named Convolutional vision Transformer (CvT), that improves Vision Transformer (ViT) in performance and efficiency by introducing convolutions into ViT to yield the best of both designs.
Ranked #3 on
Image Classification
on Flowers-102
(using extra training data)
TinyViT: Fast Pretraining Distillation for Small Vision Transformers
It achieves a top-1 accuracy of 84. 8% on ImageNet-1k with only 21M parameters, being comparable to Swin-B pretrained on ImageNet-21k while using 4. 2 times fewer parameters.
Ranked #133 on
Image Classification
on ImageNet
Dynamic Head: Unifying Object Detection Heads with Attentions
In this paper, we present a novel dynamic head framework to unify object detection heads with attentions.
Ranked #2 on
Object Detection
on COCO 2017 val
(AP75 metric)
Dynamic Convolution: Attention over Convolution Kernels
Light-weight convolutional neural networks (CNNs) suffer performance degradation as their low computational budgets constrain both the depth (number of convolution layers) and the width (number of channels) of CNNs, resulting in limited representation capability.
Ranked #905 on
Image Classification
on ImageNet
MiniViT: Compressing Vision Transformers with Weight Multiplexing
The central idea of MiniViT is to multiplex the weights of consecutive transformer blocks.
Ranked #209 on
Image Classification
on ImageNet
(using extra training data)
TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance
In this paper, we propose a novel cross-modal distillation method, called TinyCLIP, for large-scale language-image pre-trained models.
Mobile-Former: Bridging MobileNet and Transformer
This structure leverages the advantages of MobileNet at local processing and transformer at global interaction.
Florence: A New Foundation Model for Computer Vision
Computer vision foundation models, which are trained on diverse, large-scale dataset and can be adapted to a wide range of downstream tasks, are critical for this mission to solve real-world computer vision applications.
Ranked #1 on
Action Recognition In Videos
on Kinetics-600
MicroNet: Improving Image Recognition with Extremely Low FLOPs
This paper aims at addressing the problem of substantial performance degradation at extremely low computational cost (e. g. 5M FLOPs on ImageNet classification).
BEVT: BERT Pretraining of Video Transformers
This design is motivated by two observations: 1) transformers learned on image datasets provide decent spatial priors that can ease the learning of video transformers, which are often times computationally-intensive if trained from scratch; 2) discriminative clues, i. e., spatial and temporal information, needed to make correct predictions vary among different videos due to large intra-class and inter-class variations.
Ranked #8 on
Action Recognition
on Diving-48
Reduce Information Loss in Transformers for Pluralistic Image Inpainting
The indices of quantized pixels are used as tokens for the inputs and prediction targets of transformer.
Ranked #6 on
Seeing Beyond the Visible
on KITTI360-EX
Revisiting Dynamic Convolution via Matrix Decomposition
It has two limitations: (a) it increases the number of convolutional weights by K-times, and (b) the joint optimization of dynamic attention and static convolution kernels is challenging.
Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learning
For the choice of teacher models, we observe that students taught by video teachers perform better on temporally-heavy video tasks, while image teachers transfer stronger spatial representations for spatially-heavy video tasks.
Ranked #1 on
Self-Supervised Action Recognition
on HMDB51
Should All Proposals be Treated Equally in Object Detection?
We formulate this as a learning problem where the goal is to assign operators to proposals, in the detection head, so that the total computational cost is constrained and the precision is maximized.
Stronger NAS with Weaker Predictors
We propose a paradigm shift from fitting the whole architecture space using one strong predictor, to progressively fitting a search path towards the high-performance sub-space through a set of weaker predictors.
Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Label-Efficient Representations
This motivates us to shift the paradigm from combining loss at the end, to choosing the proper learning method per network layer.
Real-Time Image Segmentation via Hybrid Convolutional-Transformer Architecture Search
Manually replacing convolution layers with multi-head self-attention is non-trivial due to the costly overhead in memory to maintain high resolution.
Towards Better Analysis of Machine Learning Models: A Visual Analytics Perspective
Interactive model analysis, the process of understanding, diagnosing, and refining a machine learning model with the help of interactive visualization, is very important for users to efficiently solve real-world artificial intelligence and data mining problems.
Towards Better Analysis of Deep Convolutional Neural Networks
Deep convolutional neural networks (CNNs) have achieved breakthrough performance in many pattern recognition tasks such as image classification.
Analyzing the Noise Robustness of Deep Neural Networks
Deep neural networks (DNNs) are vulnerable to maliciously generated adversarial examples.
Analyzing the Noise Robustness of Deep Neural Networks
The key is to compare and analyze the datapaths of both the adversarial and normal examples.
DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search
One common way is searching on a smaller proxy dataset (e. g., CIFAR-10) and then transferring to the target task (e. g., ImageNet).
Diagnosing Concept Drift with Visual Analytics
Concept drift is a phenomenon in which the distribution of a data stream changes over time in unforeseen ways, causing prediction models built on historical data to become inaccurate.
Weak NAS Predictor Is All You Need
Rather than expecting a single strong predictor to model the whole space, we seek a progressive line of weak predictors that can connect a path to the best architecture, thus greatly simplifying the learning task of each predictor.
MicroNet: Towards Image Recognition with Extremely Low FLOPs
In this paper, we present MicroNet, which is an efficient convolutional neural network using extremely low computational cost (e. g. 6 MFLOPs on ImageNet classification).
Residual Mixture of Experts
In this paper, we propose Residual Mixture of Experts (RMoE), an efficient training pipeline for MoE vision transformers on downstream tasks, such as segmentation and detection.
Detection Hub: Unifying Object Detection Datasets via Query Adaptation on Language Embedding
Detection Hub further achieves SoTA performance on UODB benchmark with wide variety of datasets.
Visual Analysis of Neural Architecture Spaces for Summarizing Design Principles
To ease this process, we develop ArchExplorer, a visual analysis method for understanding a neural architecture space and summarizing design principles.
Video Mobile-Former: Video Recognition with Efficient Global Spatial-temporal Modeling
To avoid significant computational cost incurred by computing self-attention between the large number of local patches in videos, we propose to use very few global tokens (e. g., 6) for a whole video in Transformers to exchange information with 3D-CNNs with a cross-attention mechanism.
Self-Supervised Learning based on Heat Equation
When transferring to object detection with frozen backbone, QB-Heat outperforms MoCo-v2 and supervised pre-training on ImageNet by 7. 9 and 4. 5 AP respectively.
Image as First-Order Norm+Linear Autoregression: Unveiling Mathematical Invariance
This paper introduces a novel mathematical property applicable to diverse images, referred to as FINOLA (First-Order Norm+Linear Autoregressive).
Foundation Models Meet Visualizations: Challenges and Opportunities
Recent studies have indicated that foundation models, such as BERT and GPT, excel in adapting to a variety of downstream tasks.
On the Hidden Waves of Image
We term this phenomenon hidden waves, as it reveals that, although the speeds of the set of wave equations and autoregressive coefficient matrices are latent, they are both learnable and shared across images.
Fully Authentic Visual Question Answering Dataset from Online Communities
Visual Question Answering (VQA) entails answering questions about images.
An Evaluation of GPT-4V and Gemini in Online VQA
While there is much excitement about the potential of large multimodal models (LMM), a comprehensive evaluation is critical to establish their true capabilities and limitations.
