Search Results for author:
Found 32 papers, 26 papers with code
Common 7B Language Models Already Possess Strong Math Capabilities
This paper shows that the LLaMA-2 7B model with common pre-training already exhibits strong mathematical abilities, as evidenced by its impressive accuracy of 97. 7% and 72. 0% on the GSM8K and MATH benchmarks, respectively, when selecting the best response from 256 random generations.
FP8-LM: Training FP8 Large Language Models
In this paper, we explore FP8 low-bit data formats for efficient training of large language models (LLMs).
TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance
In this paper, we propose a novel cross-modal distillation method, called TinyCLIP, for large-scale language-image pre-trained models.
Exploring Lightweight Hierarchical Vision Transformers for Efficient Visual Tracking
The Bridge Module incorporates the high-level information of deep features into the shallow large-resolution features.
EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention
Comprehensive experiments demonstrate EfficientViT outperforms existing efficient models, striking a good trade-off between speed and accuracy.
Unified Sequence-to-Sequence Learning for Single- and Multi-Modal Visual Object Tracking
In this paper, we introduce a new sequence-to-sequence learning framework for RGB-based and multi-modal object tracking.
Ranked #1 on
Rgb-T Tracking
on LasHeR
iCLIP: Bridging Image Classification and Contrastive Language-Image Pre-Training for Visual Recognition
This paper presents a method that effectively combines two prevalent visual recognition methods, i. e., image classification and contrastive language-image pre-training, dubbed iCLIP.
Attentive Mask CLIP
To address this issue, we propose an attentive token removal approach for CLIP training, which retains tokens with a high semantic correlation to the text description.
Expanding Language-Image Pretrained Models for General Video Recognition
Extensive experiments demonstrate that our approach is effective and can be generalized to different video recognition scenarios.
Ranked #8 on
Zero-Shot Action Recognition
on Kinetics
TinyViT: Fast Pretraining Distillation for Small Vision Transformers
It achieves a top-1 accuracy of 84. 8% on ImageNet-1k with only 21M parameters, being comparable to Swin-B pretrained on ImageNet-21k while using 4. 2 times fewer parameters.
Ranked #135 on
Image Classification
on ImageNet
PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies
In this work, we revisit the classical PointNet++ through a systematic study of model training and scaling strategies, and offer two major contributions.
Ranked #3 on
3D Semantic Segmentation
on OpenTrench3D
MiniViT: Compressing Vision Transformers with Weight Multiplexing
The central idea of MiniViT is to multiplex the weights of consecutive transformer blocks.
Ranked #212 on
Image Classification
on ImageNet
(using extra training data)
Searching the Search Space of Vision Transformer
Vision Transformer has shown great visual representation power in substantial vision tasks such as recognition and detection, and thus been attracting fast-growing efforts on manually designing more effective architectures.
Learning to Track Objects from Unlabeled Videos
In this paper, we propose to learn an Unsupervised Single Object Tracker (USOT) from scratch.
Rethinking and Improving Relative Position Encoding for Vision Transformer
We then propose new relative position encoding methods dedicated to 2D images, called image RPE (iRPE).
Ranked #140 on
Object Detection
on COCO minival
AutoFormer: Searching Transformers for Visual Recognition
Specifically, the performance of these subnets with weights inherited from the supernet is comparable to those retrained from scratch.
Ranked #1 on
Fine-Grained Image Classification
on Oxford 102 Flowers
(Top 1 Accuracy metric)
Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training
To tackle this, we propose a fully Transformer visual embedding for VLP to better learn visual relation and further promote inter-modal alignment.
Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training
To tackle this, we propose a fully Transformer visual embedding for VLP to better learn visual relation and further promote inter-modal alignment.
LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search
Object tracking has achieved significant progress over the past few years.
One-Shot Neural Ensemble Architecture Search by Diversity-Guided Search Space Shrinking
In this paper, we propose a one-shot neural ensemble architecture search (NEAS) solution that addresses the two challenges.
Learning Spatio-Temporal Transformer for Visual Tracking
In this paper, we present a new tracking architecture with an encoder-decoder transformer as the key component.
Ranked #19 on
Visual Object Tracking
on TrackingNet
Multi-Scale 2D Temporal Adjacent Networks for Moment Localization with Natural Language
It is a challenging problem because a target moment may take place in the context of other temporal moments in the untrimmed video.
Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search
The experiments on ImageNet verify such path distillation method can improve the convergence ratio and performance of the hypernetwork, as well as boosting the training of subnetworks.
Revisiting Anchor Mechanisms for Temporal Action Localization
To address this problem, this paper proposes a novel anchor-free action localization module that assists action localization by temporal points.
Towards Accurate Pixel-wise Object Tracking by Attention Retrieval
We first build a look-up-table (LUT) with the ground-truth mask in the starting frame, and then retrieves the LUT to obtain an attention map for spatial constraints.
Cyclic Differentiable Architecture Search
First, the search network generates an initial architecture for evaluation, and the weights of the evaluation network are optimized.
Ranked #17 on
Neural Architecture Search
on NAS-Bench-201, CIFAR-10
Ocean: Object-aware Anchor-free Tracking
In this paper, we propose a novel object-aware anchor-free network to address this issue.
Ranked #2 on
Visual Object Tracking
on VOT2019
A Transductive Approach for Video Object Segmentation
Semi-supervised video object segmentation aims to separate a target object from a video sequence, given the mask in the first frame.
Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language
We address the problem of retrieving a specific moment from an untrimmed video by a query sentence.
Learning Sparse 2D Temporal Adjacent Networks for Temporal Action Localization
In this report, we introduce the Winner method for HACS Temporal Action Localization Challenge 2019.
Deeper and Wider Siamese Networks for Real-Time Visual Tracking
Siamese networks have drawn great attention in visual tracking because of their balanced accuracy and speed.
Ranked #2 on
Visual Object Tracking
on VOT2017
Illumination Estimation Based on Bilayer Sparse Coding
In this paper, we propose a novel bilayer sparse coding model for illumination estimation that considers image similarity in terms of both low level color distribution and high level image scene content simultaneously.
