Search Results for author:
Found 61 papers, 43 papers with code
Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding
We categorize Mamba into four roles for modeling videos, deriving a Video Mamba Suite composed of 14 models/modules, and evaluating them on 12 video understanding tasks.
Ranked #1 on
Temporal Action Localization
on FineAction
Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures
Our evaluations demonstrate that VRWKV surpasses ViT's performance in image classification and has significantly faster speeds and lower memory usage processing high-resolution inputs.
PromptRR: Diffusion Models as Prompt Generators for Single Image Reflection Removal
For the prompt generation, we first propose a prompt pre-training strategy to train a frequency prompt encoder that encodes the ground-truth image into LF and HF prompts.
MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizer
Developing generative models for interleaved image-text data has both research and practical value.
Efficient Deformable ConvNets: Rethinking Dynamic and Sparse Operator for Vision Applications
The advancements in speed and efficiency of DCNv4, combined with its robust performance across diverse vision tasks, show its potential as a foundational building block for future vision models.
CRA-PCN: Point Cloud Completion with Intra- and Inter-level Cross-Resolution Transformers
Point cloud completion is an indispensable task for recovering complete point clouds due to incompleteness caused by occlusion, limited sensor resolution, etc.
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
However, the progress in vision and vision-language foundation models, which are also critical elements of multi-modal AGI, has not kept pace with LLMs.
Ranked #1 on
Zero-Shot Video Retrieval
on MSR-VTT-full
(using extra training data)
Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?
We initially observed that the nuScenes dataset, characterized by relatively simple driving scenarios, leads to an under-utilization of perception information in end-to-end models incorporating ego status, such as the ego vehicle's velocity.
Deep Video Restoration for Under-Display Camera
With the pipeline, we build the first large-scale UDC video restoration dataset called PexelsUDC, which includes two subsets named PexelsUDC-T and PexelsUDC-P corresponding to different displays for UDC.
Memory-and-Anticipation Transformer for Online Action Understanding
Based on this idea, we present Memory-and-Anticipation Transformer (MAT), a memory-anticipation-based approach, to address the online action detection and anticipation tasks.
Ranked #1 on
Action Detection
on THUMOS' 14
FB-BEV: BEV Representation from Forward-Backward View Transformations
Currently, the two most prominent VTM paradigms are forward projection and backward projection.
The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open World
We present the All-Seeing (AS) project: a large-scale data and model for recognizing and understanding everything in the open world.
AVSegFormer: Audio-Visual Segmentation with Transformer
In this paper, we propose AVSegFormer, a novel framework for AVS tasks that leverages the transformer architecture.
GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions
Second, we introduce a residual dense transformer block (RDTB) as the final GridFormer layer.
VideoLLM: Modeling Video Sequence with Large Language Models
Building upon this insight, we propose a novel framework called VideoLLM that leverages the sequence reasoning capabilities of pre-trained LLMs from natural language processing (NLP) for video sequence understanding.
Graph Propagation Transformer for Graph Representation Learning
The core insight of our method is to fully consider the information propagation among nodes and edges in a graph when building the attention module in the transformer blocks.
Ranked #2 on
Graph Regression
on PCQM4M-LSC
(Validation MAE metric)
VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks
We hope this model can set a new baseline for generalist vision and language models.
MRSN: Multi-Relation Support Network for Video Action Detection
Action detection is a challenging video understanding task, requiring modeling spatio-temporal and interaction relations.
DDP: Diffusion Model for Dense Visual Prediction
We propose a simple, efficient, yet powerful framework for dense visual predictions based on the conditional diffusion pipeline.
Ranked #2 on
Monocular Depth Estimation
on SUN-RGBD
Champion Solution for the WSDM2023 Toloka VQA Challenge
In this report, we present our champion solution to the WSDM2023 Toloka Visual Question Answering (VQA) Challenge.
Restoring Vision in Hazy Weather with Hierarchical Contrastive Learning
HCD consists of a hierarchical dehazing network (HDN) and a novel hierarchical contrastive loss (HCL).
InternVideo-Ego4D: A Pack of Champion Solutions to Ego4D Challenges
In this report, we present our champion solutions to five tracks at Ego4D challenge.
Ranked #1 on
State Change Object Detection
on Ego4D
Exploring State Change Capture of Heterogeneous Backbones @ Ego4D Hands and Objects Challenge 2022
Our method achieves an accuracy of 0. 796 on OSCC while achieving an absolute temporal localization error of 0. 516 on PNR.
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions
Compared to the great progress of large-scale vision transformers (ViTs) in recent years, large-scale models based on convolutional neural networks (CNNs) are still in an early state.
Ranked #1 on
Instance Segmentation
on COCO test-dev
(AP50 metric, using extra
training data)
A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal
Second, we discuss the challenges of face restoration.
On Efficient Reinforcement Learning for Full-length Game of StarCraft II
In this work, we investigate a set of RL techniques for the full-length game of StarCraft II.
Incremental Few-Shot Semantic Segmentation via Embedding Adaptive-Update and Hyper-class Representation
Second, to resist overfitting issues caused by few training samples, a hyper-class embedding is learned by clustering all category embeddings for initialization and aligned with category embedding of the new class for enhancement, where learned knowledge assists to learn new knowledge, thus alleviating performance dependence on training data scale.
SeedFormer: Patch Seeds based Point Cloud Completion with Upsample Transformer
Point cloud completion has become increasingly popular among generation tasks of 3D point clouds, as it is a challenging yet indispensable problem to recover the complete shape of a 3D object from its partial observation.
Ranked #7 on
Point Cloud Completion
on Completion3D
Uncertainty-based Network for Few-shot Image Classification
Specifically, we first data augment and classify the query instance and calculate the mutual information of these classification scores.
Vision Transformer Adapter for Dense Predictions
This work investigates a simple yet powerful dense prediction task adapter for Vision Transformer (ViT).
Ranked #4 on
Semantic Segmentation
on PASCAL Context
BasicTAD: an Astounding RGB-Only Baseline for Temporal Action Detection
Empirical results demonstrate that our PlusTAD is very efficient and significantly outperforms the previous methods on the datasets of THUMOS14 and FineAction.
Ranked #1 on
Temporal Action Localization
on THUMOS14
BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers
In a nutshell, BEVFormer exploits both spatial and temporal information by interacting with spatial and temporal space through predefined grid-shaped BEV queries.
Refine-Net: Normal Refinement Neural Network for Noisy Point Clouds
Differently, our network is designed to refine the initial normal of each point by extracting additional information from multiple feature representations.
DCAN: Improving Temporal Action Detection via Dual Context Aggregation
Specifically, we design the Multi-Path Temporal Context Aggregation (MTCA) to achieve smooth context aggregation on boundary level and precise evaluation of boundaries.
Ranked #19 on
Temporal Action Localization
on ActivityNet-1.3
FAST: Faster Arbitrarily-Shaped Text Detector with Minimalist Kernel Representation
We propose an accurate and efficient scene text detection framework, termed FAST (i. e., faster arbitrarily-shaped text detector).
Ranked #2 on
Scene Text Detection
on MSRA-TD500
Spectrum-to-Kernel Translation for Accurate Blind Image Super-Resolution
In this paper, we propose a novel blind SR framework to super-resolve LR images degraded by arbitrary blur kernel with accurate kernel estimation in frequency domain.
ARTS: Eliminating Inconsistency between Text Detection and Recognition with Auto-Rectification Text Spotter
Recent approaches for end-to-end text spotting have achieved promising results.
Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformers
Specifically, we supervise the attention modules in the mask decoder in a layer-wise manner.
Ranked #4 on
Panoptic Segmentation
on COCO test-dev
Learning Class-level Prototypes for Few-shot Learning
Few-shot learning aims to recognize new categories using very few labeled samples.
Adaptive Graph Convolution for Point Cloud Analysis
Convolution on 3D point clouds that generalized from 2D grid-like domains is widely researched yet far from perfect.
Ranked #10 on
3D Point Cloud Classification
on IntrA
PVT v2: Improved Baselines with Pyramid Vision Transformer
We hope this work will facilitate state-of-the-art Transformer researches in computer vision.
Ranked #23 on
Object Detection
on COCO-O
PAN++: Towards Efficient and Accurate End-to-End Spotting of Arbitrarily-Shaped Text
By systematically comparing with existing scene text representations, we show that our kernel representation can not only describe arbitrarily-shaped text but also well distinguish adjacent text.
An Introduction of mini-AlphaStar
StarCraft II (SC2) is a real-time strategy game in which players produce and control multiple units to fight against opponent's units.
Towards Ultra-Resolution Neural Style Transfer via Thumbnail Instance Normalization
(1) We divide input image into small patches and adopt TIN, successfully transferring image style with arbitrary high-resolution.
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Unlike the recently-proposed Transformer model (e. g., ViT) that is specially designed for image classification, we propose Pyramid Vision Transformer~(PVT), which overcomes the difficulties of porting Transformer to various dense prediction tasks.
Ranked #5 on
Semantic Segmentation
on SynPASS
Frequency Consistent Adaptation for Real World Super Resolution
From this point of view, we design a novel Frequency Consistent Adaptation (FCA) that ensures the frequency domain consistency when applying existing SR methods to the real scene.
AE TextSpotter: Learning Visual and Linguistic Representation for Ambiguous Text Spotting
Unlike previous works that merely employed visual features for text detection, this work proposes a novel text spotter, named Ambiguity Eliminating Text Spotter (AE TextSpotter), which learns both visual and linguistic features to significantly reduce ambiguity in text detection.
Dynamic Sampling Networks for Efficient Action Recognition in Videos
The existing action recognition methods are mainly based on clip-level classifiers such as two-stream CNNs or 3D CNNs, which are trained from the randomly selected clips and applied to densely sampled clips during testing.
Ranked #9 on
Action Recognition
on ActivityNet
Channel Relationship Prediction with Forget-Update Module for Few-shot Classification
The proposed pipeline, which consists of channel vector sequence construction module and forget-update module, can infer the relationship between the query sample and support samples in few-shot classification scenario.
A New Unified Method for Detecting Text from Marathon Runners and Sports Players in Video
The proposed method fuses gradient magnitude and direction coherence of text pixels in a new way for detecting candidate regions.
TAM: Temporal Adaptive Module for Video Recognition
Video data is with complex temporal dynamics due to various factors such as camera motion, speed variation, and different activities.
TEINet: Towards an Efficient Architecture for Video Recognition
To relieve this problem, we propose an efficient temporal module, termed as Temporal Enhancement-and-Interaction (TEI Module), which could be plugged into the existing 2D CNNs (denoted by TEINet).
Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network
Recently, some methods have been proposed to tackle arbitrary-shaped text detection, but they rarely take the speed of the entire pipeline into consideration, which may fall short in practical applications. In this paper, we propose an efficient and accurate arbitrary-shaped text detector, termed Pixel Aggregation Network (PAN), which is equipped with a low computational-cost segmentation head and a learnable post-processing.
Ranked #8 on
Scene Text Detection
on SCUT-CTW1500
Shape Robust Text Detection with Progressive Scale Expansion Network
Due to the fact that there are large geometrical margins among the minimal scale kernels, our method is effective to split the close text instances, making it easier to use segmentation-based methods to detect arbitrary-shaped text instances.
Ranked #12 on
Scene Text Detection
on SCUT-CTW1500
Efficient Reinforcement Learning for StarCraft by Abstract Forward Models and Transfer Learning
Injecting human knowledge is an effective way to accelerate reinforcement learning (RL).
On Reinforcement Learning for Full-length Game of StarCraft
The reinforcement training algorithm for this architecture is also investigated.
Hierarchical Reinforcement Learning
reinforcement-learning
+4
A New COLD Feature based Handwriting Analysis for Ethnicity/Nationality Identification
For each line segment, the proposed method estimates angle and length, which gives a point in polar domain.
Shape Robust Text Detection with Progressive Scale Expansion Network
To address these problems, we propose a novel Progressive Scale Expansion Network (PSENet), designed as a segmentation-based detector with multiple predictions for each text instance.
Ranked #12 on
Scene Text Detection
on ICDAR 2017 MLT
Mixed Link Networks
Basing on the analysis by revealing the equivalence of modern networks, we find that both ResNet and DenseNet are essentially derived from the same "dense topology", yet they only differ in the form of connection -- addition (dubbed "inner link") vs. concatenation (dubbed "outer link").
Temporal Action Localization by Structured Maximal Sums
We pose action localization as a structured prediction over arbitrary-length temporal windows, where each window is scored as the sum of frame-wise classification scores.
