Search Results for author:
Found 52 papers, 44 papers with code
EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought
In this work, we introduce EmbodiedGPT, an end-to-end multi-modal foundation model for embodied AI, empowering embodied agents with multi-modal understanding and execution capabilities.
VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks
We hope this model can set a new baseline for generalist vision and language models.
Tram: A Token-level Retrieval-augmented Mechanism for Source Code Summarization
Most previous approaches rely on the sentence-level retrieval and combination paradigm (retrieval of similar code snippets and use of the corresponding code and summary pairs) on the encoder side.
VideoChat: Chat-Centric Video Understanding
In this study, we initiate an exploration into video understanding by introducing VideoChat, an end-to-end chat-centric video understanding system.
InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Language
Different from existing interactive systems that rely on pure language, by incorporating pointing instructions, the proposed iGPT significantly improves the efficiency of communication between users and chatbots, as well as the accuracy of chatbots in vision-centric tasks, especially in complicated visual scenarios where the number of objects is greater than 2.
A Survey of Historical Learning: Learning Models with Learning History
The various types of elements, deposited in the training history, are a large amount of wealth for improving learning deep models.
Champion Solution for the WSDM2023 Toloka VQA Challenge
In this report, we present our champion solution to the WSDM2023 Toloka Visual Question Answering (VQA) Challenge.
Planning-oriented Autonomous Driving
Oriented at this, we revisit the key components within perception and prediction, and prioritize the tasks such that all these tasks contribute to planning.
VLG: General Video Recognition with Web Textual Knowledge
Our VLG is first pre-trained on video and language datasets to learn a shared feature space, and then devises a flexible bi-modal attention head to collaborate high-level semantic concepts under different settings.
Uni-Perceiver v2: A Generalist Model for Large-Scale Vision and Vision-Language Tasks
In this paper, we propose Uni-Perceiver v2, which is the first generalist model capable of handling major large-scale vision and vision-language tasks with competitive performance.
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions
Compared to the great progress of large-scale vision transformers (ViTs) in recent years, large-scale models based on convolutional neural networks (CNNs) are still in an early state.
Ranked #1 on
Instance Segmentation
on COCO minival
(using extra training data)
Demystify Transformers & Convolutions in Modern Image Deep Networks
Although the novel feature transformation designs are often claimed as the source of gain, some backbones may benefit from advanced engineering techniques, which makes it hard to identify the real gain from the key feature transformation operators.
On Efficient Reinforcement Learning for Full-length Game of StarCraft II
In this work, we investigate a set of RL techniques for the full-length game of StarCraft II.
Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe
As sensor configurations get more complex, integrating multi-source information from different sensors and representing features in a unified view come of vital importance.
Incremental Few-Shot Semantic Segmentation via Embedding Adaptive-Update and Hyper-class Representation
Second, to resist overfitting issues caused by few training samples, a hyper-class embedding is learned by clustering all category embeddings for initialization and aligned with category embedding of the new class for enhancement, where learned knowledge assists to learn new knowledge, thus alleviating performance dependence on training data scale.
Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEs
To mitigate such interference, we introduce the Conditional Mixture-of-Experts (Conditional MoEs) to generalist models.
Uniform Masking: Enabling MAE Pre-training for Pyramid-based Vision Transformers with Locality
Masked AutoEncoder (MAE) has recently led the trends of visual self-supervision area by an elegant asymmetric encoder-decoder design, which significantly optimizes both the pre-training efficiency and fine-tuning accuracy.
Ranked #33 on
Object Detection
on COCO minival
Vision Transformer Adapter for Dense Predictions
This work investigates a simple yet powerful dense prediction task adapter for Vision Transformer (ViT).
Ranked #1 on
Semantic Segmentation
on COCO-Stuff full
Hybrid Cloud-Edge Collaborative Data Anomaly Detection in Industrial Sensor Networks
The sensor data detection model based on Gaussian and Bayesian algorithms can detect the anomalous sensor data in real-time and upload them to the cloud for further analysis, filtering the normal sensor data and reducing traffic load.
BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers
In a nutshell, BEVFormer exploits both spatial and temporal information by interacting with spatial and temporal space through predefined grid-shaped BEV queries.
Ranked #6 on
3D Object Detection
on DAIR-V2X-I
WegFormer: Transformers for Weakly Supervised Semantic Segmentation
Although convolutional neural networks (CNNs) have achieved remarkable progress in weakly supervised semantic segmentation (WSSS), the effective receptive field of CNN is insufficient to capture global context information, leading to sub-optimal results.
Weakly supervised Semantic Segmentation
Weakly-Supervised Semantic Segmentation
VL-LTR: Learning Class-wise Visual-Linguistic Representation for Long-Tailed Visual Recognition
Deep learning-based models encounter challenges when processing long-tailed data in the real world.
Ranked #2 on
Long-tail Learning
on Places-LT
(using extra training data)
FAST: Faster Arbitrarily-Shaped Text Detector with Minimalist Kernel Representation
We propose an accurate and efficient scene text detection framework, termed FAST (i. e., faster arbitrarily-shaped text detector).
Ranked #1 on
Scene Text Detection
on MSRA-TD500
ARTS: Eliminating Inconsistency between Text Detection and Recognition with Auto-Rectification Text Spotter
Recent approaches for end-to-end text spotting have achieved promising results.
Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformers
Specifically, we supervise the attention modules in the mask decoder in a layer-wise manner.
Ranked #4 on
Panoptic Segmentation
on COCO test-dev
An empirical evaluation of attention-based multi-head models for improved turbofan engine remaining useful life prediction
The results presented in this study demonstrate the importance of multi-head models and attention mechanisms to an improved understanding of the remaining useful life of industrial assets.
Learning Class-level Prototypes for Few-shot Learning
Few-shot learning aims to recognize new categories using very few labeled samples.
Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers
Unlike existing CNN-based methods, we adopt a transformer encoder, which learns more powerful and robust representations.
Ranked #6 on
Medical Image Segmentation
on CVC-ColonDB
PVT v2: Improved Baselines with Pyramid Vision Transformer
We hope this work will facilitate state-of-the-art Transformer researches in computer vision.
Ranked #70 on
Object Detection
on COCO minival
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
We present SegFormer, a simple, efficient yet powerful semantic segmentation framework which unifies Transformers with lightweight multilayer perception (MLP) decoders.
Ranked #2 on
Semantic Segmentation
on SELMA
PolarMask++: Enhanced Polar Representation for Single-Shot Instance Segmentation and Beyond
Extensive experiments demonstrate the effectiveness of both PolarMask and PolarMask++, which achieve competitive results on instance segmentation in the challenging COCO dataset with single-model and single-scale training and testing, as well as new state-of-the-art results on rotate text detection and cell segmentation.
Ranked #78 on
Instance Segmentation
on COCO test-dev
(using extra training data)
PAN++: Towards Efficient and Accurate End-to-End Spotting of Arbitrarily-Shaped Text
By systematically comparing with existing scene text representations, we show that our kernel representation can not only describe arbitrarily-shaped text but also well distinguish adjacent text.
An Introduction of mini-AlphaStar
StarCraft II (SC2) is a real-time strategy game in which players produce and control multiple units to fight against opponent's units.
Towards Ultra-Resolution Neural Style Transfer via Thumbnail Instance Normalization
(1) We divide input image into small patches and adopt TIN, successfully transferring image style with arbitrary high-resolution.
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Unlike the recently-proposed Transformer model (e. g., ViT) that is specially designed for image classification, we propose Pyramid Vision Transformer~(PVT), which overcomes the difficulties of porting Transformer to various dense prediction tasks.
Ranked #5 on
Semantic Segmentation
on SynPASS
DetCo: Unsupervised Contrastive Learning for Object Detection
Unlike most recent methods that focused on improving accuracy of image classification, we present a novel contrastive learning approach, named DetCo, which fully explores the contrasts between global image and local image patches to learn discriminative representations for object detection.
Segmenting Transparent Object in the Wild with Transformer
This work presents a new fine-grained transparent object segmentation dataset, termed Trans10K-v2, extending Trans10K-v1, the first large-scale transparent object segmentation dataset.
Ranked #3 on
Semantic Segmentation
on Trans10K
Polygon-free: Unconstrained Scene Text Detection with Box Annotations
For example, without using polygon annotations, PSENet achieves an 80. 5% F-score on TotalText [3] (vs. 80. 9% of fully supervised counterpart), 31. 1% better than training directly with upright bounding box annotations, and saves 80%+ labeling costs.
Generalized Focal Loss V2: Learning Reliable Localization Quality Estimation for Dense Object Detection
Such a property makes the distribution statistics of a bounding box highly correlated to its real localization quality.
Ranked #57 on
Object Detection
on COCO test-dev
AE TextSpotter: Learning Visual and Linguistic Representation for Ambiguous Text Spotting
Unlike previous works that merely employed visual features for text detection, this work proposes a novel text spotter, named Ambiguity Eliminating Text Spotter (AE TextSpotter), which learns both visual and linguistic features to significantly reduce ambiguity in text detection.
Differentiable Hierarchical Graph Grouping for Multi-Person Pose Estimation
The modules of HGG can be trained end-to-end with the keypoint detection network and is able to supervise the grouping process in a hierarchical manner.
Ranked #3 on
Keypoint Detection
on OCHuman
Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection
Specifically, we merge the quality estimation into the class prediction vector to form a joint representation of localization quality and classification, and use a vector to represent arbitrary distribution of box locations.
Ranked #106 on
Object Detection
on COCO test-dev
Scene Text Image Super-Resolution in the Wild
For example, it outperforms LapSRN by over 5% and 8%on the recognition accuracy of ASTER and CRNN.
Segmenting Transparent Objects in the Wild
To address this important problem, this work proposes a large-scale dataset for transparent object segmentation, named Trans10K, consisting of 10, 428 images of real scenarios with carefully manual annotations, which are 10 times larger than the existing datasets.
Ranked #4 on
Semantic Segmentation
on Trans10K
False Data Injection Attacks and the Distributed Countermeasure in DC Microgrids
In this paper, we consider a hierarchical control based DC microgrid (DCmG) equipped with unknown input observer (UIO) based detectors, where the potential false data injection (FDI) attacks and the distributed countermeasure are investigated.
PolarMask: Single Shot Instance Segmentation with Polar Representation
In this paper, we introduce an anchor-box free and single shot instance segmentation method, which is conceptually simple, fully convolutional and can be used as a mask prediction module for instance segmentation, by easily embedding it into most off-the-shelf detection methods.
Ranked #97 on
Instance Segmentation
on COCO test-dev
TextSR: Content-Aware Text Super-Resolution Guided by Recognition
Nonetheless, most of the previous methods may not work well in recognizing text with low resolution which is often seen in natural scene images.
Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network
Recently, some methods have been proposed to tackle arbitrary-shaped text detection, but they rarely take the speed of the entire pipeline into consideration, which may fall short in practical applications. In this paper, we propose an efficient and accurate arbitrary-shaped text detector, termed Pixel Aggregation Network (PAN), which is equipped with a low computational-cost segmentation head and a learnable post-processing.
Ranked #6 on
Scene Text Detection
on SCUT-CTW1500
Shape Robust Text Detection with Progressive Scale Expansion Network
Due to the fact that there are large geometrical margins among the minimal scale kernels, our method is effective to split the close text instances, making it easier to use segmentation-based methods to detect arbitrary-shaped text instances.
Ranked #10 on
Scene Text Detection
on SCUT-CTW1500
Selective Kernel Networks
A building block called Selective Kernel (SK) unit is designed, in which multiple branches with different kernel sizes are fused using softmax attention that is guided by the information in these branches.
Ranked #95 on
Image Classification
on CIFAR-100
(using extra training data)
Shape Robust Text Detection with Progressive Scale Expansion Network
To address these problems, we propose a novel Progressive Scale Expansion Network (PSENet), designed as a segmentation-based detector with multiple predictions for each text instance.
Ranked #12 on
Scene Text Detection
on ICDAR 2017 MLT
Mixed Link Networks
Basing on the analysis by revealing the equivalence of modern networks, we find that both ResNet and DenseNet are essentially derived from the same "dense topology", yet they only differ in the form of connection -- addition (dubbed "inner link") vs. concatenation (dubbed "outer link").
