Search Results for author:
Found 124 papers, 93 papers with code
VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models
Visual reasoning is a core component of human intelligence and a critical capability for advanced multimodal models.
InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models
We introduce InternVL3, a significant advancement in the InternVL series featuring a native multimodal pre-training paradigm.
Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy
While recent vision-language-action models trained on diverse robot datasets exhibit promising generalization capabilities with limited in-domain data, their reliance on compact action heads to predict discretized or continuous actions constrains adaptability to heterogeneous action spaces.
Ranked #3 on
Robot Manipulation
on SimplerEnv-Google Robot
(using extra training data)
LangBridge: Interpreting Image as a Combination of Language Embeddings
Based on this insight, we propose LangBridge, a novel adapter that explicitly maps visual tokens to linear combinations of LLM vocabulary embeddings.
GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing
We present Generation Chain-of-Thought (GoT), a novel paradigm that enables generation and editing through an explicit language reasoning process before outputting images.
VisualPRM: An Effective Process Reward Model for Multimodal Reasoning
We introduce VisualPRM, an advanced multimodal Process Reward Model (PRM) with 8B parameters, which improves the reasoning abilities of existing Multimodal Large Language Models (MLLMs) across different model scales and families with Best-of-N (BoN) evaluation strategies.
MI-DETR: An Object Detection Model with Multi-time Inquiries Mechanism
Based on analyzing the character of cascaded decoder architecture commonly adopted in existing DETR-like models, this paper proposes a new decoder architecture.
Ranked #3 on
Object Detection
on COCO 2017 val
Parameter-Inverted Image Pyramid Networks for Visual Perception and Multimodal Understanding
For multimodal understanding, our PIIP-LLaVA achieves 73. 0% accuracy on TextVQA and 74. 5% on MMBench with only 2. 8M training data.
Ranked #84 on
Visual Question Answering
on MM-Vet
HoVLE: Unleashing the Power of Monolithic Vision-Language Models with Holistic Vision-Language Embedding
The challenge for current monolithic VLMs actually lies in the lack of a holistic embedding module for both vision and language inputs.
V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encoding
In our work, we first conduct an empirical analysis of the long-context capabilities of VLMs using our augmented long-context multimodal datasets.
SynerGen-VL: Towards Synergistic Image Understanding and Generation with Vision Experts and Token Folding
The remarkable success of Large Language Models (LLMs) has extended to the multimodal domain, achieving outstanding performance in image understanding and generation.
PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Models
To this end, we extend each image into a "static" video and introduce a unified token compression strategy called Progressive Visual Token Compression (PVC), where the tokens of each frame are progressively encoded and adaptively compressed to supplement the information not extracted from previous frames.
Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
We introduce InternVL 2. 5, an advanced multimodal large language model (MLLM) series that builds upon InternVL 2. 0, maintaining its core model architecture while introducing significant enhancements in training and testing strategies as well as data quality.
Ranked #1 on
Video Question Answering
on NExT-QA
HoloDrive: Holistic 2D-3D Multi-Modal Street Scene Generation for Autonomous Driving
Generative models have significantly improved the generation and prediction quality on either camera images or LiDAR point clouds for autonomous driving.
MuLan: Adapting Multilingual Diffusion Models for Hundreds of Languages with Negligible Cost
In this work, we explore a cost-effective framework for multilingual image generation.
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization
To address this, we introduce a preference optimization (PO) process to enhance the multimodal reasoning capabilities of MLLMs.
DI-MaskDINO: A Joint Object Detection and Instance Segmentation Model
This paper is motivated by an interesting phenomenon: the performance of object detection lags behind that of instance segmentation (i. e., performance imbalance) when investigating the intermediate results from the beginning transformer decoder layer of MaskDINO (i. e., the SOTA model for joint detection and segmentation).
Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance
Multimodal large language models (MLLMs) have demonstrated impressive performance in vision-language tasks across a broad spectrum of domains.
Diffusion Transformer Policy
In contrast, we model the continuous action sequence with a large multi-modal diffusion transformer, dubbed as Diffusion Transformer Policy, in which we directly denoise action chunks by a large transformer model rather than a small action head for action embedding.
PUMA: Empowering Unified MLLM with Multi-granular Visual Generation
This work represents a significant step towards a truly unified MLLM capable of adapting to the granularity demands of various visual tasks.
big.LITTLE Vision Transformer for Efficient Visual Recognition
In this paper, we introduce the big. LITTLE Vision Transformer, an innovative architecture aimed at achieving efficient visual recognition.
Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-training
In particular, EViP is designed as a progressive learning process for visual experts, which aims to fully exploit the visual knowledge from noisy data to high-quality data.
Ranked #111 on
Visual Question Answering
on MM-Vet
MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models
To fill this gap, we introduce the Multimodal Multi-image Understanding (MMIU) benchmark, a comprehensive evaluation suite designed to assess LVLMs across a wide range of multi-image tasks.
MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversity
To construct MMInstruct, we propose an instruction generation data engine that leverages GPT-4V, GPT-3. 5, and manual correction.
Ranked #126 on
Visual Question Answering
on MM-Vet
InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output
This long-context capability allows IXC-2. 5 to excel in tasks requiring extensive input and output contexts.
Ranked #4 on
Video Question Answering
on TVBench
Hierarchical Memory for Long Video QA
This paper describes our champion solution to the LOVEU Challenge @ CVPR'24, Track 1 (Long Video VQA).
CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamics
Given the scarcity of motion capture data on multi-humanoid collaboration and the efficiency challenges associated with multi-agent learning, these tasks cannot be straightforwardly addressed using training paradigms designed for single-agent scenarios.
OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text
In this paper, we introduce OmniCorpus, a 10 billion-scale image-text interleaved dataset.
Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams
Our model is able to process extremely long video streams in real-time and respond to user queries simultaneously.
Ranked #1 on
Zero-Shot Video Question Answer
on MSRVTT-QA
VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks
It not only allows flexible transmission of task information and gradient feedback between the MLLM and multiple downstream decoders but also effectively resolves training conflicts in multi-tasking scenarios.
Vision Model Pre-training on Interleaved Image-Text Data via Latent Compression Learning
Recently, vision model pre-training has evolved from relying on manually annotated datasets to leveraging large-scale, web-crawled image-text data.
Needle In A Multimodal Haystack
In this work, we present Needle In A Multimodal Haystack (MM-NIAH), the first benchmark specifically designed to systematically evaluate the capability of existing MLLMs to comprehend long multimodal documents.
Learning 1D Causal Visual Representation with De-focus Attention Networks
The issue of "over-focus" hinders the model's ability to extract diverse visual features and to receive effective gradients for optimization.
Parameter-Inverted Image Pyramid Networks
Our core idea is to use models with different parameter sizes to process different resolution levels of the image pyramid, thereby balancing computational efficiency and performance.
Ranked #12 on
Semantic Segmentation
on ADE20K
LLMs Meet Multimodal Generation and Editing: A Survey
With the recent advancement in large language models (LLMs), there is a growing interest in combining LLMs with multimodal learning.
FLoRA: Low-Rank Core Space for N-dimension
To tackle the diversity of dimensional spaces across different foundation models and provide a more precise representation of the changes within these spaces, this paper introduces a generalized parameter-efficient fine-tuning framework, FLoRA, designed for various dimensional parameter space.
How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites
Compared to both open-source and proprietary models, InternVL 1. 5 shows competitive performance, achieving state-of-the-art results in 8 of 18 benchmarks.
Ranked #6 on
Multiple-choice
on Neptune-Full
InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD
The Large Vision-Language Model (LVLM) field has seen significant advancements, yet its progression has been hindered by challenges in comprehending fine-grained visual content due to limited resolution.
Ranked #47 on
Visual Question Answering
on MM-Vet
Bounding Box Stability against Feature Dropout Reflects Detector Generalization across Environments
However, in the real-world where test ground truths are not provided, it is non-trivial to find out whether bounding boxes are accurate, thus preventing us from assessing the detector generalization ability.
Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures
Our evaluations demonstrate that VRWKV surpasses ViT's performance in image classification and has significantly faster speeds and lower memory usage processing high-resolution inputs.
The All-Seeing Project V2: Towards General Relation Comprehension of the Open World
In addition, we design a new benchmark, termed Circular-based Relation Probing Evaluation (CRPE) for comprehensively evaluating the relation comprehension capabilities of MLLMs.
Ranked #103 on
Visual Question Answering
on MM-Vet
RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis
Robotic behavior synthesis, the problem of understanding multimodal inputs and generating precise physical control for robots, is an important part of Embodied AI.
Ranked #198 on
Visual Question Answering
on MM-Vet
Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modeling
For the first stage, we propose a diffusion-based motion field predictor, which focuses on deducing the trajectories of the reference image's pixels.
MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizer
Developing generative models for interleaved image-text data has both research and practical value.
Efficient Deformable ConvNets: Rethinking Dynamic and Sparse Operator for Vision Applications
The advancements in speed and efficiency of DCNv4, combined with its robust performance across diverse vision tasks, show its potential as a foundational building block for future vision models.
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
However, the progress in vision and vision-language foundation models, which are also critical elements of multi-modal AGI, has not kept pace with LLMs.
Ranked #1 on
Zero-Shot Video Retrieval
on MSR-VTT-full
(using extra training data)
A Survey of Reasoning with Foundation Models
Reasoning, a crucial ability for complex problem-solving, plays a pivotal role in various real-world settings such as negotiation, medical diagnosis, and criminal investigation.
DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving
In this work, we delve into the potential of large language models (LLMs) in autonomous driving (AD).
Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft
Many reinforcement learning environments (e. g., Minecraft) provide only sparse rewards that indicate task completion or failure with binary values.
InstructSeq: Unifying Vision Tasks with Instruction-conditioned Multi-modal Sequence Generation
In this work, we introduce InstructSeq, an instruction-conditioned multi-modal modeling framework that unifies diverse vision tasks through flexible natural language control and handling of both visual and textual data.
Point2RBox: Combine Knowledge from Synthetic Visual Patterns for End-to-end Oriented Object Detection with Single Point Supervision
To our best knowledge, Point2RBox is the first end-to-end solution for point-supervised OOD.
ControlLLM: Augment Language Models with Tools by Searching on Graphs
We present ControlLLM, a novel framework that enables large language models (LLMs) to utilize multi-modal tools for solving complex real-world tasks.
Mini-DALLE3: Interactive Text to Image by Prompting Large Language Models
The revolution of artificial intelligence content generation has been rapidly accelerated with the booming text-to-image (T2I) diffusion models.
The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open World
We present the All-Seeing (AS) project: a large-scale data and model for recognizing and understanding everything in the open world.
ADDP: Learning General Representations for Image Recognition and Generation with Alternating Denoising Diffusion Process
In each denoising step, our method first decodes pixels from previous VQ tokens, then generates new VQ tokens from the decoded pixels.
FlowFormer: A Transformer Architecture and Its Masked Cost Volume Autoencoding for Optical Flow
This paper introduces a novel transformer-based network architecture, FlowFormer, along with the Masked Cost Volume AutoEncoding (MCVA) for pretraining it to tackle the problem of optical flow estimation.
Denoising Diffusion Semantic Segmentation with Mask Prior Modeling
In this paper, we propose to ameliorate the semantic segmentation quality of existing discriminative approaches with a mask prior modeled by a recently-developed denoising diffusion generative model.
Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory
These agents, equipped with the logic and common sense capabilities of LLMs, can skillfully navigate complex, sparse-reward environments with text-based interactions.
EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought
In this work, we introduce EmbodiedGPT, an end-to-end multi-modal foundation model for embodied AI, empowering embodied agents with multi-modal understanding and execution capabilities.
VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks
We hope this model can set a new baseline for generalist vision and language models.
InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Language
Different from existing interactive systems that rely on pure language, by incorporating pointing instructions, the proposed iGPT significantly improves the efficiency of communication between users and chatbots, as well as the accuracy of chatbots in vision-centric tasks, especially in complicated visual scenarios where the number of objects is greater than 2.
Video Dehazing via a Multi-Range Temporal Alignment Network with Physical Prior
Video dehazing aims to recover haze-free frames with high visibility and contrast.
VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimation
We first propose a TRi-frame Optical Flow (TROF) module that estimates bi-directional optical flows for the center frame in a three-frame manner.
FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation
FlowFormer introduces a transformer architecture into optical flow estimation and achieves state-of-the-art performance.
FeatAug-DETR: Enriching One-to-Many Matching for DETRs with Feature Augmentation
The first method is One-to-many Matching via Data Augmentation (denoted as DataAug-DETR).
Learning Weather-General and Weather-Specific Features for Image Restoration Under Multiple Adverse Weather Conditions
Inspired by this observation, we design an efficient unified framework with a two-stage training strategy to explore the weather-general and weather-specific features.
Ranked #4 on
Image Restoration
on CDD-11
Planning-oriented Autonomous Driving
Oriented at this, we revisit the key components within perception and prediction, and prioritize the tasks such that all these tasks contribute to planning.
Ranked #3 on
Trajectory Planning
on nuScenes
BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision
The proposed method is verified with a wide spectrum of traditional and modern image backbones and achieves new SoTA results on the large-scale nuScenes dataset.
Ranked #5 on
3D Object Detection
on Rope3D
Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information
It has been proved that combining multiple pre-training strategies and data from various modalities/sources can greatly boost the training of large-scale models.
Ranked #2 on
Object Detection
on COCO test-dev
Uni-Perceiver v2: A Generalist Model for Large-Scale Vision and Vision-Language Tasks
In this paper, we propose Uni-Perceiver v2, which is the first generalist model capable of handling major large-scale vision and vision-language tasks with competitive performance.
Demystify Transformers & Convolutions in Modern Image Deep Networks
Vision transformers have gained popularity recently, leading to the development of new vision backbones with improved features and consistent performance gains.
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions
Compared to the great progress of large-scale vision transformers (ViTs) in recent years, large-scale models based on convolutional neural networks (CNNs) are still in an early state.
Ranked #1 on
Instance Segmentation
on COCO test-dev
(AP50 metric, using extra
training data)
Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe
As sensor configurations get more complex, integrating multi-source information from different sensors and representing features in a unified view come of vital importance.
Frozen CLIP Models are Efficient Video Learners
Video recognition has been dominated by the end-to-end learning paradigm -- first initializing a video recognition model with weights of a pretrained image model and then conducting end-to-end training on videos.
Ranked #31 on
Action Classification
on Kinetics-400
Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification
On top of that, the performance of Tip-Adapter can be further boosted to be state-of-the-art on ImageNet by fine-tuning the cache model for 10$\times$ fewer epochs than existing methods, which is both effective and efficient.
Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEs
To mitigate such interference, we introduce the Conditional Mixture-of-Experts (Conditional MoEs) to generalist models.
Siamese Image Modeling for Self-Supervised Vision Representation Learning
Driven by these analysis, we propose Siamese Image Modeling (SiameseIM), which predicts the dense representations of an augmented view, based on another masked view from the same image but with different augmentations.
Vision Transformer Adapter for Dense Predictions
This work investigates a simple yet powerful dense prediction task adapter for Vision Transformer (ViT).
Ranked #5 on
Semantic Segmentation
on PASCAL Context
ConvMAE: Masked Convolution Meets Masked Autoencoders
Masked auto-encoding for feature pretraining and multi-scale hybrid convolution-transformer architectures can further unleash the potentials of ViT, leading to state-of-the-art performances on image classification, detection and semantic segmentation.
BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers
In a nutshell, BEVFormer exploits both spatial and temporal information by interacting with spatial and temporal space through predefined grid-shaped BEV queries.
FlowFormer: A Transformer Architecture for Optical Flow
We introduce optical Flow transFormer, dubbed as FlowFormer, a transformer-based neural network architecture for learning optical flow.
Ranked #4 on
Optical Flow Estimation
on KITTI 2015 (train)
Searching Parameterized AP Loss for Object Detection
In this paper, we propose Parameterized AP Loss, where parameterized functions are introduced to substitute the non-differentiable components in the AP calculation.
Exploring the Equivalence of Siamese Self-Supervised Learning via A Unified Gradient Framework
These methods appear to be quite different in the designed loss functions from various motivations.
Uni-Perceiver: Pre-training Unified Architecture for Generic Perception for Zero-shot and Few-shot Tasks
The model is pre-trained on several uni-modal and multi-modal tasks, and evaluated on a variety of downstream tasks, including novel tasks that did not appear in the pre-training stage.
VL-LTR: Learning Class-wise Visual-Linguistic Representation for Long-Tailed Visual Recognition
Deep learning-based models encounter challenges when processing long-tailed data in the real world.
Ranked #4 on
Long-tail Learning
on iNaturalist 2018
(using extra training data)
Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling
To further enhance CLIP's few-shot capability, CLIP-Adapter proposed to fine-tune a lightweight residual feature adapter and significantly improves the performance for few-shot classification.
FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting
On the contrary, the soft composition operates by stitching different patches into a whole feature map where pixels in overlapping regions are summed up.
Ranked #3 on
Video Inpainting
on DAVIS
Influence Selection for Active Learning
To obtain the Influence of the unlabeled sample in the active learning scenario, we design the Untrained Unlabeled sample Influence Calculation(UUIC) to estimate the unlabeled sample's expected gradient with which we calculate its Influence.
Fast Convergence of DETR with Spatially Modulated Co-Attention
However, DETR suffers from its slow convergence.
Collaborative Visual Navigation
As a fundamental problem for Artificial Intelligence, multi-agent system (MAS) is making rapid progress, mainly driven by multi-agent reinforcement learning (MARL) techniques.
Scalable Transformers for Neural Machine Translation
In this paper, we propose a novel Scalable Transformers, which naturally contains sub-Transformers of different scales and have shared parameters.
Decoupled Spatial-Temporal Transformer for Video Inpainting
Seamless combination of these two novel designs forms a better spatial-temporal attention scheme and our proposed model achieves better performance than state-of-the-art video inpainting approaches with significant boosted efficiency.
AutoLoss-Zero: Searching Loss Functions from Scratch for Generic Tasks
However, the automatic design of loss functions for generic tasks with various evaluation metrics remains under-investigated.
Exploring Cross-Image Pixel Contrast for Semantic Segmentation
Inspired by the recent advance in unsupervised contrastive representation learning, we propose a pixel-wise contrastive framework for semantic segmentation in the fully supervised setting.
Fast Convergence of DETR with Spatially Modulated Co-Attention
The recently proposed Detection Transformer (DETR) model successfully applies Transformer to objects detection and achieves comparable performance with two-stage object detection frameworks, such as Faster-RCNN.
Unsupervised Object Detection with LiDAR Clues
We further identify another major issue, seldom noticed by the community, that the long-tailed and open-ended (sub-)category distribution should be accommodated.
Auto Seg-Loss: Searching Metric Surrogates for Semantic Segmentation
In this paper, we propose to automate the design of metric-specific loss functions by searching differentiable surrogate losses for each metric.
Deformable DETR: Deformable Transformers for End-to-End Object Detection
DETR has been recently proposed to eliminate the need for many hand-designed components in object detection while demonstrating good performance.
Ranked #7 on
2D Object Detection
on SARDet-100K
1st Place Solution of LVIS Challenge 2020: A Good Box is not a Guarantee of a Good Mask
This article introduces the solutions of the team lvisTraveler for LVIS Challenge 2020.
Ranked #1 on
Instance Segmentation
on LVIS v1.0 test-dev
Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation
Moreover, our approach ranked 1st place in the Weakly-Supervised Semantic Segmentation Track of CVPR2020 Learning from Imperfect Data Challenge.
Resolution Adaptive Networks for Efficient Inference
Adaptive inference is an effective mechanism to achieve a dynamic tradeoff between accuracy and computational cost in deep networks.
Hierarchical Human Parsing with Typed Part-Relation Reasoning
As human bodies are underlying hierarchically structured, how to model human structures is the central theme in this task.
Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation
This is typically done by augmenting static operators with learned free-form sampling grids in the image space, dynamically tuned to the data and task for adapting the receptive field.
Ranked #186 on
Object Detection
on COCO test-dev
VL-BERT: Pre-training of Generic Visual-Linguistic Representations
We introduce a new pre-trainable generic representation for visual-linguistic tasks, called Visual-Linguistic BERT (VL-BERT for short).
Ranked #1 on
Visual Question Answering (VQA)
on VCR (Q-A) dev
MMDetection: Open MMLab Detection Toolbox and Benchmark
In this paper, we introduce the various features of this toolbox.
An Empirical Study of Spatial Attention Mechanisms in Deep Networks
Attention mechanisms have become a popular component in deep neural networks, yet there has been little examination of how different influencing factors and methods for computing attention from these factors affect performance.
Deformable ConvNets v2: More Deformable, Better Results
The superior performance of Deformable Convolutional Networks arises from its ability to adapt to the geometric variations of objects.
Ranked #130 on
Object Detection
on COCO test-dev
Integrated Object Detection and Tracking with Tracklet-Conditioned Detection
Accurate detection and tracking of objects is vital for effective video understanding.
Ranked #22 on
Video Object Detection
on ImageNet VID
Towards High Performance Video Object Detection for Mobiles
In this paper, we present a light weight network architecture for video object detection on mobiles.
Learning Region Features for Object Detection
While most steps in the modern object detection methods are learnable, the region feature extraction step remains largely hand-crafted, featured by RoI pooling methods.
Relation Networks for Object Detection
Although it is well believed for years that modeling relations between objects would help object recognition, there has not been evidence that the idea is working in the deep learning era.
Towards High Performance Video Object Detection
There has been significant progresses for image object detection in recent years.
Flow-Guided Feature Aggregation for Video Object Detection
The accuracy of detection suffers from degenerated object appearances in videos, e. g., motion blur, video defocus, rare poses, etc.
Ranked #28 on
Video Object Detection
on ImageNet VID
Deformable Convolutional Networks
Convolutional neural networks (CNNs) are inherently limited to model geometric transformations due to the fixed geometric structures in its building modules.
Ranked #3 on
Vessel Detection
on Vessel detection Dateset
Fully Convolutional Instance-aware Semantic Segmentation
It inherits all the merits of FCNs for semantic segmentation and instance mask proposal.
Ranked #100 on
Instance Segmentation
on COCO test-dev
Deep Feature Flow for Video Recognition
Yet, it is non-trivial to transfer the state-of-the-art image recognition networks to videos as per-frame evaluation is too slow and unaffordable.
Ranked #9 on
Video Semantic Segmentation
on Cityscapes val
R-FCN: Object Detection via Region-based Fully Convolutional Networks
In contrast to previous region-based detectors such as Fast/Faster R-CNN that apply a costly per-region subnetwork hundreds of times, our region-based detector is fully convolutional with almost all computation shared on the entire image.
Ranked #4 on
Real-Time Object Detection
on PASCAL VOC 2007
ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation
Large-scale data is of crucial importance for learning semantic segmentation models, but annotating per-pixel masks is a tedious and inefficient procedure.
Instance-sensitive Fully Convolutional Networks
In contrast to the previous FCN that generates one score map, our FCN is designed to compute a small set of instance-sensitive score maps, each of which is the outcome of a pixel-wise classifier of a relative position to instances.
Instance-aware Semantic Segmentation via Multi-task Network Cascades
We develop an algorithm for the nontrivial end-to-end training of this causal, cascaded structure.
Ranked #3 on
Multi-Human Parsing
on PASCAL-Part
BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
Recent leading approaches to semantic segmentation rely on deep convolutional networks trained with human-annotated, pixel-level segmentation masks.
Ranked #46 on
Semantic Segmentation
on PASCAL VOC 2012 test
Generative Modeling of Convolutional Neural Networks
(2) We propose a generative gradient for pre-training CNNs by a non-parametric importance sampling scheme, which is fundamentally different from the commonly used discriminative gradient, and yet has the same computational architecture and cost as the latter.
Convolutional Feature Masking for Joint Object and Stuff Segmentation
The current leading approaches for semantic segmentation exploit shape information by extracting CNN features from masked image regions.
Ranked #64 on
Semantic Segmentation
on PASCAL Context
Unsupervised Learning of Dictionaries of Hierarchical Compositional Models
Given a set of unannotated training images, a dictionary of such hierarchical templates are learned so that each training image can be represented by a small number of templates that are spatially translated, rotated and scaled versions of the templates in the learned dictionary.
