Search Results for author:
Found 62 papers, 40 papers with code
ChatSearch: a Dataset and a Generative Retrieval Model for General Conversational Image Retrieval
In this paper, we investigate the task of general conversational image retrieval on open-domain images.
HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modeling
Large Language Models (LLMs) have achieved remarkable success in various fields, prompting several studies to explore their potential in recommendation systems.
OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generation
To exploit the complementary nature of image and video data, we further propose a progressive training strategy, where OmniTokenizer is first trained on image data on a fixed resolution to develop the spatial encoding capacity and then jointly trained on image and video data on multiple resolutions to learn the temporal dynamics.
Ranked #10 on
Video Prediction
on Kinetics-600 12 frames, 64x64
Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation
(3) A text-conditional image generation model with 775M parameters, from two-stage training on LAION-COCO and high aesthetics quality images, demonstrating competitive performance of visual quality and text alignment.
Ranked #27 on
Image Generation
on ImageNet 256x256
Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Models
We introduce Groma, a Multimodal Large Language Model (MLLM) with grounded and fine-grained visual perception ability.
Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction
We present Visual AutoRegressive modeling (VAR), a new generation paradigm that redefines the autoregressive learning on images as coarse-to-fine "next-scale prediction" or "next-resolution prediction", diverging from the standard raster-scan "next-token prediction".
Ranked #15 on
Image Generation
on ImageNet 256x256
Generative Region-Language Pretraining for Open-Ended Object Detection
To address it, we formulate object detection as a generative problem and propose a simple framework named GenerateU, which can detect dense objects and generate their names in a free-form way.
UniRef++: Segment Every Reference Object in Spatial and Temporal Spaces
We evaluate our unified models on various benchmarks.
General Object Foundation Model for Images and Videos at Scale
We present GLEE in this work, an object-level foundation model for locating and identifying objects in images and videos.
Ranked #1 on
Referring Expression Segmentation
on RefCOCO
(using extra training data)
Long-tail Video Object Segmentation
Multi-Object Tracking
+8
Optimization Efficient Open-World Visual Region Recognition
Building on the success of powerful image-level vision-language (ViL) foundation models like CLIP, recent efforts have sought to harness their capabilities by either training a contrastive model from scratch with an extensive collection of region-label pairs or aligning the outputs of a detection model with image-level representations of region proposals.
CoDet: Co-Occurrence Guided Region-Word Alignment for Open-Vocabulary Object Detection
CoDet then leverages visual similarities to discover the co-occurring objects and align them with the shared concept.
Ranked #4 on
Open Vocabulary Object Detection
on LVIS v1.0
(using extra training data)
EVE: Efficient Vision-Language Pre-training with Masked Prediction and Modality-Aware MoE
Owing to the combination of the unified architecture and pre-training task, EVE is easy to scale up, enabling better downstream performance with fewer resources and faster training speed.
Exploring Transformers for Open-world Instance Segmentation
Open-world instance segmentation is a rising task, which aims to segment all objects in the image by learning from a limited number of base-category objects.
ChatBridge: Bridging Modalities with Large Language Model as a Language Catalyst
We show that only language-paired two-modality data is sufficient to connect all modalities.
Meta Compositional Referring Expression Segmentation
Then, following a novel meta optimization scheme to optimize the model to obtain good testing performance on the virtual testing sets after training on the virtual training set, our framework can effectively drive the model to better capture semantics and visual representations of individual concepts, and thus obtain robust generalization performance even when handling novel compositions.
Token Boosting for Robust Self-Supervised Visual Transformer Pre-training
Pre-training VTs on such corrupted data can be challenging, especially when we pre-train via the masked autoencoding approach, where both the inputs and masked ``ground truth" targets can potentially be unreliable in this case.
Learning Instance-Level Representation for Large-Scale Multi-Modal Pretraining in E-commerce
Extensive experimental results show that, without further fine-tuning, ECLIP surpasses existing methods by a large margin on a broad range of downstream tasks, demonstrating the strong transferability to real-world E-commerce applications.
Multi-Level Contrastive Learning for Dense Prediction Task
We extend our pretext task to supervised pre-training, which achieves a similar performance to self-supervised learning.
EGC: Image Generation and Classification via a Diffusion Energy-Based Model
Learning image classification and image generation using the same set of network parameters is a challenging problem.
Universal Instance Perception as Object Discovery and Retrieval
All instance perception tasks aim at finding certain objects specified by some queries such as category names, language expressions, and target annotations, but this complete field has been split into multiple independent subtasks.
Described Object Detection
Generalized Referring Expression Comprehension
+15
Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling
This is the first use of sparse convolution for 2D masked modeling.
Ranked #1 on
Instance Segmentation
on COCO 2017 val
Segment Every Reference Object in Spatial and Temporal Spaces
In this work, we end the current fragmented situation and propose UniRef to unify the three reference-based object segmentation tasks with a single architecture.
QueryPose: Sparse Multi-Person Pose Regression via Spatial-Aware Part-Level Query
The existing end-to-end methods rely on dense representations to preserve the spatial detail and structure for precise keypoint localization.
Learning Object-Language Alignments for Open-Vocabulary Object Detection
In this paper, we propose a novel open-vocabulary object detection framework directly learning from image-text pair data.
DMRNet++: Learning Discriminative Features with Decoupled Networks and Enriched Pairs for One-Step Person Search
Person search aims at localizing and recognizing query persons from raw video frames, which is a combination of two sub-tasks, i. e., pedestrian detection and person re-identification.
Ranked #3 on
Person Search
on PRW
Self-supervised Video Representation Learning with Motion-Aware Masked Autoencoders
As a result, our model can extract effectively both static appearance and dynamic motion spontaneously, leading to superior spatiotemporal representation learning capability.
MAMO: Masked Multimodal Modeling for Fine-Grained Vision-Language Representation Learning
Our method performs joint masking on image-text input and integrates both implicit and explicit targets for the masked signals to recover.
Embracing Consistency: A One-Stage Approach for Spatio-Temporal Video Grounding
Spatio-Temporal video grounding (STVG) focuses on retrieving the spatio-temporal tube of a specific object depicted by a free-form textual expression.
Rethinking Resolution in the Context of Efficient Video Recognition
Our key finding is that the major cause of degradation is not information loss in the down-sampling process, but rather the mismatch between network architecture and input scale.
MCIBI++: Soft Mining Contextual Information Beyond Image for Semantic Segmentation
To this end, we propose a novel soft mining contextual information beyond image paradigm named MCIBI++ to further boost the pixel-level representations.
Single-Stage Open-world Instance Segmentation with Cross-task Consistency Regularization
Based on the single-stage instance segmentation framework, we propose a regularization model to predict foreground pixels and use its relation to instance segmentation to construct a cross-task consistency loss.
You Should Look at All Objects
Feature pyramid network (FPN) is one of the key components for object detectors.
Towards Grand Unification of Object Tracking
We present a unified method, termed Unicorn, that can simultaneously solve four tracking problems (SOT, MOT, VOS, MOTS) with a single network using the same model parameters.
Multi-Object Tracking
Multi-Object Tracking and Segmentation
+3
Masked Generative Distillation
The current distillation algorithm usually improves students' performance by imitating the output of the teacher.
Birds of A Feather Flock Together: Category-Divergence Guidance for Domain Adaptive Segmentation
In two typical cross-domain semantic segmentation tasks, i. e., GTA5 to Cityscapes and SYNTHIA to Cityscapes, our method achieves the state-of-the-art segmentation accuracy.
MetaFormer: A Unified Meta Framework for Fine-Grained Recognition
Fine-Grained Visual Classification(FGVC) is the task that requires recognizing the objects belonging to multiple subordinate categories of a super-category.
Ranked #1 on
Fine-Grained Image Classification
on CUB-200-2011
Content-Variant Reference Image Quality Assessment via Knowledge Distillation
The comparisons of distribution differences between HQ and LQ images can help our model better assess the image quality.
Language as Queries for Referring Video Object Segmentation
Referring video object segmentation (R-VOS) is an emerging cross-modal task that aims to segment the target object referred by a language expression in all video frames.
Ranked #3 on
Referring Expression Segmentation
on A2D Sentences
(using extra training data)
Disentangled Contrastive Learning on Graphs
Then we propose a novel factor-wise discrimination objective in a contrastive learning manner, which can force the factorized representations to independently reflect the expressive information from different latent factors.
Trimap-guided Feature Mining and Fusion Network for Natural Image Matting
For emerging content-based feature fusion, most existing matting methods only focus on local features which lack the guidance of a global feature with strong semantic information related to the interesting object.
Ranked #4 on
Image Matting
on Composition-1K
DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motion
A typical pipeline for multi-object tracking (MOT) is to use a detector for object localization, and following re-identification (re-ID) for object association.
Focal and Global Knowledge Distillation for Detectors
Global distillation rebuilds the relation between different pixels and transfers it from teachers to students, compensating for missing global information in focal distillation.
Ranked #1 on
Knowledge Distillation
on MS COCO
Multimodal Transformer with Variable-length Memory for Vision-and-Language Navigation
Vision-and-Language Navigation (VLN) is a task that an agent is required to follow a language instruction to navigate to the goal position, which relies on the ongoing interactions with the environment during moving.
ByteTrack: Multi-Object Tracking by Associating Every Detection Box
ByteTrack also achieves state-of-the-art performance on MOT20, HiEve and BDD100K tracking benchmarks.
Ranked #1 on
Multiple Object Tracking
on BDD100K val
Objects in Semantic Topology
A more realistic object detection paradigm, Open-World Object Detection, has arisen increasing research interests in the community recently.
Weakly Supervised Person Search with Region Siamese Networks
Large-scale labeled training data is often difficult to collect, especially for person identities.
Memory Based Video Scene Parsing
Video scene parsing is a long-standing challenging task in computer vision, aiming to assign pre-defined semantic labels to pixels of all frames in a given video.
Center Prediction Loss for Re-identification
In this paper, we propose a new loss based on center predictivity, that is, a sample must be positioned in a location of the feature space such that from it we can roughly predict the location of the center of same-class samples.
Conditional Hyper-Network for Blind Super-Resolution with Multiple Degradations
We extract degradation prior at task-level with the proposed ConditionNet, which will be used to adapt the parameters of the basic SR network (BaseNet).
Domain-Invariant Disentangled Network for Generalizable Object Detection
By disentangling representations on both image and instance levels, DIDN is able to learn domain-invariant representations that are suitable for generalized object detection.
Unsupervised Real-World Super-Resolution: A Domain Adaptation Perspective
A popular attempts towards the challenge is unpaired generative adversarial networks, which generate "real" LR counterparts from real HR images using image-to-image translation and then perform super-resolution from "real" LR->SR.
Exploring Balanced Feature Spaces for Representation Learning
Motivated by this question, we conduct a series of studies on the performance of self-supervised contrastive learning and supervised learning methods over multiple datasets where training instance distributions vary from a balanced one to a long-tailed one.
Ranked #40 on
Long-tail Learning
on CIFAR-10-LT (ρ=10)
TransTrack: Multiple Object Tracking with Transformer
In this work, we propose TransTrack, a simple but efficient scheme to solve the multiple object tracking problems.
Ranked #12 on
Multi-Object Tracking
on SportsMOT
(using extra training data)
Multi-Object Tracking
Multiple Object Tracking with Transformer
+3
What Makes for End-to-End Object Detection?
We identify that classification cost in matching cost is the main ingredient: (1) previous detectors only consider location cost, (2) by additionally introducing classification cost, previous detectors immediately produce one-to-one prediction during inference.
Slimmable Generative Adversarial Networks
In particular, for real-time generation tasks, different devices require generators of different sizes due to varying computing power.
Sparse R-CNN: End-to-End Object Detection with Learnable Proposals
In our method, however, a fixed sparse set of learned object proposals, total length of $N$, are provided to object recognition head to perform classification and location.
Ranked #5 on
2D Object Detection
on CeyMo
Controllable Orthogonalization in Training DNNs
Orthogonality is widely used for training deep neural networks (DNNs) due to its ability to maintain all singular values of the Jacobian close to 1 and reduce redundancy in representation.
Towards Good Practices for Instance Segmentation
Instance Segmentation is an interesting yet challenging task in computer vision.
Deformable Tube Network for Action Detection in Videos
To perform action detection, we design a 3D convolution network with skip connections for tube classification and regression.
Knowing Where to Look? Analysis on Attention of Visual Question Answering System
Attention mechanisms have been widely used in Visual Question Answering (VQA) solutions due to their capacity to model deep cross-domain interactions.
Towards Good Practices for Multi-modal Fusion in Large-scale Video Classification
Leveraging both visual frames and audio has been experimentally proven effective to improve large-scale video classification.
Temporal Action Localization by Structured Maximal Sums
We pose action localization as a structured prediction over arbitrary-length temporal windows, where each window is scored as the sum of frame-wise classification scores.
