Search Results for author:
Found 64 papers, 23 papers with code
UniToken: Harmonizing Multimodal Understanding and Generation through Unified Visual Encoding
We introduce UniToken, an auto-regressive generation model that encodes visual inputs through a combination of discrete and continuous representations, enabling seamless integration of unified visual understanding and image generation tasks.
FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction
This work challenges the residual prediction paradigm in visual autoregressive modeling and presents FlexVAR, a new Flexible Visual AutoRegressive image generation paradigm.
Mitigating Hallucination for Large Vision Language Model by Inter-Modality Correlation Calibration Decoding
In this method, we design a Cross-Modal Value-Enhanced Decoding(CMVED) module to alleviate hallucination by a novel contrastive decoding mechanism.
CLIP-GS: Unifying Vision-Language Representation with 3D Gaussian Splatting
CLIP-GS leverages contrastive loss between 3DGS and the visual-text embeddings of CLIP, and we introduce an image voting loss to guide the directionality and convergence of gradient optimization.
DriveMM: All-in-One Large Multimodal Model for Autonomous Driving
Large Multimodal Models (LMMs) have demonstrated exceptional comprehension and interpretation capabilities in Autonomous Driving (AD) by incorporating large language models.
TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Ability
Rapid development of large language models (LLMs) has significantly advanced multimodal large language models (LMMs), particularly in vision-language tasks.
VidCompress: Memory-Enhanced Temporal Compression for Video Understanding in Large Language Models
Video-based multimodal large language models (Video-LLMs) possess significant potential for video understanding tasks.
MRStyle: A Unified Framework for Color Style Transfer with Multi-Modality Reference
For text reference, we align the text feature of stable diffusion priors with the style feature of our IRStyle to perform text-guided color style transfer (TRStyle).
Making Large Language Models Better Planners with Reasoning-Decision Alignment
Inspired by the knowledge-driven nature of human driving, recent approaches explore the potential of large language models (LLMs) to improve understanding and decision-making in traffic scenarios.
3D Weakly Supervised Semantic Segmentation with 2D Vision-Language Guidance
In this paper, we propose 3DSS-VLG, a weakly supervised approach for 3D Semantic Segmentation with 2D Vision-Language Guidance, an alternative approach that a 3D model predicts dense-embedding for each point which is co-embedded with both the aligned image and text spaces from the 2D vision-language model.
Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularization
Previous works primarily focus on prompt learning to adapt the CLIP into a variety of downstream tasks, however, suffering from task overfitting when fine-tuned on a small data set.
OV-DINO: Unified Open-Vocabulary Detection with Language-Aware Selective Fusion
To address these challenges, we propose a novel unified open-vocabulary detection method called OV-DINO, which is pre-trained on diverse large-scale datasets with language-aware selective fusion in a unified framework.
Ranked #5 on
Zero-Shot Object Detection
on MSCOCO
(AP metric, using extra
training data)
MindBench: A Comprehensive Benchmark for Mind Map Structure Recognition and Analysis
Multimodal Large Language Models (MLLM) have made significant progress in the field of document analysis.
Fewer Tokens and Fewer Videos: Extending Video Understanding Abilities in Large Vision-Language Models
Amidst the advancements in image-based Large Vision-Language Models (image-LVLM), the transition to video-based models (video-LVLM) is hindered by the limited availability of quality video data.
AlignSAM: Aligning Segment Anything Model to Open Context via Reinforcement Learning
In this paper, we propose a novel framework, termed AlignSAM, designed for automatic prompting for aligning SAM to an open context through reinforcement learning.
Matten: Video Generation with Mamba-Attention
In this paper, we introduce Matten, a cutting-edge latent diffusion model with Mamba-Attention architecture for video generation.
Lumen: Unleashing Versatile Vision-Centric Capabilities of Large Multimodal Models
To address this issue, we propose a novel LMM architecture named Lumen, a Large multimodal model with versatile vision-centric capability enhancement.
InstaGen: Enhancing Object Detection by Training on Synthetic Dataset
The grounding head is trained to align the text embedding of category names with the regional visual feature of the diffusion model, using supervision from an off-the-shelf object detector, and a novel self-training scheme on (novel) categories not covered by the detector.
LLaVA-MoLE: Sparse Mixture of LoRA Experts for Mitigating Data Conflicts in Instruction Finetuning MLLMs
To address this issue, we propose to apply an efficient Mixture of Experts (MoE) design, which is a sparse Mixture of LoRA Experts (MoLE) for instruction finetuning MLLMs.
Weakly-Supervised 3D Visual Grounding based on Visual Linguistic Alignment
During the inference stage, the learned text-3D correspondence will help us ground the text queries to the 3D target objects even without 2D images.
Instance-aware Multi-Camera 3D Object Detection with Structural Priors Mining and Self-Boosting Learning
Camera-based bird-eye-view (BEV) perception paradigm has made significant progress in the autonomous driving field.
UniDiff: Advancing Vision-Language Models with Generative and Discriminative Learning
Recent advances in vision-language pre-training have enabled machines to perform better in multimodal object discrimination (e. g., image-text semantic alignment) and image synthesis (e. g., text-to-image generation).
FastPillars: A Deployment-friendly Pillar-based 3D Detector
The deployment of 3D detectors strikes one of the major challenges in real-world self-driving scenarios.
Multiple Object Tracking Challenge Technical Report for Team MT_IoT
This is a brief technical report of our proposed method for Multiple-Object Tracking (MOT) Challenge in Complex Environments.
Ranked #10 on
Multi-Object Tracking
on DanceTrack
(using extra training data)
AeDet: Azimuth-invariant Multi-view 3D Object Detection
However, the typical convolution ignores the radial symmetry of the BEV features and increases the difficulty of the detector optimization.
Expansion and Shrinkage of Localization for Weakly-Supervised Semantic Segmentation
To tackle with this issue, this paper proposes an Expansion and Shrinkage scheme based on the offset learning in the deformable convolution, to sequentially improve the recall and precision of the located object in the two respective stages.
Weakly Supervised Semantic Segmentation via Progressive Patch Learning
"Progressive Patch Learning" further extends the feature destruction and patch learning to multi-level granularities in a progressive manner.
Weakly supervised Semantic Segmentation
Weakly-Supervised Semantic Segmentation
MSMDFusion: Fusing LiDAR and Camera at Multiple Scales with Multi-Depth Seeds for 3D Object Detection
Recent approaches aim at exploring the semantic densities of camera features through lifting points in 2D camera images (referred to as seeds) into 3D space, and then incorporate 2D semantics via cross-modal interaction or fusion techniques.
ARMANI: Part-level Garment-Text Alignment for Unified Cross-Modal Fashion Design
ARMANI discretizes an image into uniform tokens based on a learned cross-modal codebook in its first stage and uses a Transformer to model the distribution of image tokens for a real image given the tokens of the control signals in its second stage.
MT-Net Submission to the Waymo 3D Detection Leaderboard
In this technical report, we introduce our submission to the Waymo 3D Detection leaderboard.
PromptDet: Towards Open-vocabulary Detection using Uncurated Images
The goal of this work is to establish a scalable pipeline for expanding an object detector towards novel/unseen categories, using zero manual annotations.
Suspected Object Matters: Rethinking Model's Prediction for One-stage Visual Grounding
Recently, one-stage visual grounders attract high attention due to their comparable accuracy but significantly higher efficiency than two-stage grounders.
MORE: Multi-Order RElation Mining for Dense Captioning in 3D Scenes
3D dense captioning is a recently-proposed novel task, where point clouds contain more geometric information than the 2D counterpart.
Ranked #7 on
3D dense captioning
on ScanRefer Dataset
Two-stage Visual Cues Enhancement Network for Referring Image Segmentation
Referring Image Segmentation (RIS) aims at segmenting the target object from an image referred by one given natural language expression.
Delving into the Imbalance of Positive Proposals in Two-stage Object Detection
The first imbalance lies in the large number of low-quality RPN proposals, which makes the R-CNN module (i. e., post-classification layers) become highly biased towards the negative proposals in the early training stage.
MTL-NAS: Task-Agnostic Neural Architecture Search towards General-Purpose Multi-Task Learning
We propose to incorporate neural architecture search (NAS) into general-purpose multi-task learning (GP-MTL).
NMS by Representative Region: Towards Crowded Pedestrian Detection by Proposal Pairing
To acquire the visible parts, a novel Paired-Box Model (PBM) is proposed to simultaneously predict the full and visible boxes of a pedestrian.
PS-RCNN: Detecting Secondary Human Instances in a Crowd via Primary Object Suppression
PS-RCNN first detects slightly/none occluded objects by an R-CNN module (referred as P-RCNN), and then suppress the detected instances by human-shaped masks so that the features of heavily occluded instances can stand out.
Ranked #2 on
Object Detection
on WiderPerson
Central Similarity Quantization for Efficient Image and Video Retrieval
In this work, we propose a new \emph{global} similarity metric, termed as \emph{central similarity}, with which the hash codes of similar data pairs are encouraged to approach a common center and those for dissimilar pairs to converge to different centers, to improve hash learning efficiency and retrieval accuracy.
Real-Time Referring Expression Comprehension by Single-Stage Grounding Network
Experiments on RefCOCO, RefCOCO+, and RefCOCOg datasets demonstrate that our proposed SSG without relying on any region proposals can achieve comparable performance with other advanced models.
A Sufficient Condition for Convergences of Adam and RMSProp
Adam and RMSProp are two of the most influential adaptive stochastic algorithms for training deep neural networks, which have been pointed out to be divergent even in the convex setting via a few simple counterexamples.
Modeling Varying Camera-IMU Time Offset in Optimization-Based Visual-Inertial Odometry
Our approach is able to handle the rolling-shutter effects and imperfect sensor synchronization in a unified way.
Temporally Grounding Natural Sentence in Video
We introduce an effective and efficient method that grounds (i. e., localizes) natural sentences in long, untrimmed video sequences.
Joint Task-Recursive Learning for Semantic Segmentation and Depth Estimation
In this paper, we propose a novel joint Task-Recursive Learning (TRL) framework for the closing-loop semantic segmentation and monocular depth estimation tasks.
Ranked #88 on
Semantic Segmentation
on NYU Depth v2
A Unified Analysis of AdaGrad with Weighted Aggregation and Momentum Acceleration
Integrating adaptive learning rate and momentum techniques into SGD leads to a large class of efficiently accelerated adaptive stochastic algorithms, such as AdaGrad, RMSProp, Adam, AccAdaGrad, \textit{etc}.
Policy Optimization with Demonstrations
Exploration remains a significant challenge to reinforcement learning methods, especially in environments where reward signals are sparse.
Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi-Supervised Semantic Segmentation
Despite remarkable progress, weakly supervised segmentation methods are still inferior to their fully supervised counterparts.
Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentation
It can produce dense and reliable object localization maps and effectively benefit both weakly- and semi- supervised semantic segmentation.
Modular Generative Adversarial Networks
Existing methods for multi-domain image-to-image translation (or generation) attempt to directly map an input image (or a random vector) to an image in one of the output domains.
Left-Right Comparative Recurrent Model for Stereo Matching
Left-right consistency check is an effective way to enhance the disparity estimation by referring to the information from the opposite view.
Predicting Scene Parsing and Motion Dynamics in the Future
The ability of predicting the future is important for intelligent systems, e. g. autonomous vehicles and robots to plan early and make decisions accordingly.
Learning with Rethinking: Recurrently Improving Convolutional Neural Networks through Feedback
However, most of the existing CNN models only learn features through a feedforward structure and no feedback information from top to bottom layers is exploited to enable the networks to refine themselves.
FoveaNet: Perspective-aware Urban Scene Parsing
Thus, they suffer from heterogeneous object scales caused by perspective projection of cameras on actual scenes and inevitably encounter parsing failures on distant objects as well as other boundary and recognition errors.
Neural Person Search Machines
We investigate the problem of person search in the wild in this work.
Ranked #4 on
Person Re-Identification
on CUHK-SYSU
Video-based Person Re-identification with Accumulative Motion Context
Video based person re-identification plays a central role in realistic security and video surveillance.
Deep Self-Taught Learning for Weakly Supervised Object Localization
To overcome this issue, we propose a deep self-taught learning approach, which makes the detector learn the object-level features reliable for acquiring tight positive samples and afterwards re-train itself based on them.
Multi-View Image Generation from a Single-View
This paper addresses a challenging problem -- how to generate multi-view cloth images from only a single view input.
Video-based Person Re-identification with Accumulative Motion Context
Video based person re-identification plays a central role in realistic security and video surveillance.
Video Scene Parsing with Predictive Feature Learning
In this way, the network can effectively learn to capture video dynamics and temporal context, which are critical clues for video scene parsing, without requiring extra manual annotations.
Multi-Path Feedback Recurrent Neural Network for Scene Parsing
In this paper, we consider the scene parsing problem and propose a novel Multi-Path Feedback recurrent neural network (MPF-RNN) for parsing scene images.
Scale-aware Pixel-wise Object Proposal Networks
In particular, in order to improve the localization accuracy, a fully convolutional network is employed which predicts locations of object proposals for each pixel.
