Search Results for author:
Found 55 papers, 31 papers with code
T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation
In this work, we conduct the first systematic study on compositional text-to-video generation.
PredBench: Benchmarking Spatio-Temporal Prediction across Diverse Disciplines
In this paper, we introduce PredBench, a benchmark tailored for the holistic evaluation of spatio-temporal prediction networks.
GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editing
For a complex problem, the MLLM agent decomposes it into simpler sub-problems and constructs a tree structure to systematically plan the procedure of generation, editing, and self-correction with step-by-step verification.
Tailor3D: Customized 3D Assets Editing and Generation with Dual-Side Images
Afterward, it proceeds to edit the back views.
ScanReason: Empowering 3D Visual Grounding with Reasoning Capabilities
Although great progress has been made in 3D visual grounding, current models still rely on explicit textual descriptions for grounding and lack the ability to reason human intentions from implicit instructions.
BEACON: Benchmark for Comprehensive RNA Tasks and Language Models
RNA plays a pivotal role in translating genetic instructions into functional outcomes, underscoring its importance in biological processes and disease mechanisms.
4Diffusion: Multi-view Video Diffusion Model for 4D Generation
We first design a unified diffusion model tailored for multi-view video generation by incorporating a learnable motion module into a frozen 3D-aware diffusion model to capture multi-view spatial-temporal correlations.
DiM: Diffusion Mamba for Efficient High-Resolution Image Synthesis
In addition, to further improve training efficiency for high-resolution image generation with DiM, we investigate "weak-to-strong" training strategy that pretrains DiM on low-resolution images ($256\times 256$) and then finetune it on high-resolution images ($512 \times 512$).
TC4D: Trajectory-Conditioned Text-to-4D Generation
We learn local deformations that conform to the global trajectory using supervision from a text-to-video model.
Editing Massive Concepts in Text-to-Image Diffusion Models
We further propose a comprehensive benchmark, named ImageNet Concept Editing Benchmark (ICEB), for evaluating massive concept editing for T2I models with two subtasks, free-form prompts, massive concept categories, and extensive evaluation metrics.
Divide and Conquer: Language Models can Plan and Self-Correct for Compositional Text-to-Image Generation
Given a complex text prompt containing multiple concepts including objects, attributes, and relationships, the LLM agent initially decomposes it, which entails the extraction of individual objects, their associated attributes, and the prediction of a coherent scene layout.
Point Transformer V3: Simpler Faster Stronger
This paper is not motivated to seek innovation within the attention mechanism.
EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI
In the realm of computer vision and robotics, embodied agents are expected to explore their environment and carry out human instructions.
A Survey of Reasoning with Foundation Models
Reasoning, a crucial ability for complex problem-solving, plays a pivotal role in various real-world settings such as negotiation, medical diagnosis, and criminal investigation.
Point Transformer V3: Simpler, Faster, Stronger
This paper is not motivated to seek innovation within the attention mechanism.
Ranked #1 on
3D Semantic Segmentation
on ScanNet++
(using extra training data)
EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planning
The pursuit of artificial general intelligence (AGI) has been accelerated by Multimodal Large Language Models (MLLMs), which exhibit superior reasoning, generalization capabilities, and proficiency in processing multimodal inputs.
CorresNeRF: Image Correspondence Priors for Neural Radiance Fields
We present CorresNeRF, a novel method that leverages image correspondence priors computed by off-the-shelf methods to supervise NeRF training.
DreamComposer: Controllable 3D Object Generation via Multi-View Conditions
However, due to the lack of information from multiple views, these works encounter difficulties in generating controllable novel views.
Drag-A-Video: Non-rigid Video Editing with Point-based Interaction
In this paper, we propose a new diffusion-based method for interactive point-based video manipulation, called Drag-A-Video.
HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting
In this paper, we propose an efficient yet effective framework, HumanGaussian, that generates high-quality 3D humans with fine-grained geometry and realistic appearance.
HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusion
Our model enforces the joint learning of image appearance, spatial relationship, and geometry in a unified network, where each branch in the model complements to each other with both structural awareness and textural richness.
OV-PARTS: Towards Open-Vocabulary Part Segmentation
Secondly, part segmentation introduces an open granularity challenge due to the diverse and often ambiguous definitions of parts in the open world.
Open Vocabulary Semantic Segmentation
Open-Vocabulary Semantic Segmentation
+1
Understanding Masked Autoencoders From a Local Contrastive Perspective
Masked AutoEncoder (MAE) has revolutionized the field of self-supervised learning with its simple yet effective masking and reconstruction strategies.
Object2Scene: Putting Objects in Context for Open-Vocabulary 3D Detection
Instead of leveraging 2D images, we propose Object2Scene, the first approach that leverages large-scale large-vocabulary 3D object datasets to augment existing 3D scene datasets for open-vocabulary 3D object detection.
Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Training
In contrast, such privilege has not yet fully benefited 3D deep learning, mainly due to the limited availability of large-scale 3D datasets.
Ranked #3 on
3D Semantic Segmentation
on SemanticKITTI
(val mIoU metric, using extra
training data)
T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional Text-to-image Generation
Despite the stunning ability to generate high-quality images by recent text-to-image models, current approaches often struggle to effectively compose objects with different attributes and relationships into a complex and coherent scene.
UniG3D: A Unified 3D Object Generation Dataset
However, the quality and diversity of existing 3D object generation methods are constrained by the inadequacies of existing 3D object datasets, including issues related to text quality, the incompleteness of multi-modal data representation encompassing 2D rendered images and 3D assets, as well as the size of the dataset.
SAM3D: Segment Anything in 3D Scenes
In this work, we propose SAM3D, a novel framework that is able to predict masks in 3D point clouds by leveraging the Segment-Anything Model (SAM) in RGB images without further training or finetuning.
TVTSv2: Learning Out-of-the-box Spatiotemporal Visual Representations at Scale
We argue that tuning a text encoder end-to-end, as done in previous work, is suboptimal since it may overfit in terms of styles, thereby losing its original generalization ability to capture the semantics of various language registers.
Hierarchical Diffusion Autoencoders and Disentangled Image Manipulation
To mitigate those limitations, we propose Hierarchical Diffusion Autoencoders (HDAE) that exploit the fine-grained-to-abstract and lowlevel-to-high-level feature hierarchy for the latent space of diffusion models.
RIFormer: Keep Your Vision Backbone Effective While Removing Token Mixer
Extensive experiments and ablative analysis also demonstrate that the inductive bias of network architecture, can be incorporated into simple network structure with appropriate optimization strategy.
DDP: Diffusion Model for Dense Visual Prediction
We propose a simple, efficient, yet powerful framework for dense visual predictions based on the conditional diffusion pipeline.
Ranked #3 on
Monocular Depth Estimation
on SUN-RGBD
Masked Scene Contrast: A Scalable Framework for Unsupervised 3D Representation Learning
As a pioneering work, PointContrast conducts unsupervised 3D representation learning via leveraging contrastive learning over raw RGB-D frames and proves its effectiveness on various downstream tasks.
Ranked #13 on
Semantic Segmentation
on ScanNet
(using extra training data)
RIFormer: Keep Your Vision Backbone Effective but Removing Token Mixer
Extensive experiments and ablative analysis also demonstrate that the inductive bias of network architecture, can be incorporated into simple network structure with appropriate optimization strategy.
Back to the Source: Diffusion-Driven Adaptation To Test-Time Corruption
We update the target data instead, and project all test inputs toward the source domain with a generative diffusion model.
GLeaD: Improving GANs with A Generator-Leading Task
Generative adversarial network (GAN) is formulated as a two-player game between a generator (G) and a discriminator (D), where D is asked to differentiate whether an image comes from real data or is produced by G. Under such a formulation, D plays as the rule maker and hence tends to dominate the competition.
Shape-Guided Diffusion with Inside-Outside Attention
We introduce precise object silhouette as a new form of user control in text-to-image diffusion models, which we dub Shape-Guided Diffusion.
Point Transformer V2: Grouped Vector Attention and Partition-based Pooling
In this work, we analyze the limitations of the Point Transformer and propose our powerful and efficient Point Transformer V2 model with novel designs that overcome the limitations of previous work.
Ranked #1 on
3D Semantic Segmentation
on nuScenes
Learning Transferable Spatiotemporal Representations from Natural Script Knowledge
Pre-training on large-scale video data has become a common recipe for learning transferable spatiotemporal representations in recent years.
Back to the Source: Diffusion-Driven Test-Time Adaptation
We instead update the target data, by projecting all test inputs toward the source domain with a generative diffusion model.
The ArtBench Dataset: Benchmarking Generative Models with Artworks
We introduce ArtBench-10, the first class-balanced, high-quality, cleanly annotated, and standardized dataset for benchmarking artwork generation.
MILES: Visual BERT Pre-training with Injected Language Semantics for Video-text Retrieval
Dominant pre-training work for video-text retrieval mainly adopt the "dual-encoder" architectures to enable efficient retrieval, where two separate encoders are used to contrast global video and text representations, but ignore detailed local semantics.
Ranked #7 on
Zero-Shot Video Retrieval
on MSVD
Bridging Video-text Retrieval with Multiple Choice Questions
As an additional benefit, our method achieves competitive results with much shorter pre-training videos on single-modality downstream tasks, e. g., action recognition with linear evaluation.
Ranked #8 on
Zero-Shot Video Retrieval
on MSVD
More Control for Free! Image Synthesis with Semantic Diffusion Guidance
We investigate fine-grained, continuous control of this model class, and introduce a novel unified framework for semantic diffusion guidance, which allows either language or image guidance, or both.
Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions
We propose a novel algorithm, named Open-Edit, which is the first attempt on open-domain image manipulation with open-vocabulary instructions.
Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis
Semantic image synthesis aims at generating photorealistic images from semantic layouts.
CAMP: Cross-Modal Adaptive Message Passing for Text-Image Retrieval
Text-image cross-modal retrieval is a challenging task in the field of language and vision.
Ranked #9 on
Image Retrieval
on Flickr30K 1K test
Improving Referring Expression Grounding with Cross-modal Attention-guided Erasing
Referring expression grounding aims at locating certain objects or persons in an image with a referring expression, where the key challenge is to comprehend and align various types of information from visual and textual domain, such as visual attributes, location and interactions with surrounding regions.
Localization Guided Learning for Pedestrian Attribute Recognition
Pedestrian attribute recognition has attracted many attentions due to its wide applications in scene understanding and person analysis from surveillance videos.
Improving Deep Visual Representation for Person Re-identification by Global and Local Image-language Association
Person re-identification is an important task that requires learning discriminative visual features for distinguishing different person identities.
Ranked #24 on
Text based Person Retrieval
on CUHK-PEDES
Show, Tell and Discriminate: Image Captioning by Self-retrieval with Partially Labeled Data
The aim of image captioning is to generate captions by machine to describe image contents.
Orientation Invariant Feature Embedding and Spatial Temporal Regularization for Vehicle Re-Identification
In our vehicle ReID framework, an orientation invariant feature embedding module and a spatial-temporal regularization module are proposed.
HydraPlus-Net: Attentive Deep Features for Pedestrian Analysis
Pedestrian analysis plays a vital role in intelligent video surveillance and is a key component for security-centric computer vision systems.
Ranked #2 on
Pedestrian Attribute Recognition
on RAP
Object Detection in Videos with Tubelet Proposal Networks
Object detection in videos has drawn increasing attention recently with the introduction of the large-scale ImageNet VID dataset.
