Search Results for author:
Found 12 papers, 10 papers with code
UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generation
Although existing unified models achieve strong performance in vision-language understanding and text-to-image generation, they remain limited in addressing image perception and manipulation -- capabilities increasingly demanded in practical applications.
Ranked #1 on
Image Editing
on ImgEdit-Data
VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual intervention
Current video generation models excel at short clips but fail to produce cohesive multi-shot narratives due to disjointed visual dynamics and fractured storylines.
SwapAnyone: Consistent and Realistic Video Synthesis for Swapping Any Person into Any Video
Video body-swapping aims to replace the body in an existing video with a new body from arbitrary sources, which has garnered more attention in recent years.
Next Patch Prediction for Autoregressive Visual Generation
Autoregressive models, built based on the Next Token Prediction (NTP) paradigm, show great potential in developing a unified framework that integrates both language and vision tasks.
VideoGen-of-Thought: A Collaborative Framework for Multi-Shot Video Generation
Cross-Shot Consistency: We ensure temporal and identity consistency by leveraging identity-preserving (IP) embeddings across shots, which are automatically created from the narrative.
Open-Sora Plan: Open-Source Large Video Generation Model
We introduce Open-Sora Plan, an open-source project that aims to contribute a large generation model for generating desired high-resolution videos with long durations based on various user inputs.
Cycle3D: High-quality and Consistent Image-to-3D Generation via Generation-Reconstruction Cycle
Recent 3D large reconstruction models typically employ a two-stage process, including first generate multi-view images by a multi-view diffusion model, and then utilize a feed-forward model to reconstruct images to 3D content. However, multi-view diffusion models often produce low-quality and inconsistent images, adversely affecting the quality of the final 3D reconstruction.
Envision3D: One Image to 3D with Anchor Views Interpolation
To address this issue, we propose a novel cascade diffusion framework, which decomposes the challenging dense views generation task into two tractable stages, namely anchor views generation and anchor views interpolation.
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models
In this work, we propose a simple yet effective training strategy MoE-Tuning for LVLMs.
Ranked #150 on
Visual Question Answering
on MM-Vet
Repaint123: Fast and High-quality One Image to 3D Generation with Progressive Controllable 2D Repainting
The core idea is to combine the powerful image generation capability of the 2D diffusion model and the texture alignment ability of the repainting strategy for generating high-quality multi-view images with consistency.
LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment
We thus propose VIDAL-10M with Video, Infrared, Depth, Audio and their corresponding Language, naming as VIDAL-10M.
Ranked #1 on
Zero-shot Audio Classification
on VGG-Sound
(using extra training data)
Masked Autoencoders for Point Cloud Self-supervised Learning
Then, a standard Transformer based autoencoder, with an asymmetric design and a shifting mask tokens operation, learns high-level latent features from unmasked point patches, aiming to reconstruct the masked point patches.
Ranked #2 on
Point Cloud Segmentation
on PointCloud-C
3D Part Segmentation
Few-Shot 3D Point Cloud Classification
+2
