Search Results for author:
Found 15 papers, 2 papers with code
ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinement
We present ILLUME+ that leverages dual visual tokenization and a diffusion decoder to improve both deep semantic understanding and high-fidelity image generation.
SemHiTok: A Unified Image Tokenizer via Semantic-Guided Hierarchical Codebook for Multimodal Understanding and Generation
We present SemHiTok, a unified image Tokenizer via Semantic-Guided Hierarchical codebook that provides consistent discrete feature representations for multimodal understanding and generation tasks.
FreqPrior: Improving Video Diffusion Models with Frequency Filtering Gaussian Noise
Text-driven video generation has advanced significantly due to developments in diffusion models.
Brick-Diffusion: Generating Long Videos with Brick-to-Wall Denoising
However, training models for long video generation demands significant computational power and extensive data, leading most video diffusion models to be limited to a small number of frames.
ILLUME: Illuminating Your LLMs to See, Draw, and Self-Enhance
In this paper, we introduce ILLUME, a unified multimodal large language model (MLLM) that seamlessly integrates multimodal understanding and generation capabilities within a single large language model through a unified next-token prediction formulation.
Ranked #140 on
Visual Question Answering
on MM-Vet
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions
GPT-4o, an omni-modal model that enables vocal conversations with diverse emotions and tones, marks a milestone for omni-modal foundation models.
UNIT: Unifying Image and Text Recognition in One Vision Encoder
Starting with a vision encoder pre-trained with image recognition tasks, UNIT introduces a lightweight language decoder for predicting text outputs and a lightweight vision decoder to prevent catastrophic forgetting of the original image encoding capabilities.
HiRes-LLaVA: Restoring Fragmentation Input in High-Resolution Large Vision-Language Models
High-resolution inputs enable Large Vision-Language Models (LVLMs) to discern finer visual details, enhancing their comprehension capabilities.
From Summary to Action: Enhancing Large Language Models for Complex Tasks with Open World APIs
In this work, we introduce a novel tool invocation pipeline designed to control massive real-world APIs.
Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving
Large vision-language models (VLMs) have garnered increasing interest in autonomous driving areas, due to their advanced capabilities in complex reasoning tasks essential for highly autonomous vehicle behavior.
Gaining Wisdom from Setbacks: Aligning Large Language Models via Mistake Analysis
The rapid development of large language models (LLMs) has not only provided numerous opportunities but also presented significant challenges.
PARTNER: Level up the Polar Representation for LiDAR 3D Object Detection
Recently, polar-based representation has shown promising properties in perceptual tasks.
LIFT: Learning 4D LiDAR Image Fusion Transformer for 3D Object Detection
LiDAR and camera are two common sensors to collect data in time for 3D object detection under the autonomous driving context.
Learning Transferable Features for Point Cloud Detection via 3D Contrastive Co-training
First, 3D-CoCo is inspired by our observation that the bird-eye-view (BEV) features are more transferable than low-level geometry features.
PointAugmenting: Cross-Modal Augmentation for 3D Object Detection
On one hand, PointAugmenting decorates point clouds with corresponding point-wise CNN features extracted by pretrained 2D detection models, and then performs 3D object detection over the decorated point clouds.
