Search Results for author:
Found 10 papers, 8 papers with code
Emu3: Next-Token Prediction is All You Need
While next-token prediction is considered a promising path towards artificial general intelligence, it has struggled to excel in multimodal tasks, which are still dominated by diffusion models (e. g., Stable Diffusion) and compositional approaches (e. g., CLIP combined with LLMs).
OmniGen: Unified Image Generation
In this work, we introduce OmniGen, a new diffusion model for unified image generation.
DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception
To facilitate the cutting-edge research of MLLMs on comprehensive vision perception, we thereby propose Perceptual Fusion, using a low-budget but highly effective caption engine for complete and accurate image descriptions.
Ranked #78 on
Visual Question Answering
on MM-Vet
Unveiling Encoder-Free Vision-Language Models
Training pure VLMs that accept the seamless vision and language inputs, i. e., without vision encoders, remains challenging and rarely explored.
Seeing Clearly, Answering Incorrectly: A Multimodal Robustness Benchmark for Evaluating MLLMs on Leading Questions
To enhance MLLMs' understanding capability and robustness, we further present a training set with paired positive and negative visual question-answer samples.
Efficient Multimodal Learning from Data-centric Perspective
Multimodal Large Language Models (MLLMs) have demonstrated notable capabilities in general visual understanding and reasoning tasks.
Universal Prompt Optimizer for Safe Text-to-Image Generation
To guide the optimizer to have the ability of converting toxic prompt to clean prompt while preserving semantic information, we design a novel reward function measuring toxicity and text alignment of generated images and train the optimizer through Proximal Policy Optimization.
Generative Multimodal Models are In-Context Learners
The human ability to easily solve multimodal tasks in context (i. e., with only a few demonstrations or simple instructions), is what current multimodal systems have largely struggled to imitate.
Ranked #3 on
Personalized Image Generation
on DreamBooth
Emu: Generative Pretraining in Multimodality
We present Emu, a Transformer-based multimodal foundation model, which can seamlessly generate images and texts in multimodal context.
Ranked #1 on
Visual Question Answering
on VizWiz
Fine-Grained Visual Prompting
Previous works have suggested that incorporating visual prompts, such as colorful boxes or circles, can improve the ability of models to recognize objects of interest.
