Search Results for author:
Found 8 papers, 1 papers with code
FastVLM: Efficient Vision Encoding for Vision Language Models
At different operational resolutions, the vision encoder of a VLM can be optimized along two axes: reducing encoding latency and minimizing the number of visual tokens passed to the LLM, thereby lowering overall latency.
Revisit Large-Scale Image-Caption Data in Pre-training Multimodal Foundation Models
Our findings reveal that a hybrid approach that keeps both synthetic captions and AltTexts can outperform the use of synthetic captions alone, improving both alignment and performance, with each model demonstrating preferences for particular caption formats.
MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning
We present MM1. 5, a new family of multimodal large language models (MLLMs) designed to enhance capabilities in text-rich image understanding, visual referring and grounding, and multi-image reasoning.
Ranked #55 on
Visual Question Answering
on MM-Vet
Understanding Alignment in Multimodal LLMs: A Comprehensive Study
A primary objective of alignment for MLLMs is to encourage these models to align responses more closely with image information.
MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs
We introduce MIA-Bench, a new benchmark designed to evaluate multimodal large language models (MLLMs) on their ability to strictly adhere to complex instructions.
MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training
Through careful and comprehensive ablations of the image encoder, the vision language connector, and various pre-training data choices, we identified several crucial design lessons.
Ranked #73 on
Visual Question Answering
on MM-Vet
Noise Robust Named Entity Understanding for Voice Assistants
Named Entity Recognition (NER) and Entity Linking (EL) play an essential role in voice assistant interaction, but are challenging due to the special difficulties associated with spoken user queries.
