FastVLM: Efficient Vision Encoding for Vision Language Models

1 code implementation • 17 Dec 2024 • Pavan Kumar Anasosalu Vasu, Fartash Faghri, Chun-Liang Li, Cem Koc, Nate True, Albert Antony, Gokul Santhanam, James Gabriel, Peter Grasch, Oncel Tuzel, Hadi Pouransari

At different operational resolutions, the vision encoder of a VLM can be optimized along two axes: reducing encoding latency and minimizing the number of visual tokens passed to the LLM, thereby lowering overall latency.

Dataset Decomposition: Faster LLM Training with Variable Sequence Length Curriculum

1 code implementation • 21 May 2024 • Hadi Pouransari, Chun-Liang Li, Jen-Hao Rick Chang, Pavan Kumar Anasosalu Vasu, Cem Koc, Vaishaal Shankar, Oncel Tuzel

During training, we use variable sequence length and batch-size, sampling simultaneously from all buckets with a curriculum.

2k 8k +1