Search Results for author:
Found 12 papers, 10 papers with code
Perception Encoder: The best visual embeddings are not at the output of the network
Together with the core contrastive checkpoint, our PE family of models achieves state-of-the-art performance on a wide variety of tasks, including zero-shot image and video classification and retrieval; document, image, and video Q&A; and spatial tasks such as detection, depth estimation, and tracking.
PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding
In this paper, we study building a Perception Language Model (PLM) in a fully open and reproducible framework for transparent research in image and video understanding.
VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding
Building on the advances of language models, Large Multimodal Models (LMMs) have contributed significant improvements in video understanding.
Ranked #1 on
VCGBench-Diverse
on VideoInstruct
PALO: A Polyglot Large Multimodal Model for 5B People
PALO offers visual reasoning capabilities in 10 major languages, including English, Chinese, Hindi, Spanish, French, Arabic, Bengali, Russian, Urdu, and Japanese, that span a total of ~5B people (65% of the world population).
GLaMM: Pixel Grounding Large Multimodal Model
In this work, we present Grounding LMM (GLaMM), the first model that can generate natural language responses seamlessly intertwined with corresponding object segmentation masks.
Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models
Conversation agents fueled by Large Language Models (LLMs) are providing a new way to interact with visual data.
Ranked #5 on
Question Answering
on NExT-QA (Open-ended VideoQA)
VCGBench-Diverse
Video-based Generative Performance Benchmarking (Consistency)
+6
SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applications
Using our proposed efficient additive attention, we build a series of models called "SwiftFormer" which achieves state-of-the-art performance in terms of both accuracy and mobile inference speed.
UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentation
Owing to the success of transformer models, recent works study their applicability in 3D medical segmentation tasks.
Fine-tuned CLIP Models are Efficient Video Learners
Since training on a similar scale for videos is infeasible, recent approaches focus on the effective transfer of image-based CLIP to the video domain.
MaPLe: Multi-modal Prompt Learning
Pre-trained vision-language (V-L) models such as CLIP have shown excellent generalization ability to downstream tasks.
Ranked #3 on
Prompt Engineering
on Food-101
Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detection
Two popular forms of weak-supervision used in open-vocabulary detection (OVD) include pretrained CLIP model and image-level supervision.
Ranked #1 on
Open Vocabulary Object Detection
on OpenImages-v4
Class-agnostic Object Detection with Multi-modal Transformer
This has been a long-standing question in computer vision.
